If you’re thinking about designing an ideal data structure for your Internet of Things application, then here’s what you should do: don’t do it.
As it turns out, the Internet of Things requires a huge deal of flexibility. Why? Because there are millions -if not billions- of heterogeneous objects that will begin interacting with each other in ways we can’t predict. The structured and rigid tables offered by traditional databases won’t help us because they require a pre-defined set of properties and tables, which again, we can’t predict.
To understand why this is important, let’s take an example of a water management application.
The Problem
Let’s say we want to measure water levels in a large number of wells. A simplified data architecture for this application would look like this:
This looks just fine and should work perfectly using a relational database. But then, 2 years after the system has been up and running, someone has an idea:
“Hey, now that we bought these new Internet-enabled diesel generators to power the water pumps, let’s see their live data!“.
To make this change, we would have to add a new table called “Power Plants” and a new column to the table “Wells”, marked in red below:
But in a relational database, this change would require us to re-write all the lines of the table so that they include a value in the new column. With two years of historical data, this could take a couple of hours, blocking the database as it happens.
Can you imagine this procedure with every future change? How does it impact the system’s stability, uptime and technical staff requirements?
The Solution
An interesting way of handling IoT data is the document-oriented approach. Instead of fixed tables, columns, and rows, you have documents describing each object. You can think of it as thousands of papers (documents) lying around the floor, instead of having them perfectly organized in different cabinet’s folders.
As messy as it might sound, these documents do not have a predefined schema like relational databases, which is perfect for today’s and future IoT applications. This also helps spread the data across multiple servers, making it more cloud-friendly.
Getting back to our example, the data about each Well is now stored in a document and not in a table:
As you can see, we merged all the properties of the well into a single document, regardless of the relationships between the well (water level, precipitation, humidity), its environment (city, country) and the power generator (oil pressure, energy usage, etc.).
This simplification allows for any future change of the document properties.
How to do it in the real world?
At Ubidots, it’s been our priority to offer a flexible data-schema and put it at our users’ fingertips through a RESTful API (and we proudly use MongoDB for it :).
We have a basic schema that is compatible with most IoT applications:
- Data Sources that contain Variables
- Variables that contain Values
- Values are time-stamped data points containing the sensor information.
But users can also specify additional attributes and properties to fit their needs. In our example, this could be additional information about each data source, variable or value:
It’s worth noting that the last ones (values) are stored over time, which means you get to keep their historical traces.
Check out our documentation in order to learn how to add such context to your data:
- Create a data source
- Create a variable within a data source
- Create a value within a variable
- Sending the ISS’s Geo data to Ubidots
Wrapping it up
There’s a wide range of IoT applications, each requiring different data schemas. In this article, we reviewed document-oriented databases as a good alternative for future IoT developments and how Ubidots can support them.
Have an IoT application in mind? Sign up and build your own applications today!
