Skip to content

IoT Time Series Aggregates Service

Idea

The IoT Time Series (TS) Aggregates Service is used to read aggregated interval data that is calculated based on raw time series data. The IoT Data Service automatically creates aggregated summaries of all numeric time series data. This allows applications to retrieve smaller data sets that cover a long time range with much better performance than processing all the raw time series data. For example, an aspect could create new data every second, which adds up to ~2.5 million records per month. An application could use the IoT TS Aggregates Service to a summary for each day of the month, obtaining only 30 records.

Access

For accessing this service you need to have the respective roles listed in IoT Time Series Aggregates Service roles and scopes.

A user can only read data within their tenant and subtenants.

Basics

Aggregated Data

The data summarized in an aggregate is the data with timestamps greater than the start time and less than or equal to the end time of the interval. This ensures that count type data is correctly reflected in the aggregate. For example, if a device provides data at the end of every five minutes, the hourly aggregation includes the data with timestamps from five minutes after the hour until the end of the hour (e.g., 3:05 to 4:00).

Each aggregate summarizes the following information:

  • first value and time of first value of the interval
  • last value and time of last value of the interval
  • minimum value and time of the minimum value during the interval
  • maximum value and time of the maximum value during the interval
  • the count of the values during the interval
  • the sum of the values during the interval
  • the average of the values during the interval
  • the count of the good, uncertain, and bad values during the interval

Pre-Calculated Aggregates

The IoT TS Aggregates Service automatically creates pre-calculated aggregates and stores them for later retrieval to enhance performance of aggregate queries. Aggregates of the following interval lengths are pre-calculated:

  • 2 minutes
    Aggregates end at two minutes after the hour, four minutes after the hour, and so on. They are created within two minutes after the two-minute interval has ended.
  • 1 hour
    Aggregates end at the top of the hour. They are created within eight minutes after the end of the hour.
  • 1 day
    Aggregates end at midnight in the asset's time zone. They are created within fifteen minutes after midnight in UTC time.

The aggregated data is wall clock aligned based on the asset's time zone.

Note

Aggregated data is not immediately available. E.g., if a request for aggregated data with interval length of 1 hour is made at 10:00, the response will not contain the aggregate from 9:00 to 10:00, since it has not been calculated yet.

On Demand Aggregates

If a client requests aggregated data, which cannot be calculated using pre-calculated aggregates (e.g. interval length of 1 minute or starting at 10:01), the service calculates the missing items on demand.
The IoT TS Aggregates Service supports the following interval lengths:

  • 1-120 seconds
  • 1-60 minutes
  • 1-24 hours
  • 1+ days
  • 1+ weeks
  • 1+ months

Aggregate Creation

Data Availability

Aggregated data is only created if time series data exists for that time range. Aggregates are created based on the data that is available at the time. Data that arrives late is not included in the initial aggregated results, but triggers a recalculation after a delay.

Quality Code Data

If a variable is configured to contain quality code data, the service uses it to rate the aggregates. The quality code is compared to the good and uncertain thresholds defined in the tenant (by default the values are set according to the OPC UA standard).

Good data is defined as anything less than or equal to the goodThreshold value.
Uncertain data is defined as anything greater than the goodThreshold value and less than or equal to the uncertainThreshold value.
Bad data is defined as anything greater than the uncertainThreshold.

Only good and uncertain data is used to calculate the summary values listed above. Bad data is ignored for these calculations. Each aggregate contains counts of the three types of data: countgood, countuncertain, and countbad. If no quality code data is available for a time series record, it is considered good.

Requesting Aggregates

Ideal Interval Length

For best performance, choose an interval length, which is an exact multiple of a pre-calculated aggregation interval, such as 10 minutes, 2 hours, 5 days, 3 weeks, or 2 months. Requests for other interval lengths take longer to process, because they require using raw time series and pre-calculated aggregations to calculate results.

Start Time Precision

The requested start and end times cannot be defined with higher precision than the requested interval length. The total time range must be a multiple of the requested interval length. For example, assume a request for data aggregated into intervals of 5 minutes. There, the start time must be defined with minute precision and the total time range must be a multiple of 5 minutes. When requesting data aggregated into intervals of 10 seconds, the start time must be set with second precision and the total time range must be a multiple of 10 seconds.

Interval Precision

The supported interval lengths overlap, which provides flexibility to the application. For example, 1 hour and 60 minutes represent the same length, but allow different behavior. A request for aggregation intervals of 1 hour can only be used for time ranges which start and end at the top of the hour, like 5:00 – 10:00. In contrast, a request for aggregation intervals of 60 minutes can also be used for time ranges like 5:30 – 10:30, which would otherwise be rejected. Accordingly, if the tenant is configured with the first day of the week being Monday, a time range of 1 week must start and end on a Monday. A time range of 7 days could start on any day of the week. For more information refer to the Samples.

Parameter Selection

There is an optional select parameter that defines which data to return. For more information refer to the Selecting documentation

Features

The IoT TS Aggregates Service exposes its API for realizing the following tasks:

  • Read aggregated interval data using pre-calculated aggregates and on demand aggregation
  • Specify the time range
  • Specify the interval length
  • Select only the summary information you are interested in

Limitations

  • On demand aggregates can only be created from up to 5,000 raw time series entries
  • A query can only return up to 200 aggregate intervals in one response
  • This service does not support time zones that are 15 or 45 minutes off a UTC hour, such as Nepal standard time (UTC+05:45). Time zones that are 30 minutes off a UTC hour are supported, such as India (UTC+05:30). Time zones that are an integer number of hours off GMT are supported.

To get the current list of restrictions go to release notes and choose the latest date. From there go to "MindAccess Developer Plan Subscribers and MindAccess Operator Plan Subscribers" and pick the IoT service you are interested in.

Example Scenario

A wind turbine produces time series data for the speed of the blades. The data is sent continuously to MindSphere.

The IoT TS Aggregates Service API allows you to collect aggregated data for the speed of the windmill.

Any questions left?

Ask the community


Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.