Uploading Time Series Data¶
Data Source Configuration¶
The Open Edge Device Kit uses a data source configuration for uploading time series data to MindSphere properly. The data source configuration is defined in a file named data.cfg
with the format shown below.
Data Source Configuration Sample
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | { "dataConfiguration": { "agentId": "f5f6c04336fa4116a6417b8b1eef7495", "configurationId": "1", "uploadCycle": "10", "description": null, "dataSources": [{ "name": "my_data_source_1", "dataSourceId": "12cs3", "description": "my_descripton", "protocol": "my_protocol", "readCycleInSeconds": "5", "protocolData": { "myField1": "", "myField2": "", "myField3": "" }, "dataPoints": [{ "dataPointId": "0000", "name": "my_variable1", "description": "my_descripton_1", "unit": "PERCENT", "dataType": "INT", "dataPointData": { "myField1": "", "myField2": "" } }, { "dataPointId": "0001", "name": "my_variable_2", "description": "my_descripton_2", "unit": "PERCENT", "dataType": "LONG", "dataPointData": { "myField1": "", "myField2": "" } }] }] } } |
For each data source, the Open Edge Device Kit publishes the data source configuration message after successfully applying the configuration. This message should trigger the data reader modules on the device to publish time series data.
Time Series Data Upload¶
After the data source configuration has been applied, the Open Edge Device Kit is ready to upload time series data to MindSphere. It expects the data to be published by data readers using the following topics for data upload:
runtime/inject/data/timeseries/{protocol}/{data_source_id}
runtime/inject/diag/timeseries/{protocol}/{data_source_id}
Services receive the {data_source_id}
if they are subscribed to cloud/monitoring/update/configuration/{protocol}
.
Publishing Data¶
-
Establish a connection to the MQTT broker where
<mqtt_host>
is the host name and<mqtt_port>
is the port:1 2 3
String broker = String.format("tcp://%s:%d", <mqtt_host>, <mqtt_port>); MqttClient mqttClient = new MqttClient(broker, "MQTT_client_id"); mqttClient.connect();
-
Prepare the data in JSON format according to the templates below. Placeholders are indicated by angular brackets
<>
.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
String jsonTemplate = "[\n" + " {\n" + " \"timestamp\": \"<time_stamp>\",\n" + " \"values\": [\n" + " {\n" + " \"dataPointId\": \"<data_point_id>\",\n" + " \"value\": \"<data_value>\",\n" + " \"qualityCode\": \"<quality_code>\"\n" + " }\n" + " ]\n" + " }\n" + "]"; jsonTemplate = jsonTemplate.replace("<time_stamp>", data.getTimestamp()); jsonTemplate = jsonTemplate.replace("<data_point_id>", data.getDataPointID()); jsonTemplate = jsonTemplate.replace("<data_value>", data.getDataValue()); jsonTemplate = jsonTemplate.replace("<quality_code>", data.getQualityCode());
-
Publish the data to the Open Edge Device Kit according to the templates below. Placeholders are indicated by angular brackets
<>
.1 2 3 4 5 6
String topicTemplate = "runtime/inject/data/timeseries/<protocol>/<data_source_id>"; topicTemplate = topicTemplate.replace("<protocol>", dataTimeSeriesProtocol); topicTemplate = topicTemplate.replace("<data_source_id>", dataTimeSeriesSource); mqttClient.publish(topicTemplate, jsonTemplate.getBytes());
Info
All data readers should subscribe to the topic below using their respective <protocol>
and <data_source_id>
parameters. When they receive this message, they shall stop all activities.
1 | runtime/data/timeseries/stop/<protocol>/<data_source_id> |
Caching Data¶
The Open Edge Device Kit provides a caching mechanism for time series data upload to prevent data loss. The received time series data is encrypted and cached on local file system, if the MindConnect Exchange endpoint is not reachable. The stored data is uploaded when the endpoint is reachable again.
Attention
The cache size is limited and incoming data will start to overwrite the old cached data after a while. The cache size can be adjusted using the parameter offlineCache
> size
in the settings file.
Any questions left?
Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.