Skip to content

Uploading Time Series Data

Data Source Configuration

The Open Edge Device Kit uses a data source configuration for uploading time series data to MindSphere properly. The data source configuration is defined in a file named data.cfg with the format shown below.

Data Source Configuration Sample 
{
    "dataConfiguration": {
        "agentId": "f5f6c04336fa4116a6417b8b1eef7495",
        "configurationId": "1",
        "uploadCycle": "10",
        "description": null,
        "dataSources": [{
            "name": "my_data_source_1",
            "dataSourceId": "12cs3",
            "description": "my_descripton",
            "protocol": "my_protocol",
            "readCycleInSeconds": "5",
            "protocolData": {
                "myField1": "",
                "myField2": "",
                "myField3": ""
            },
            "dataPoints": [{
                "dataPointId": "0000",
                "name": "my_variable1",
                "description": "my_descripton_1",
                "unit": "PERCENT",
                "dataType": "INT",
                "dataPointData": {
                    "myField1": "",
                    "myField2": ""
                }
            },
            {
                "dataPointId": "0001",
                "name": "my_variable_2",
                "description": "my_descripton_2",
                "unit": "PERCENT",
                "dataType": "LONG",
                "dataPointData": {
                    "myField1": "",
                    "myField2": ""
                }
            }]
        }]
    }
}

For each data source, the Open Edge Device Kit publishes the data source configuration message after successfully applying the configuration. This message should trigger the data reader modules on the device to publish time series data.

Time Series Data Upload

After the data source configuration has been applied, the Open Edge Device Kit is ready to upload time series data to MindSphere. It expects the data to be published by data readers using the following topics for data upload:

Services receive the {data_source_id} if they are subscribed to cloud/monitoring/update/configuration/{protocol}.

Publishing Data

  1. Establish a connection to the MQTT broker where <mqtt_host> is the host name and <mqtt_port> is the port:

    String broker = String.format("tcp://%s:%d", <mqtt_host>, <mqtt_port>);
MqttClient mqttClient = new MqttClient(broker, "MQTT_client_id");
mqttClient.connect();

  2. Prepare the data in JSON format according to the templates below. Placeholders are indicated by angular brackets <>.

    String jsonTemplate = "[\n" +
    "  {\n" +
    "    \"timestamp\": \"<time_stamp>\",\n" +
    "    \"values\": [\n" +
    "      {\n" +
    "        \"dataPointId\": \"<data_point_id>\",\n" +
    "        \"value\": \"<data_value>\",\n" +
    "        \"qualityCode\": \"<quality_code>\"\n" +
    "      }\n" +
    "    ]\n" +
    "  }\n" +
    "]";

jsonTemplate = jsonTemplate.replace("<time_stamp>", data.getTimestamp());
jsonTemplate = jsonTemplate.replace("<data_point_id>", data.getDataPointID());
jsonTemplate = jsonTemplate.replace("<data_value>", data.getDataValue());
jsonTemplate = jsonTemplate.replace("<quality_code>", data.getQualityCode());

  1. Publish the data to the Open Edge Device Kit according to the templates below. Placeholders are indicated by angular brackets <>.

    String topicTemplate = "runtime/inject/data/timeseries/<protocol>/<data_source_id>";

topicTemplate = topicTemplate.replace("<protocol>", dataTimeSeriesProtocol);
topicTemplate = topicTemplate.replace("<data_source_id>", dataTimeSeriesSource);

mqttClient.publish(topicTemplate, jsonTemplate.getBytes());

Info

All data readers should subscribe to the topic below using their respective <protocol> and <data_source_id> parameters. When they receive this message, they shall stop all activities.

runtime/data/timeseries/stop/<protocol>/<data_source_id>

Caching Data

The Open Edge Device Kit provides a caching mechanism for time series data upload to prevent data loss. The received time series data is encrypted and cached on local file system, if the MindConnect Exchange endpoint is not reachable. The stored data is uploaded when the endpoint is reachable again.

Attention

The cache size is limited and incoming data will start to overwrite the old cached data after a while. The cache size can be adjusted using the parameter offlineCache > size in the settings file.

Any questions left?

Ask the community

Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.

Last update: December 16, 2021