Skip to content

Data Exchange Service

Idea

The Data Exchange (DE) Service allows customers to upload and download data for any purpose. It is intended for both regular users and for tools that upload data into MindSphere.

Access

The Data Exchange provides its operations as a REST API. Uploading, downloading, organizing and listing folder structure can be done using API calls.

For accessing this service you need to have one of the roles listed in Data Exchange roles and scopes.

Basics

Customers and other services can use DE as a file storage and management provider. The maximum supported file size is 5TB under multipart upload and download. Note that when receiving requests through MindGate, MindGate applies its own limits in addition. This includes the timeouts and request sizes.

DE does not cache, analyze or scan the contents of the transferred bytes for viruses. It uses the Spring Cloud Resource Manager along with streaming mechanisms that provide multithreaded support for working with S3 storage. This type of backend communication was implemented to ensure maximum performance under minimum resource load.

Visibility spaces

By default, when uploading a file, the newly uploaded file inherits its parent's visibility. It can be either be public or private. Public visibility means that the file is visible in tenant space. Private visibility means that the file is visible in user space only, meaning that only the user that has uploaded it can also download, delete or update it.

Public space

Public space is accessible by all users within the tenant. Files and directories are made public, if they are uploaded or created in public directories. If the file or directory needs to be placed in root, i.e., they have no parent, the parentId attribute must be set to _PUBLIC_ROOT_ID.

Private space

Private space is accessible by the respective author only. Files and directories are made private, if they are uploaded or created in private directories. If the file or directory needs to be placed in root, i.e., they have no parent, the parentId attribute need to be set to _PRIVATE_ROOT_ID.

Features

The Data Exchange Service exposes its API for realizing the following tasks:

  • Secure upload testbed or production data
  • Download data from MindSphere that you have previously uploaded
  • Download results produced by other MindSphere services or apps
  • Have a performant long term storage for your temporary or permanent data, for both users and services
  • Organize files in folders
  • Store confidential files at user level, making them inaccessible for other users of your tenant

Restrictions

  • File size limits are internally limited to S3 ones, but when using DE via MindGate, MindGate limits apply
  • File and directory names are limited to 1024 characters
  • Number of objects within a parent is limited to 10,000 items
  • Recursive listing of directory contents is not provided

Example Scenario

A developer wants to train a model for anomaly detection using the Anomaly Detection Service. The developer uses the Data Exchange Service to upload a training set into MindSphere. This data can then be used by the Anomaly Detection Service.

Any questions left?

Ask the community


Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.