swagger: '2.0'
info:
title: Data Lake API
description: >-
Service for storing Objects, download objects, add extended metadata tags,
subscribe for notifications, import tenant specific MindSphere Time
Series data, and enable data access using cross account access and STS in
Integrated MindSphere Data Lake.
- Objects reside in user specified path like below example. Extension in object name is optional. Root path can be denoted as "/"
/basefolder/subfolder/objectname.objectext
- The following generic error codes might occur at any of the specified operations. Generic errors are prefixed with 'mdsp.core.generic.'.
- missingParameter
- invalidParameter
- missingRequestBodyProperty
- invalidRequestBodyProperty
- unauthorized
- forbidden
- noMatch
- unsupportedMediaType
- tooManyRequests
- internalServerError
version: "3.4.1"
x-visibility: external
schemes:
- https
basePath: /api/datalake/v3
tags:
- name: Objects Metadata Catalog Operations
description: Data Lake objects metadata catalog operations.
- name: Object Access Operations
description: Data Lake object access operations.
- name: Object Operations
description: Data Lake object operations.
- name: Object Event Subscription Operations
description: >-
Tenants can choose to be notified for object events (add, update or
delete) for their space in Data Lake.
- name: Time Series Bulk Import
description: Time Series Bulk Import Operations.
- name: Object Operations with Access Token
description: >-
Tenants can request for access token to make direct object operations in object storage
securityDefinitions:
datalakeservice:
type: oauth2
flow: accessCode
authorizationUrl: 'https://oauth.simple.api/authorization'
tokenUrl: 'https://oauth.simple.api/token'
scopes:
dl.ds.r: read object
dl.ds.w: write object
dl.ds.d: delete object
dl.de.r: read event subscription
dl.de.w: create event subscription
dl.de.d: delete event subscription
dl.da.r: read object for data access
dl.da.w: write object for data access
dl.da.d: delete object for data access
dl.dat.r: read object for data access token
dl.dat.w: write scope to enable write data access token
dl.dat.d: delete scope to enable delete data access token
dl.tsi.w: create time series import
dl.tsi.r: read time series imports
dl.tsi.d: delete time series import
paths:
/generateUploadObjectUrls:
post:
summary: Generate signed URLs to upload an object.
tags:
- Object Access Operations
description: Generate signed URLs to upload one or more objects.
operationId: generateUploadObjectUrls
security:
- datalakeservice:
- dl.ds.w
parameters:
- name: generateUrlPayload
in: body
description: upload payload with path details array and optional subtenat id
schema:
$ref: '#/definitions/GenerateUrlPayload'
produces:
- application/json
responses:
'201':
description: Signed URL response for upload
schema:
$ref: '#/definitions/SignedUrlResponse'
'400':
description: |
Bad request. Possible errors:
Code Suffix |
Message (parametrized) |
pathTooLong |
Value of request body property complete path, which is, the default path {defaultpath} and input path must be no longer than 1024 characters. |
pathHasInvalidCharacters |
Characters used for values of request body property paths[].path must be in the character set '[a-zA-Z0-9.!*'() _-/=]'.Spaces are not allowed at beginning or at end. Also, consecutive spaces are not allowed. |
pathInvalid |
Provided property paths[].path is invalid. |
noSuchSubtenant |
Provided subtenant does not belong to tenant. |
tooManyPaths |
A maximum of 30 paths can be provided per request. |
agentPathMismatch |
Agent ID from agent authorization token and path requested for upload do not match. |
schema:
$ref: '#/definitions/Errors'
'403':
$ref: '#/responses/SubtenantIdNotAllowed'
default:
description: Other error with any status code and response body format.
/generateDownloadObjectUrls:
post:
summary: Generate signed URLs to download for single object.
tags:
- Object Access Operations
description: Generate signed URLs to download for single object.
operationId: generateDownloadObjectUrls
security:
- datalakeservice:
- dl.ds.r
parameters:
- name: generateUrlPayload
in: body
description: download payload with path details array and optional subtenat id
schema:
$ref: '#/definitions/GenerateUrlPayload'
produces:
- application/json
responses:
'201':
description: Signed URL response for download
schema:
$ref: '#/definitions/SignedUrlResponse'
'400':
description: |
Bad request. Possible errors:
Code Suffix |
Message (parametrized) |
maxEnabledLimitReached |
Maximum limit, {limit} reached for the enabled cross account accesses. |
maxDisabledLimitReached |
Maximum limit, {limit} reached for the disabled cross account accesses. |
pathLengthExceedError |
Path in request body property, must not be more than 500 characters long. |
pathHasInvalidCharacters |
Characters used for values of request body property path must be in the character set '[a-zA-Z0-9.!*'() _-/=]'.Spaces are not allowed at beginning or at end. Also, consecutive spaces are not allowed. |
immutableAccess |
Special Read Only access at root level for the storage account is already present. Create of a new access is not supported for this cross account. |
parameterTooLong |
Input parameter {name} must be no longer than {length} characters. |
schema:
$ref: '#/definitions/Errors'
'404':
$ref: '#/responses/CrossAccountNotFound'
'409':
description: |
Conflict. Possible errors:
Code Suffix |
Message (parametrized) |
maxEnabledLimitReached |
Maximum limit, {limit} reached for the enabled cross account accesses. |
maxDisabledLimitReached |
Maximum limit, {limit} reached for the disabled cross account accesses. |
pathTooLong |
Value of request body property path must be no longer than 500 characters. |
pathHasInvalidCharacters |
Characters used for values of request body property path must be in the character set '[a-zA-Z0-9.!*'() _-/=]'.Spaces are not allowed at beginning or at end. Also, consecutive spaces are not allowed. |
immutableAccess |
Special Read Only access at root level for the storage account. Update is not supported for this access. |
parameterTooLong |
Input parameter {name} must be no longer than {length} characters. |
schema:
$ref: '#/definitions/Errors'
'404':
$ref: '#/responses/CrossAccountOrAccessNotFound'
'409':
description: |
Conflict. Possible errors:
Code Suffix |
Message (parametrized) |
noFilterProvided |
"Query parameter 'filter' cannot be Null or Empty and should have only one operator from the list [or,and,none]. |
onlyOneOperatorIsAllowed |
Filter criteria accepts only one operator from the list [or,and,none]. |
orOperatorIsEmpty |
Filter criteria for operator 'or' cannot be empty or null. |
andOperatorIsEmpty |
Filter criteria for operator 'and' cannot be empty or null. |
propertiesLessThanTwo |
'and/or' operator must have minimum two properties from this list [name,size,tags,lastModified,location]. |
propertiesMoreThanOne |
'none' operator must have only one property from this list [name,size,tags,lastModified,location]. |
nameIsEmpty |
'name' cannot be empty or null and must have 'contains' as property. |
locationIsEmpty |
'location' cannot be empty or null and must have 'contains' as property. |
sizeIsEmpty |
'size' cannot be empty or null and must have only one property from this list [greaterThan,lessThan,eq,between]. |
lastModifiedIsEmpty |
'lastModified' cannot be empty or null and must have only one property from this list [eq,before,after,between]. |
tagIsEmpty |
tags cannot be empty or null and must have a property 'in'. Property 'in' uses a list of Strings. |
sizeEqPropertyIsInvalid |
Value must be greater than zero for 'size' filter with property 'eq'. |
sizeGreaterThanPropertyIsInvalid |
Value must be greater than zero for 'size' filter with property 'greaterThan' |
sizeLessThanPropertyIsInvalid |
Value must be greater than zero for 'size' filter with property 'lessThan'. |
sizeBetweenIsInvalid |
Value for the property 'between' for 'size' filter must be a list of two elements with value greater than zero. |
betweenRangeIsInvalid |
Range values set for the property 'between' is invalid. First value should be less than second and difference between them should be greater than one. |
eqDateFormatIsInvalid |
Date format set for the property 'eq' is invalid. Date Format must be in the ISO Format e.g. 2017-01-20T11:00:00.000Z. |
beforeDateFormatIsInvalid |
Date format set for the property 'before' is invalid. Date Format must be in the ISO Format e.g. 2017-01-20T11:00:00.000Z. |
afterDateFormatIsInvalid |
Date format set for the property 'after' is invalid. Date format must be in the ISO Format e.g. 2017-01-20T11:00:00.000Z. |
betweenDateFormatIsInvalid |
Value for the property 'between' must be a list of two elements in the ISO date format. e.g. [2017-01-20T11:00:00.000Z,2017-01-20T11:00:01.000Z]. |
zeroMaxRecords |
Query parameter 'maxRecords' must be greater than zero. |
deserialzeError |
Error occured while deserializing one among the parameters[filter,sort]. |
parseError |
Unable to parse given in the parameter from the list [filter,sort]. |
missingParameterValue |
key is used but value is not provided for the parameter from the list [filter,sort,size]. |
schema:
$ref: '#/definitions/Errors'
default:
description: Other error with any status code and response body format.
/objectEventSubscriptions:
post:
summary: Create object event subscription.
tags:
- Object Event Subscription Operations
description: >-
Allows users to subscribe for event notifications generated when the objects
of a tenant or subtenant are created, updated, or deleted. Multiple subscriptions
for the same path can be created when each has a different
destination. Similarly, multiple subscriptions for the same destination
can be created when each has a different path. Maximum 15 subscriptions can be created for a tenant or for a subtenant. Path in request payload should be upto folders and not upto object e.g. "myfolder/mysubfolder"
### Notification Content
Based on the configured subscriptions, event notification messages are published to the destination. The event notification content is formatted in JSON according to this example. If object operation happened in subtenant folder, both tenantId and subtenantId will be part of the message. If object operation happened in tenant folder, only tenantId will be part of the message.:
{
"path" : "/myfolder/mysubfolder/myobj.ext",
"storageAccount" : "dlbucketname",
"storagePath" : "/data/ten=mytenid/myfolder/mysubfolder/myobj.ext"
"tenantId" : "mytenid",
"subtenantId" : "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4"
"eventType" : "MODIFY",
"timestamp" : "2019-05-17T14:47:44.351Z"
}
where
- 'path' path applicable for supplying to Data Lake APIs like /objectMetadata
- 'storageAccount' Bucket name in case of AWS S3
- 'storagePath' Complete S3 prefix path including the object name. Clients should not rely on a given path structure, but should treat storagePath as opaque.
- 'tenantId' Tenant Id on whose bucket the object operation happened, resulting in notification message
- 'subtenantId' Optional. Subtenant Id, in case the object operation happened for a subtenant, resuling in notification message
- 'eventType' is either 'MODIFY' in case of creation or update, and 'DELETE' in case of object deletion.
- 'timestamp' records the time when the event happened. It is formatted according to ISO 8601 extended date/time format 'YYYY-MM-DDThh:mm:ss.sssZ'.
operationId: createObjectEventSubscription
security:
- datalakeservice:
- dl.de.w
parameters:
- name: subscription
in: body
description: object event subscrition details
required: true
schema:
$ref: '#/definitions/Subscription'
consumes:
- application/json
produces:
- application/json
responses:
'201':
description: created
headers:
Location:
type: string
description: URL of the resource
ETag:
type: integer
description: ETag of the resource
schema:
$ref: '#/definitions/SubscriptionResponse'
'400':
description: |
Bad request. Possible errors:
Code Suffix |
Message (parametrized) |
nameError |
Name should only have hyphen & alphanumeric characters and start & end with alphanumeric. |
nameLengthError |
Name length is more than max permissible length({0}). |
destinationSyntaxIncorrect |
Destination syntax is not correct. |
maxLimitReached |
Maximum limit, 15 reached for subscriptions for the tenant/subtenant. |
pathLengthExceedError |
Path in request body property, must not be more than 500 characters long. |
pathHasInvalidCharacters |
Characters used for values of request body property path must be in the character set '[a-zA-Z0-9.!*'() _-/=]'.Spaces are not allowed at beginning or at end. Also, consecutive spaces are not allowed. |
noSuchSubtenant |
Provided subtenant does not belong to tenant. |
schema:
$ref: '#/definitions/Errors'
'403':
$ref: '#/responses/SubtenantIdNotAllowed'
'409':
description: |
Conflict. Possible errors:
Code Suffix |
Message (parametrized) |
nameError |
Name should only have hyphen & alphanumeric characters and start & end with alphanumeric. |
nameLengthError |
Name length is more than max permissible length({0}). |
destinationSyntaxIncorrect |
Destination syntax is not correct. |
pathLengthExceedError |
Path in request body property, must not be more than 500 characters long. |
pathHasInvalidCharacters |
Characters used for values of request body property path must be in the character set '[a-zA-Z0-9.!*'() _-/=]'.Spaces are not allowed at beginning or at end. Also, consecutive spaces are not allowed. |
statusInvalid |
Status field value is invalid. Accepted values are {status}. |
schema:
$ref: '#/definitions/Errors'
'404':
description: |
Not Found. Possible errors:
Code Suffix |
Message (parametrized) |
destinationPathLengthError |
Destination path, which is, the default path {defaultpath} and input path must be no longer than 1024 characters. |
destinationError |
Destination must not be empty. |
destinationCharactersError |
Characters used in the destinationPath must be in following character set [a-zA-Z0-9.!*'() _-].Spaces are not allowed at beginning or at end. Also, consecutive spaces are not allowed. |
noSuchSubtenant |
Provided subtenant does not belong to tenant. |
parameterTooLong |
Input parameter {name} must be no longer than {length} characters. |
durationLimit |
Duration between {to} and {from} should be less than or equal to {days} days. |
timestampMissing |
Value for fields [to] or [from] should not be empty. |
timestampInvalid |
Value for fields [to] or [from] is in invalid format. |
schema:
$ref: '#/definitions/Errors'
'403':
$ref: '#/responses/SubtenantIdNotAllowed'
'409':
description: |
Conflict. Possible errors: