Skip to content

Configuration of Events

You can choose to be notified for object events (add, update or delete) for your space in Data Lake. It allows you to subscribe for event notifications generated when the objects of a tenant or subtenant are created, updated or deleted.

Multiple subscriptions for the same path can be created when each has a different destination. Similarly, multiple subscriptions for the same destination can be created when each has a different path.

Maximum 15 subscriptions can be created for a tenant or subtenant. Path in request payload should be upto folders and not upto object e.g. “myfolder/mysubfolder”

Subscribe to the event notification

To create Event Subscription - Once this step is performed IDL will subscribe to the intended SNS topic.

POST /objectEventSubscriptions

Content-Type: application/json

Request example:

  {
        "path": "myfolder/mysubfolder",
        "destination": "aws-sns://arn:aws:sns:region:account-id:topicname",
        "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4"
    }

  {
      "path": "/notification/nfr",
      "destination": "Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=nfr;SharedAccessKey=0LFL2eOdFJRCWDo6bDjtapEMq0XXX45dywFEChfhj4E;EntityPath=idl-notification-nfr-topic"
    }

Response example:

json { "id": "7dc53df5703e49b38670b1c468f47f1f", "storageAccount": "dlbucketname", "storagePath": "data/sub=subtenantId/myfolder/mysubfolder", "destination": "aws-sns://arn:aws:sns:region:account-id:topicname", "eTag": 1, "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" }

  {
    "id": "7dc53df5703e49b38670b1c468f47f1f"
    "destination": "Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=idl-policy;SharedAccessKey=
    0LFL2eOdFJRCWDo6bDjtapEMq0XXX45dywFEChfhj4E=;EntityPath=idl-subscription-topic",
    "storageAccount": "datalake-rc-punazdl",
    "storagePath": "data/ten=punazdl/notification/nfr","eTag": 1
  }

Get the details of events subscribed

You can get the details of SNS topics to which you are already subscribed. It will list object event subscriptions for the tenant or subtenant. If requester is tenant, all the subscriptions for the tenant as well as its all subtenants are returned. If requester is a subtenant, all the subscriptions for the subtenant are returned. If tenant wants to filter results for a particular subtenant, filter query parameter subtenantId can be used. This filter query parameter is applicable only if the requester is tenant.

GET /objectEventSubscriptions/

Response example:

{
    "id": "7dc53df5703e49b38670b1c468f47f1f",
    "storageAccount": "dlbucketname",
    "storagePath": "data/sub=subtenantId/myfolder/mysubfolder",
    "destination": "aws-sns://arn:aws:sns:region:account-id:topicname",
    "eTag": 1,
    "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4"
}

{
    "subscriptions": [
        {
            "id": "d27c4973-35fc-46a8-bc49-7335fcc6a8d2",
            "destination": "Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=idl-policy;
            SharedAccessKey=0LFL2eOdFJRCWDo6bSkoapEMq0HUK23dywFEChfhj4E=;EntityPath=idl-subscription-topic",
            "storageAccount": "datalake-rc-punazdl",
            "storagePath": "data/ten=punazdl/EndToEndTest",
            "eTag": 1
        },
        {
            "id": "e6450b8f-1e22-4d8c-850b-8f1e22cd8c74",
            "destination": "Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=idl-policy;SharedAccessKey=0LFL2eOdFJRCWDo6bDjtapEMq0XXX45dywFEChfhj4E=;EntityPath=idl-subscription-topic",
            "storageAccount": "datalake-rc-punazdl",
            "storagePath": "data/sub=7d598ccff0d82fa826c1b8ce9ce88a4a/EndToEndTest",
            "subtenantId": "7d598ccff0d82fa826c1b8ce9ce88a4a",
            "eTag": 1
        },
        {
            "id": "e17fde2e-b752-4e6f-bfde-2eb7524e6fce",
            "destination": "Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=idl-policy;
            SharedAccessKey=0LFL2eOdFJRCWDo6bDjtapEMq0XXX45dywFEChfhj4E=;EntityPath=idl-subscription-topic",
            "storageAccount": "datalake-rc-punazdl",
            "storagePath": "data/ten=punazdl/notification/nfr",
            "eTag": 1
        }
    ],
    "page": {
        "size": 10,
        "totalElements": 3,
        "totalPages": 1,
        "number": 0
    }
}

Edit created subscription

You can also edit the created subscription.

PATCH /objectEventSubscriptions/7dc53df5703e49b38670b1c468f47f1f

Content-Type: application/json

Request example:

{
    "path": "myCustomPath",
    "destination": "aws-sns://arn:aws:sns:region:account-id:topicname"
}

{
    "path": "myCustomPath",
    "destination": "aws-sns://arn:aws:sns:region:account-id:topicname"
}

Response example:

{
    "id": "7dc53df5703e49b38670b1c468f47f1f",
    "storageAccount": "dlbucketname",
    "storagePath": "data/sub=subtenantId/myfolder/mysubfolder",
    "destination": "aws-sns://arn:aws:sns:region:account-id:topicname",
    "eTag": 1,
    "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4"
}

{
    "id": "7dc53df5703e49b38670b1c468f47f1f",
    "destination": "Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=idl-policy; SharedAccessKey=0LFL2eOdFJRCWDo6bDjtapEMq0XXX45dywFEChfhj4E=;EntityPath=idl-subscription-topic",
    "storageAccount": "datalake-rc-punazdl",
    "storagePath": "data/ten=punazdl/notification/nfr",
    "eTag": 2
}

Delete the subscription

You can delete the created subscription if you do not want any further notifications.

DELETE /objectEventSubscriptions/7dc53df5703e49b38670b1c468f47f1f

Response example

Status 204 No Content

Get the details of events for the given id

You can get the details by ID of SNS topics to which you are already subscribed.

GET /objectEventSubscriptions/7dc53df5703e49b38670b1c468f47f1f

Response Example:

{
  "id": "7dc53df5703e49b38670b1c468f47f1f",
  "destination":"Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=idl-policy;SharedAccessKey=0LFL2eOdFJRCWDo6bDjtapEMq0XXX45dywFEChfhj4E=;EntityPath=idl-subscription-topic",
  "storageAccount": "datalake-rc-punazdl",
  "storagePath": "data/ten=punazdl/notification/nfr",
  "eTag": 2
}

Send event notification

To enable Integrated Data Lake Service to send event notifications on provided SNS topic, the required permissions "SNS:GetTopicAttributes" and "SNS:Publish" should be granted.

The following examples explain the functionality:

  1. Allowing access specifically to Integrated Data Lake Service’s AWS account (Recommended approach):

    • Replace "<IDL AWS Account Id>" with Integrated Data Lake Service’s AWS account id, which is ‘463822690055’
    • Replace "<SNS Topic ARN>" with arn of the SNS topic subscribing to Integrated Data Lake Service

  {
  "Version": "2008-10-17",
  "Id": "<Policy ID>",
  "Statement": [
    {
      "Sid":  "AllowAccessToIDL",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<IDL AWS Account Id>"
      },
      "Action": [
        "SNS:GetTopicAttributes",
        "SNS:Publish"
      ],
      "Resource": "<SNS Topic ARN>"
    }
  ]
  }
2.Allowing access specifically to all AWS accounts (Not Recommended as it allows every AWS account to access SNS topic) Replace "<SNS Topic ARN>" with arn of the SNS topic subscribing to Integrated Data Lake Service

{
  "Version": "2008-10-17",
  "Id": "<Policy ID>",
  "Statement": [
    {
      "Sid":  "AllowAccessToAll",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "SNS:GetTopicAttributes",
        "SNS:Publish"
      ],
      "Resource": "<SNS Topic ARN>"
    }
  ]
  }

Any questions left?

Ask the community

Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.

Last update: December 16, 2021