Skip to content

Uploading the files to Data Registry

The files are uploaded to the Data Registry from the Data Explorer in Integrated Data Lake application. The files uploaded could be in CSV, JSON, PARQUET, or XML format.

When the file is uploaded in Integrated Data Lake application, a notification is triggered in Data Contextualization. Data Contextualization uses the folder metadata tag to determine the registry for the file and the application then proceeds to ingest the file into the designated registry and initiates the schema discovery process.

Data Registry flow chart

Uploading the files for Enterprise Data Registry

In Enterprise Data Registry, the user can initiate the schema discovery process by uploading files in Integrated Data Lake after creating the registry. The user needs to tag the files in the Integrated Data Lake to indicate association of the files to a specific Data Registry. The user can use following procedure to tag and upload the files in the Integrated Data Lake.

Note

For Enterprise Data Registry, the folder must be created in the "sdi" folder of the Integrated Data Lake. User can create different folders for different registries.

Uploading a file in Integrated Data Lake

To upload the file into Integrated Data Lake, follow these steps:

  1. In the Integrated Data Lake, navigate to the 'Data Explorer' window and click "Create Folder".
  2. Create a folder inside the "sdi" folder.
  3. Enter the folder name and click right arrow to save the folder.
  4. Click eye-icon to add the metadata tags for the created folder.
  5. Select "Metadata Tags", click "Add Tag".
  6. Enter metadata tag to the folder in the following format, registryId_ABCD where "ABCD" is the "registryId".

  7. Click "Save".

    Metadata tag

  8. In the Upload Objects, click "Add Files" and select the file to be uploaded.

    Upload file

  9. Click "Upload".

Note

  • If a user tags a folder with a registryId, all files copied in that folder will be ingested in the same registry.
  • A user can optionally add a metadata tag on a file. The metadata must be in the same format as the folder metadata. When both file metadata and (parent) folder metadata are present, Data Contextualization gives preference to the file metadata. So, file metadata can be used to override the registryId specified on the folder metadata.

Result

Uploading the file is successful and the upload status changes to successfully uploaded.

Uploaded file

In Data Explorer window, click Refresh "Refresh View" button to view the uploaded files in the table.

Uploading a file for IoT Data Registry

After creating a registry in the IoT Data Registry, user needs to import the IoT data to Integrated Data Lake. To import the IoT data into Integrated Data Lake, refer Importing data into Integrated Data Lake.

Note

For IoT Data Registry, the folder must be created in the "sdi" folder of the Integrated Data Lake. User can create different folders for different registries.

Last update: January 22, 2024