Data Catalog Plugin API

note

Read this page in wide view mode from here

• DataCatalogPlugin
  • postValidate a Data Catalog Entity provision request (async)
  • getGet status of an async validation task
  • postValidate a Data Catalog Entity provision request (sync)
  • postProvision Data Catalog entity
  • postUnprovision Data Catalog entity
  • getGet status of Data Catalog entity provision/unprovision task
  • getGet linked Data Catalog entity

API docs by Redocly

Data Catalog Plugin API (2.2.0)

A Data Catalog Plugin is a microservice that is responsible for provisioning and unprovisioning metadata into a Data Catalog during the deployment phase of a resource through Witboost.

It takes in a system descriptor enriched with components' provisioning results and returns the result of the provisioned Data Catalog entities.

The Data Catalog contract is similar to a technology adapter, with a few differences. It takes in the whole system descriptor already enriched with tech adapters' deploy info.

DataCatalogPlugin

Validate a Data Catalog Entity provision request (async)

Validate output ports metadata attached in the provisioning request.

In particular, it validates the format and the existence of glossary terms and classification tags

Request Body schema: application/json

required

Details of a provisioning request to be validated

descriptorKind required	string (DescriptorKind) Enum: "DATAPRODUCT_DESCRIPTOR" "DATAPRODUCT_DESCRIPTOR_WITH_RESULTS"
descriptor required	string A provisioning request in yaml format

Responses

https://docs.witboost.com/datamesh.datacatalogplugin/v2/validate

Request samples

• Payload

Content type

application/json

Example

systemDescriptorcomponentDescriptorsystemDescriptor

{

• "descriptorKind": "DATAPRODUCT_DESCRIPTOR",
• "descriptor": "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false",
• "removeData": false

}

Response samples

• 202
• 400
• 500

Content type

application/json

"2f4b3b3b-4b3b-4b3b-4b3b-4b3b4b3b4b3b"

Get status of an async validation task

path Parameters

token required

string Example: 2a4bb060-27c9-403d-ab5d-d3776cf56c3b Token obtained after a call to the async validate endpoint

Responses

https://docs.witboost.com/datamesh.datacatalogplugin/v2/validate/{token}/status

Response samples

• 200
• 400
• 500

Content type

application/json

{

• "status": "COMPLETED",
• "info": {

  • "publicInfo": {

    • "valid": true

    },
  • "privateInfo": {

    • "valid": true

    }

  }

}

Validate a Data Catalog Entity provision request (sync)

Synchronously validate a provisioning request and return the validation result.

You want to implement this endpoint to assess that the provided resource descriptor conforms to the expected structure and has all the necessary information to be deployed on the infrastructure.

It is highly recommended to implement the asynchronous validation endpoint instead of this one.

Request Body schema: application/json

required

Details of a provisioning request to be validated

descriptorKind required	string (DescriptorKind) Enum: "DATAPRODUCT_DESCRIPTOR" "DATAPRODUCT_DESCRIPTOR_WITH_RESULTS"
descriptor required	string A provisioning request in yaml format

Responses

https://docs.witboost.com/datamesh.datacatalogplugin/v1/validate

Request samples

• Payload

Content type

application/json

Example

systemDescriptorcomponentDescriptorsystemDescriptor

{

• "descriptorKind": "DATAPRODUCT_DESCRIPTOR",
• "descriptor": "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false",
• "removeData": false

}

Response samples

• 200
• 500

Content type

application/json

{

• "valid": true

}

Provision Data Catalog entity

Provisioning is the operation that deploys the metadata of a system or a single component into a third-party Data Catalog.

This request can be handled in synchronous or asynchronous mode depending on the implementation.

Request Body schema: application/json

required

A Data Product descriptor, enriched with components' provisioning results, wrapped as a string into a simple object

descriptorKind required	string (DescriptorKind) Enum: "DATAPRODUCT_DESCRIPTOR" "DATAPRODUCT_DESCRIPTOR_WITH_RESULTS"
descriptor required	string A provisioning request in yaml format

Responses

https://docs.witboost.com/datamesh.datacatalogplugin/v1/provision

Request samples

• Payload

Content type

application/json

Example

systemDescriptorcomponentDescriptorsystemDescriptor

{

• "descriptorKind": "DATAPRODUCT_DESCRIPTOR",
• "descriptor": "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false",
• "removeData": false

}

Response samples

• 200
• 202
• 400
• 500

Content type

application/json

{

• "status": "COMPLETED",
• "info": {

  • "publicInfo": {

    • "entitiesCommitted": 10

    },
  • "privateInfo": {

    • "entitiesCommitted": 10,
    • "glossaryId": "dfasd13"

    },
  • "logs": [

    • {

      • "timestamp": "2021-09-30T10:00:00Z",
      • "level": "INFO",
      • "message": "Entity provisioned successfully"

      }

    ]

  }

}

Unprovision Data Catalog entity

Unprovisioning is the operation that removes the metadata of a system or a single component from a third-party Data Catalog.

This request is synchronous and returns the result of the unprovisioning process.

Request Body schema: application/json

required

A system descriptor and the provisioning results wrapped as simple object

descriptorKind required	string (DescriptorKind) Enum: "DATAPRODUCT_DESCRIPTOR" "DATAPRODUCT_DESCRIPTOR_WITH_RESULTS"
descriptor required	string A provisioning request in yaml format

Responses

https://docs.witboost.com/datamesh.datacatalogplugin/v1/unprovision

Request samples

• Payload

Content type

application/json

Example

systemDescriptorcomponentDescriptorsystemDescriptor

{

• "descriptorKind": "DATAPRODUCT_DESCRIPTOR",
• "descriptor": "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false",
• "removeData": false

}

Response samples

• 200
• 202
• 400
• 500

Content type

application/json

{

• "status": "COMPLETED",
• "info": {

  • "publicInfo": {

    • "entitiesRemoved": 10

    },
  • "privateInfo": {

    • "entitiesRemoved": 10,
    • "glossaryId": "dfasd13"

    },
  • "logs": [

    • {

      • "timestamp": "2021-09-30T10:00:00Z",
      • "level": "INFO",
      • "message": "Entity unprovisioned successfully"

      }

    ]

  }

}

Get status of Data Catalog entity provision/unprovision task

path Parameters

token required

string Example: f6440015-1404-4e96-a250-7f680dbc32d4 Data Catalog Provision/Unprovision Task Token obtained after a call to the provision/unprovision endpoint

Responses

https://docs.witboost.com/datamesh.datacatalogplugin/v1/provision/{token}/status

Response samples

• 200
• 400
• 500
• 501

Content type

application/json

{

• "status": "COMPLETED",
• "info": {

  • "publicInfo": {

    • "entitiesCommitted": 10

    },
  • "privateInfo": {

    • "entitiesCommitted": 10,
    • "glossaryId": "dfasd13"

    },
  • "logs": [

    • {

      • "timestamp": "2021-09-30T10:00:00Z",
      • "level": "INFO",
      • "message": "Entity provisioned successfully"

      }

    ]

  }

}

Get linked Data Catalog entity

Return the reference (id, links, etc) to the Data Catalog entity that refers to the provided Output Port

query Parameters

componentId required

string Example: componentId=urn:dmb:dp:finance:a-system:0:component:1 Output Port URN to get the reference to the Data Catalog entity

Responses

https://docs.witboost.com/datamesh.datacatalogplugin/v1/entity/reference

Response samples

• 200
• 500

Content type

application/json

{

• "reference": "https://datacatalog.witboost.com/entity/finance.a-system.0.component.1"

}