Skip to main content

Building Blocks

Intended audience: Data Teams, Platform Teams

If you are new to Witboost, you have probably already heard about projects, components, subcomponents, templates, and descriptors. These are the building blocks of the Witboost Control Plane, and they are the foundation of how you interact with the platform.

This page gives you a conceptual map of the Control Plane: what you create, how it's built, and what happens to it once you're done. No implementation details, just the big picture.

Projects: your workspace​

A project is the main container for everything you build in the Control Plane. Depending on your organization's data landscape, a project can take different shapes: a Data Product, a source system, a machine learning project, and so on. For most data teams working with a Data Mesh, your project is your Data Product.

A project belongs to a domain and has an owner. On its own, a project doesn't do much: its real purpose comes from the components you add to it.

Components: the pieces of the puzzle​

Components are the technical pieces that live inside a project and actually do the work: a table, a pipeline, an output port, a machine learning model, a storage bucket, and more.

Some components are marked as consumable, meaning that once published, other teams will be able to discover and request access to them from the Market Plane.

Other components exist only to support the internal workings of your project and are never exposed outside of it. For example, a pipeline that reads from a storage component is not consumable, but the storage component itself may be.

A project can have as many components as it needs, and they can depend on one another:

  • With a readsFrom to mark a component as a source of data for another component. For example, a pipeline that reads from a storage component.
  • With a dependsOn to mark a component as a dependency of another component. For example, an ingestion pipeline that writes to a storage component. This informs the platform that the storage component must be deployed before the ingestion pipeline can be deployed.

Subcomponents: building blocks within building blocks​

Some components are made of smaller pieces that only live together with their parent: think of a table that belongs to a database schema. These are subcomponents. Whenever the parent component is deployed, its subcomponents are deployed along with it, because they share the same lifecycle.

You won't always need subcomponents, but it's useful to recognize them when you see a component made up of several inner parts.

The most common use-case for subcomponents is when a component is a logical grouping of several related pieces that are deployed together. This gives you a way to request access to the whole group at once, instead of having to request access to each piece individually. For example, a database schema may contain several tables, and you may want to give other teams access to the whole schema instead of each table separately.

Templates: how projects and components come to life​

You never build a project or a component from a blank page. Instead, you start from a template: a guided form, prepared in advance by your Platform Team, that walks you through the choices you need to make. Fill it in, and Witboost takes care of creating the project or component for you, already aligned with your organization's standards and best practices.

When you need to create a new project with several related components at once for a common use case, you may also find blueprints: templates that come bundled together with their relationships already defined, so you don't have to wire everything up manually.

For example, a blueprint for a Data Product may include a storage component, a pipeline component, and an output port component, all pre-configured to work together. You just fill in the details and Witboost creates the whole thing for you.

The descriptor: where it all comes together​

As you add and configure components to your project, you may have noticed that Witboost stores some metadata along the way in the project and components repositories.

Those files help Witboost to compile your project into the descriptor: a single, unified definition that Witboost composes on-demand by collecting the metadata of your project together with the metadata of every one of its components and subcomponents. Think of it like a snapshot of your project at a given point in time, with all the information needed to deploy it, ownership metadata and business context.

You don't need to write it by hand; instead, the descriptor is generated by Witboost when you create or update a project's release.

Finally the descriptor is what drives the next steps in the lifecycle of your project, which:

  • gets checked against your organization's governance policies, to make sure everything is compliant before it goes live;
  • is sent to the Tech Adapters, the pieces of the platform in charge of automating the actual provisioning of infrastructure, so that what you designed becomes real;
  • gets published to the Witboost Marketplace or Data Catalogs, once your project is deployed, so other teams can discover and consume it.

In short: everything you configure through projects and components eventually flows into the descriptor, and the descriptor is what drives validation, provisioning, and publication.

Putting it all together​

For Platform Teams

If you're on a Platform Team, these same building blocks are the foundation you design around:

  • Templates and Blueprints are what you build to let data teams create standardized projects and components
  • Tech Adapters are what you implement to turn descriptors into real infrastructure.

Understanding this flow from the data team's perspective is a good starting point before diving into how to build templates and tech adapters, as well as other Witboost extension points.

What's next​

  • As a data team member, you can start creating your own projects and components using the templates and blueprints provided by your Platform Team. Check out the Getting Started with Witboost guide for a step-by-step walkthrough.

  • As a platform team member, you can start building your own templates and tech adapters to enable data teams to create standardized projects and components. Check out the Building Templates and Building Tech Adapters guides for more information.