Concepts

A few ideas come back in every chapter. Read this page once and the rest of the manual will make sense.

Catalog, schema, table

Data is organized as a tree with three levels:

  • A catalog is the top level, usually one per domain or department, for example finance.

  • A schema groups related tables inside a catalog, for example finance.facturen.

  • A table holds the data, for example finance.facturen.regels.

You always refer to a table by all three parts: catalog.schema.table. The platform catalog main is available to everyone and is the default catalog in notebooks.

Data products and owners

A data product is a catalog, schema or table that a team has taken responsibility for. The team that created it is its owner team. Owners decide who else gets access.

Product catalogs and container catalogs

When you create a catalog, Security Center asks which Kind it is. Both kinds are ordinary catalogs in the tree above; they differ in who owns what:

Kind

Who owns what

Example

Product catalog

One team owns the whole catalog. The catalog is the data product; the team creates its own schemas and tables in it.

finance with finance.facturen.regels

Container catalog

Nobody owns the catalog as a whole; it is shared. Each team creates its own schema in it, and that schema is the team’s data product (a schema product).

bronze with bronze.finance.regels of Team Financiën and bronze.marketing.leads of Team Marketing

So “container” in Security Center always means a shared catalog, never a schema. In the tree a catalog carries the label Product or Container. If you are unsure, choose a product catalog: that is what most teams need, and it is the only kind teams may create with the default settings.

Roles on a data product

Access is given through a role on a part of the tree:

Role

What it allows

owner

Full control, including managing access and deleting the product.

writer

Read and change the data.

reader

Read the data.

The owner can also create custom roles, for example regio_noord, to combine access with a row filter or a column mask.

Filtered access

A role can be filtered: some rows are hidden (a row filter) or the values of some columns are masked (a column mask). The data stays in one table; each team only sees what its role allows. In the catalog such a role carries the label filtered.

You read a filtered table in the same way as any other table, in SQL tools and in notebooks. The platform sends the query through Trino so that the filter and the mask always apply. A team with a filtered role can read the table but cannot write to it.

Access flows down the tree

Access to a catalog applies to every schema and table in it, and access to a schema applies to every table in it. You can therefore grant access at the level that fits: one table, a whole schema or a whole catalog.

Tables are Delta tables

Tables in the catalog are stored in the Delta format. When you create a table in a notebook you have to say so, see Notebooks and SQL. A table written in a notebook is automatically readable from SQL and BI tools as well (the table also exposes itself in a way Trino understands, known as UniForm). You do not have to export or copy anything.