Concepts¶
A few ideas come back in every chapter. Read this page once and the rest of the manual will make sense.
Catalog, schema, table¶
Data is organized as a tree with three levels:
A catalog is the top level, usually one per domain or department, for example
finance.A schema groups related tables inside a catalog, for example
finance.facturen.A table holds the data, for example
finance.facturen.regels.
You always refer to a table by all three parts: catalog.schema.table.
The platform catalog main is available to everyone and is the default
catalog in notebooks.
Data products and owners¶
A data product is a catalog, schema or table that a team has taken responsibility for. The team that created it is its owner team. Owners decide who else gets access.
Product catalogs and container catalogs¶
When you create a catalog, Security Center asks which Kind it is. Both kinds are ordinary catalogs in the tree above; they differ in who owns what:
Kind |
Who owns what |
Example |
|---|---|---|
Product catalog |
One team owns the whole catalog. The catalog is the data product; the team creates its own schemas and tables in it. |
|
Container catalog |
Nobody owns the catalog as a whole; it is shared. Each team creates its own schema in it, and that schema is the team’s data product (a schema product). |
|
So “container” in Security Center always means a shared catalog, never a schema. In the tree a catalog carries the label Product or Container. If you are unsure, choose a product catalog: that is what most teams need, and it is the only kind teams may create with the default settings.
Roles on a data product¶
Access is given through a role on a part of the tree:
Role |
What it allows |
|---|---|
owner |
Full control, including managing access and deleting the product. |
writer |
Read and change the data. |
reader |
Read the data. |
The owner can also create custom roles, for example regio_noord, to
combine access with a row filter or a column mask.
Filtered access¶
A role can be filtered: some rows are hidden (a row filter) or the values of some columns are masked (a column mask). The data stays in one table; each team only sees what its role allows. In the catalog such a role carries the label filtered.
You read a filtered table in the same way as any other table, in SQL tools and in notebooks. The platform sends the query through Trino so that the filter and the mask always apply. A team with a filtered role can read the table but cannot write to it.
Access flows down the tree¶
Access to a catalog applies to every schema and table in it, and access to a schema applies to every table in it. You can therefore grant access at the level that fits: one table, a whole schema or a whole catalog.
Tables are Delta tables¶
Tables in the catalog are stored in the Delta format. When you create a table in a notebook you have to say so, see Notebooks and SQL. A table written in a notebook is automatically readable from SQL and BI tools as well (the table also exposes itself in a way Trino understands, known as UniForm). You do not have to export or copy anything.