Concepts
========

A few ideas come back in every chapter. Read this page once and the rest of
the manual will make sense.

Catalog, schema, table
----------------------

Data is organized as a tree with three levels:

- A **catalog** is the top level, usually one per domain or department, for
  example ``finance``.
- A **schema** groups related tables inside a catalog, for example
  ``finance.facturen``.
- A **table** holds the data, for example ``finance.facturen.regels``.

You always refer to a table by all three parts: ``catalog.schema.table``.
The platform catalog ``main`` is available to everyone and is the default
catalog in notebooks.

Data products and owners
------------------------

A **data product** is a catalog, schema or table that a team has taken
responsibility for. The team that created it is its **owner team**. Owners
decide who else gets access.

Product catalogs and container catalogs
---------------------------------------

When you create a catalog, Security Center asks which **Kind** it is. Both
kinds are ordinary catalogs in the tree above; they differ in who owns what:

.. list-table::
   :header-rows: 1
   :widths: 25 40 35

   * - Kind
     - Who owns what
     - Example
   * - **Product catalog**
     - One team owns the whole catalog. The catalog is the data product;
       the team creates its own schemas and tables in it.
     - ``finance`` with ``finance.facturen.regels``
   * - **Container catalog**
     - Nobody owns the catalog as a whole; it is shared. Each team creates
       its own schema in it, and that schema is the team's data product (a
       **schema product**).
     - ``bronze`` with ``bronze.finance.regels`` of Team Financiën and
       ``bronze.marketing.leads`` of Team Marketing

So "container" in Security Center always means a shared *catalog*, never a
schema. In the tree a catalog carries the label **Product** or
**Container**. If you are unsure, choose a product catalog: that is what
most teams need, and it is the only kind teams may create with the default
settings.

Roles on a data product
-----------------------

Access is given through a role on a part of the tree:

.. list-table::
   :header-rows: 1

   * - Role
     - What it allows
   * - **owner**
     - Full control, including managing access and deleting the product.
   * - **writer**
     - Read and change the data.
   * - **reader**
     - Read the data.

The owner can also create **custom roles**, for example ``regio_noord``, to
combine access with a row filter or a column mask.

Filtered access
---------------

A role can be **filtered**: some rows are hidden (a *row filter*) or the
values of some columns are masked (a *column mask*). The data stays in one
table; each team only sees what its role allows. In the catalog such a role
carries the label *filtered*.

You read a filtered table in the same way as any other table, in SQL tools
and in notebooks. The platform sends the query through Trino so that the
filter and the mask always apply. A team with a filtered role can read the
table but cannot write to it.

Access flows down the tree
--------------------------

Access to a catalog applies to every schema and table in it, and access to a
schema applies to every table in it. You can therefore grant access at the
level that fits: one table, a whole schema or a whole catalog.

Tables are Delta tables
-----------------------

Tables in the catalog are stored in the **Delta** format. When you create a
table in a notebook you have to say so, see :doc:`notebooks`. A table
written in a notebook is automatically readable from SQL and BI tools as
well (the table also exposes itself in a way Trino understands, known as
*UniForm*). You do not have to export or copy anything.