scamander-providers-fabric¶

Lets Data and Analytics engineers trigger on-demand Microsoft Fabric jobs from Airflow DAGs: notebooks, pipelines, and capacity (pause/resume) management. It supports deferrable (async) execution via Airflow triggers, so a long-running Fabric item run doesn’t hold a worker slot the whole time.

Connection setup¶

For a screenshot walkthrough of creating this connection in the Airflow UI, see Fabric connection.

There are two separate connection types, since a run and a capacity operation authenticate against different Fabric/Azure resources:

Connection type	Used by	Fields
`fabric`	`FabricRunItemOperator` / `FabricHook`	Login: client ID · Password: client secret · Extra `tenantId` · Extra `refreshToken` (optional, user-delegated auth — rotated automatically back into the connection on each use) · Extra `scopes` (optional, default `https://api.fabric.microsoft.com/.default`)
`fabric_capacity`	Capacity operators / `FabricCapacityHook`	Login: client ID · Password: client secret · Extra `tenantId` · Extra `scopes` (optional, default `https://management.azure.com/.default`)

Operators¶

FabricRunItemOperator: Runs a Fabric item (notebook or pipeline) in a workspace. Key parameters: workspace_id, item_id, fabric_conn_id, job_type (e.g. "RunNotebook", "Pipeline"), wait_for_termination (default True), timeout (seconds, default 7 days), check_interval (seconds, default 60), deferrable (default from Airflow’s own operators.default_deferrable config), job_params (dict passed through to the Fabric job), use_notebook_beta_endpoints. Pushes run_id/location to XCom and exposes a “Monitor Item Run” link to the Fabric portal.
SuspendFabricCapacityOperator / ResumeFabricCapacityOperator / GetFabricCapacityOperator: Pause, resume, or fetch the state of a Fabric capacity (an Azure resource, not a workspace item). Shared parameters: fabric_capacity_conn_id, subscription_id, resource_group, capacity_name, max_retries (default 5), retry_delay (seconds, default 1). Useful for cost control, e.g. suspending a capacity outside business hours.

Usage¶

from scamander.providers.fabric.operators.fabric_operator import FabricRunItemOperator
from scamander.providers.fabric.operators.fabric_capacity_operator import (
    ResumeFabricCapacityOperator,
    SuspendFabricCapacityOperator,
)

resume_capacity = ResumeFabricCapacityOperator(
    task_id="resume_capacity",
    fabric_capacity_conn_id="fabric_capacity_default",
    subscription_id="<subscription-id>",
    resource_group="<resource-group>",
    capacity_name="<capacity-name>",
)

run_notebook = FabricRunItemOperator(
    task_id="run_notebook",
    fabric_conn_id="fabric_default",
    workspace_id="<workspace-id>",
    item_id="<notebook-item-id>",
    job_type="RunNotebook",
)

suspend_capacity = SuspendFabricCapacityOperator(
    task_id="suspend_capacity",
    fabric_capacity_conn_id="fabric_capacity_default",
    subscription_id="<subscription-id>",
    resource_group="<resource-group>",
    capacity_name="<capacity-name>",
)

resume_capacity >> run_notebook >> suspend_capacity