apache-airflow-providers-git-azure¶

Adds AzureDevOpsGitDagBundle: a Git DAG bundle for private Azure DevOps repositories with no static credential: no PAT, no SSH key, no client secret. This is what a DAG-processor/worker pod uses to actually fetch DAGs from git, as distinct from the other providers on this site, which add operators/hooks that a DAG’s own tasks use.

The bundle is transparent. It behaves exactly like the stock GitDagBundle for GitHub (or any other) URLs, PAT-based connections, and SSH-key connections, so it’s safe to use as the bundle class for every git connection, including customer-managed ones. The Entra ID logic only activates for Azure DevOps HTTPS URLs (dev.azure.com / *.visualstudio.com):

Connection contents (Azure DevOps HTTPS URL)	Auth mode
Host only	Workload identity (`WorkloadIdentityCredential`, falling back to `DefaultAzureCredential` for local development)
Login + password + `tenant_id` extra	Service principal client secret (`ClientSecretCredential`)
Login + password, no `tenant_id` extra	PAT — stock `GitHook` behaviour
SSH key (`private_key`/`key_file` extra) or a `git@…` URL	Stock `GitHook` behaviour

Why this exists¶

Azure DevOps Git over HTTPS accepts Microsoft Entra ID access tokens (audience 499b84ac-1321-427f-aa17-267ca6975798) as authentication, but they expire after roughly an hour: too short-lived to store in an Airflow connection. This bundle mints a fresh token at clone/fetch time instead, through the pod’s workload identity, so the connection only ever needs to carry the repository URL.

Security properties:

The token is never embedded in the remote URL, so it’s never persisted into the bare repository’s .git/config.
The token is passed to the git subprocess through GIT_CONFIG_COUNT/GIT_CONFIG_KEY_0=http.extraHeader/ GIT_CONFIG_VALUE_0 environment variables (git ≥ 2.31), not on the command line and not in any file.
The token is never logged, and the hook environment is cleared as soon as the git command finishes.

Prerequisites¶

Workload identity only (skip if using a PAT or service principal client secret connection instead — see “Connection setup” below):

An AKS cluster with workload identity enabled, and a managed identity with federated credentials for the Airflow dag-processor and worker service accounts.
The managed identity added to the Azure DevOps organization (requires a Basic access level) and granted Reader + Code: Read on the target repository specifically (Project Settings → Repositories → <repo> → Security) — not organization-wide — plus View project-level information, granted at the project level: Project Settings → Permissions → Users → search for the identity → expand its entry → enable View project-level information.
The Airflow pods labelled azure.workload.identity/use: "true" and the service account annotated with azure.workload.identity/client-id: <client-id>.

The service principal client secret flow has the same Azure DevOps-side requirement as step 2 above: the app registration itself (not a managed identity) must be onboarded to the org with a Basic access level and granted Reader + Code: Read on the repo, plus View project-level information on the project (Project Settings → Permissions → Users → search for the service principal → expand its entry).

Connection setup¶

For a screenshot walkthrough of creating this connection in the Airflow UI, see Git (Azure DevOps) connection.

Workload identity: a standard git connection whose host is the HTTPS repository URL, with login/password left empty:

airflow connections add scamander-example \
  --conn-type git \
  --conn-host "https://dev.azure.com/<org>/<project>/_git/<repo>"

Service principal client secret: the same connection, plus the service principal’s client ID as login, its client secret as password, and the tenant in the extra. The password is used as an Entra client secret here, not as a git password. Without the tenant_id extra, a password is instead treated as a PAT:

airflow connections add scamander-example \
  --conn-type git \
  --conn-host "https://dev.azure.com/<org>/<project>/_git/<repo>" \
  --conn-login "<client-id>" \
  --conn-password "<client-secret>" \
  --conn-extra '{"tenant_id": "<tenant-id>"}'

DAG bundle configuration¶

Point the bundle config at the new class (AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST or the corresponding Helm value):

[
  {
    "name": "scamander-example",
    "classpath": "airflow.providers.git_azure.bundles.git_azure.AzureDevOpsGitDagBundle",
    "kwargs": {
      "tracking_ref": "main",
      "git_conn_id": "scamander-example"
    }
  }
]

AzureDevOpsGitDagBundle accepts the same kwargs as the stock GitDagBundle: tracking_ref, subdir, git_conn_id, repo_url, submodules, prune_dotgit_folder, sparse_dirs.

Overriding the tracked branch per instance¶

A tracking_ref key in the connection’s extra overrides the tracking_ref from the bundle config. That means a single deployed bundle config can track a different branch/tag per Airflow instance just by editing the connection:

airflow connections add scamander-example \
  --conn-type git \
  --conn-host "https://dev.azure.com/<org>/<project>/_git/<repo>" \
  --conn-extra '{"tracking_ref": "release/2.0"}'

The bundle config’s tracking_ref kwarg is therefore optional. It’s an error only if neither the config nor the connection provides a value.