MicrosoftGraphConnector

Generic Microsoft Graph source connector.

class scamander.sources.microsoft_graph_connector.MicrosoftGraphConnector(config: Config, *args, **kwargs)

Basisklassen: Source

Source connector that reads any Microsoft Graph v1.0 resource into a Spark DataFrame.

A single reusable connector for Microsoft Graph, driven entirely by MicrosoftGraphSourceOptions — Users, Groups, Enterprise Applications (service principals), directory role assignments, or any other Graph endpoint are all just different options.endpoint values, rather than one connector class per resource type.

Authenticates via the AAD v2.0 client-credentials (app-only) flow and paginates with fetch_all_odata_pages(), which follows @odata.nextLink and surfaces a final @odata.deltaLink for delta-capable endpoints.

Expects the Config object to carry a MicrosoftGraphSourceOptions instance under config.source.options.

Parameters:

• config (Config) – Validated configuration object containing Microsoft Graph connection details under config.source.options.

• spark (SparkSession) – An active Spark session used to build the result DataFrame.

Veroorzaakt:

ValueError – If config.source.options is not a MicrosoftGraphSourceOptions instance.

Example:

from scamander.sources.microsoft_graph_connector import MicrosoftGraphConnector
from scamander.models.source_config.microsoft_graph import MicrosoftGraphSourceOptions
from scamander.models.source_config.source import SourceConfig
from scamander.models.config import Config

config = Config(
    source=SourceConfig(
        type="microsoft_graph",
        options=MicrosoftGraphSourceOptions(
            tenant_id=tenant_id,
            client_id=client_id,
            client_secret=client_secret,
            endpoint="/users/delta",
            select=["id", "displayName", "userPrincipalName", "accountEnabled"],
        ),
    ),
)
connector = MicrosoftGraphConnector(config, spark)
df = connector.connect()
new_delta_link = connector.delta_link  # persist for the next incremental run

connect() → DataFrame

Fetches every page of the configured Graph resource and returns a Spark DataFrame.

When options.delta_link is set, the first request is issued directly against it (ignoring endpoint/select/filter/expand), resuming an incremental sync. Otherwise a fresh request is built from options.endpoint and its query parameters. After a successful fetch, self.delta_link holds the new @odata.deltaLink for the caller to persist (None when the endpoint isn’t delta-capable). Records marked @removed by a delta query are kept and flagged via an added is_deleted column rather than being dropped, so removed-principal history is retained for governance/audit purposes.

Returns:

A Spark DataFrame containing every record returned by the

configured Graph resource. When the resource returns no records, an empty, zero-column DataFrame is returned rather than raising (Spark cannot infer a schema from an empty list of dicts).

Return type:

DataFrame