SqlServerConnector

SQL Server JDBC source connector.

class scamander.sources.sqlserver_connector.SqlServerConnector(config: Config, *args, **kwargs)

Bases: Source

Source connector that reads a SQL Server table or query into a Spark DataFrame via JDBC.

Expects the Config object to carry a SqlSourceOptions instance under config.source.options. Either options.table or options.query must be set to identify the data source.

Parameters:

  • config (Config) – Validated configuration object containing SQL Server connection details under config.source.options.

  • spark (SparkSession) – An active Spark session used to execute the JDBC read.

Raises:

  • ValueError – If config.source.options is not a SqlSourceOptions instance.

  • ValueError – If neither options.table nor options.query is provided.

connect() → DataFrame

Establishes a JDBC connection to SQL Server and returns data as a Spark DataFrame.

When options.query is set it is used as the data source (wrapped in a subquery for JDBC compatibility); otherwise options.table is used directly.

The connection is built from host, port, database, username, password, encrypt, and trust_server_certificate from SqlSourceOptions.

Returns:

A Spark DataFrame backed by the SQL Server source.

Return type:

DataFrame