PostgresConnector

PostgreSQL JDBC source connector.

class scamander.sources.postgresql_connector.PostgresConnector(config: Config, *args, **kwargs)

Basisklassen: Source

Source connector that reads a PostgreSQL table into a Spark DataFrame via JDBC.

Parameters:

• config (Config) – Validated configuration object containing PostgreSQL connection details under config.source.options.

• spark (SparkSession) – An active Spark session used to execute the JDBC read.

Veroorzaakt:

ValueError – If config.source.options is not a PostgresqlSourceOptions instance.

connect() → DataFrame

Establishes a JDBC connection to PostgreSQL and returns the target table as a Spark DataFrame.

Constructs the JDBC URL from the host, port, and database in self.postgres_options, then uses Spark’s JDBC reader with the provided username, password, and JDBC driver class.

Returns:

A Spark DataFrame backed by the specified PostgreSQL table.

Return type:

DataFrame