JsonRecord

Utilities for reading raw JSON files from the local filesystem into Python dicts.

Intended for pipelines (e.g. BRP) where JSON must be parsed as Python objects before a Spark DataFrame is constructed.

class scamander.utils.json_utils.JsonRecord(path: str, data: dict)

Basisklassen: object

A parsed JSON file: its local path and the decoded dict.

scamander.utils.json_utils.read_json_records(local_path: str, log: HumanLogger | None = None) → list[JsonRecord]

Read all .json files under local_path into JsonRecord objects.

Files that cannot be decoded are skipped with a warning log (or printed if no logger is supplied).

Parameters:

  • local_path – Absolute local filesystem path to the directory to scan.

  • log – Optional HumanLogger (platform.log). When omitted, warnings are printed to stdout.

Returns:

List of JsonRecord objects, one per successfully parsed file.

Veroorzaakt:

FileNotFoundError – When local_path does not exist or is not a directory.

Example:

from scamander.utils import read_json_records

records = read_json_records("/tmp/brp_export/", log=platform.log)
for record in records:
    print(record.path, record.data.keys())