ghjklw commented on issue #42850: URL: https://github.com/apache/airflow/issues/42850#issuecomment-2413124861
Hi @potiuk My mistake for assuming `airflow.cfg` was toml and not ini 🙈 Regarding the tooling for JSON schema with TOML, a fairly easy alternative relying only on largely used robust projects/stdlib would be to read the toml file as a `dict` using [`tomllib.load`](https://docs.python.org/3/library/tomllib.html) and then validating the `dict` using [`jsonschema.validate`](https://python-jsonschema.readthedocs.io/en/stable/) which actually validates a mapping/dictionary/object and not a string. See also: https://python-jsonschema.readthedocs.io/en/stable/faq/#can-jsonschema-be-used-to-validate-yaml-toml-etc An even more powerful solution, but which might require more work depending on how the configuration is implemented today would be to leverage [pydantic-settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/). We would define the configuration as Pydantic models, creating the JSON schema would be straightforward. Pydantic could handle itself the parsing of the TOML file through the [`TomlConfigSettingsSource`](https://docs.pydantic.dev/latest/concepts/pydantic_settings/#other-settings-source). An added benefit of that approach is that it would create an abstraction layer between the definition of the settings structure and the format they're stored in/how they're parsed. It would then be quite easy to use YAML/JSON... `pydantic-settings` can also take care of variables defined through environment variables. Last but not least, [`check-jsonschema`](https://github.com/python-jsonschema/check-jsonschema) has [support form TOML](https://check-jsonschema.readthedocs.io/en/stable/optional_parsers.html#toml). It can be used both as a [CLI tool](https://check-jsonschema.readthedocs.io/en/stable/usage.html) and as a [pre-commit hook](https://check-jsonschema.readthedocs.io/en/stable/precommit_usage.html). Unfortunately, I really do not have the bandwidth nor the experience with Airflow's development to offer my help with the implementation, but if anyone wants to work on it, I'd be happy to be a sparring partner/help with testing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
