Fokko opened a new pull request, #5488: URL: https://github.com/apache/iceberg/pull/5488
This PR will add the option to read from a configuration file and environment variables. This can be used to read the catalog configuration from a file instead of having to pass it through the CLI or Python. Looked at different Python projects: - https://docs.dask.org/en/stable/configuration.html - https://nvidia.github.io/spark-rapids/docs/configs.html - https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html - https://docs.python.org/3/library/configparser.html Most of them have (a variation) of dot notation config. Python itself comes with a `configparser`: https://docs.python.org/3/library/configparser.html But this one uses sections, that isn't compatible with the dot notation config `pyiceberg.catalog.uri=thrift://`. Also, the Java implementation uses dotted config, so that's also nice. Python ships out of the box with a yaml parser. The current implementation looks like this: ```yaml catalog: production: uri: thrift://prod:9083 rest-dev: uri: http://server.io credential: sometoken ``` You can also override the config using environment variables: ```sh PYICEBERG__CATALOG_PRODUCTION_URI=thrift://dev:9083 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
