amoghrajesh commented on issue #44352:
URL: https://github.com/apache/airflow/issues/44352#issuecomment-3219821895
Performed a small analysis on the pattern of conf imports used by the
providers, there are 400+ usages which are mostly either provider configs, or
operator related (task sdk), the only ones (non tests) using core are:
| Config Key | File Location | Usage | Purpose |
|------------|---------------|-------|---------|
| `core.security` |
`providers/apache/hdfs/src/airflow/providers/apache/hdfs/hooks/webhdfs.py` |
`_kerberos_security_mode = conf.get("core", "security") == "kerberos"` |
Kerberos authentication detection |
| `core.security` |
`providers/apache/hive/src/airflow/providers/apache/hive/hooks/hive.py` | `if
conf.get("core", "security") == "kerberos":` (4 occurrences) | Kerberos
authentication checks |
| `core.execution_api_server_url` |
`providers/celery/src/airflow/providers/celery/executors/celery_executor_utils.py`
| `server=conf.get("core", "execution_api_server_url",
fallback=default_execution_api_server)` | Celery worker API communication |
| `core.execution_api_server_url` |
`providers/edge3/src/airflow/providers/edge3/cli/worker.py` |
`execution_api_server_url = conf.get("core", "execution_api_server_url",
fallback="")` | Edge worker API endpoint |
| `core.internal_api_clock_grace` |
`providers/edge3/src/airflow/providers/edge3/worker_api/auth.py` | `clock_grace
= conf.getint("core", "internal_api_clock_grace", fallback=30)` | API
authentication timing tolerance |
| `core.internal_api_secret_key` |
`providers/edge3/src/airflow/providers/edge3/worker_api/auth.py` |
`secret_key=conf.get("core", "internal_api_secret_key")` | API authentication
secret |
| `core.PARALLELISM` |
`providers/edge3/src/airflow/providers/edge3/executors/edge_executor.py` |
`PARALLELISM: int = conf.getint("core", "PARALLELISM")` | System-wide task
parallelism limit |
| `core.dbapihook_resolve_target_fields` |
`providers/common/sql/src/airflow/providers/common/sql/hooks/sql.py` |
`_resolve_target_fields = conf.getboolean("core",
"dbapihook_resolve_target_fields", fallback=False)` | SQL hook field resolution
behavior |
| `core.enable_xcom_pickling` |
`providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/adx.py`
| `if conf.getboolean("core", "enable_xcom_pickling", fallback=False):` | XCom
serialization method |
| `core.enable_xcom_pickling` |
`providers/microsoft/winrm/src/airflow/providers/microsoft/winrm/operators/winrm.py`
| `enable_pickling = conf.getboolean("core", "enable_xcom_pickling",
fallback=False)` | XCom serialization method |
| `core.enable_xcom_pickling` |
`providers/ssh/src/airflow/providers/ssh/operators/ssh.py` | `enable_pickling =
conf.getboolean("core", "enable_xcom_pickling", fallback=False)` | XCom
serialization method |
| `core.default_timezone` |
`providers/fab/src/airflow/providers/fab/www/extensions/init_jinja_globals.py`
| `server_timezone = conf.get("core", "default_timezone")` | Web UI timezone
display |
### Impact for Worker Configuration:
All these core settings **must be included** in worker configuration files
since providers running on workers depend on them for proper functionality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
