amoghrajesh commented on PR #56187:
URL: https://github.com/apache/airflow/pull/56187#issuecomment-3569753487
@xBis7, I thought about it for a bit and this is what i think would work.
Let's take Stats for example. Common pattern for the problem.
Since only the validator functions need conf, this is what you could do.
1. Move everything like you have to shared library:
protocols.py — no conf dependency
base_stats_logger.py — no conf dependency
Logger classes (SafeStatsdLogger, SafeDogStatsdLogger, SafeOtelLogger) —
already receive config via constructor
Validator classes (ListValidator, PatternAllowListValidator,
PatternBlockListValidator) — no conf dependency
2. Move `get_validator` to shared too, but with a signature as such:
```python
def get_validator(
metrics_allow_list: str | None = None,
metrics_block_list: str | None = None,
) -> ListValidator:
```
And similar for:
```python
def get_current_handler_stat_name_func(
stat_name_handler: Callable[[str], str] | None = None,
statsd_influxdb_enabled: bool = False,
) -> Callable[[str], str]:
```
3. Keep factory methods for this in core airflow:
```diff
diff --git a/airflow-core/src/airflow/metrics/statsd_logger.py
b/airflow-core/src/airflow/metrics/statsd_logger.py
--- a/airflow-core/src/airflow/metrics/statsd_logger.py (revision
43aa295d037b412e62794efd872e64f069c7e20d)
+++ b/airflow-core/src/airflow/metrics/statsd_logger.py (date
1763976593904)
@@ -180,8 +180,14 @@
ipv6=conf.getboolean("metrics", "statsd_ipv6", fallback=False),
)
+ # read config values
influxdb_tags_enabled = conf.getboolean("metrics",
"statsd_influxdb_enabled", fallback=False)
- metric_tags_validator = PatternBlockListValidator(
- conf.get("metrics", "statsd_disabled_tags", fallback=None)
+ statsd_disabled_tags = conf.get("metrics", "statsd_disabled_tags",
fallback=None)
+
+ # Pass config values to shared functions (dependency injection)
+ validator = get_validator(
+ metrics_allow_list=conf.get("metrics", "metrics_allow_list",
fallback=None),
+ metrics_block_list=conf.get("metrics", "metrics_block_list",
fallback=None),
)
- return SafeStatsdLogger(statsd, get_validator(), influxdb_tags_enabled,
metric_tags_validator)
+ metric_tags_validator = PatternBlockListValidator(statsd_disabled_tags)
+ return SafeStatsdLogger(statsd, validator, influxdb_tags_enabled,
metric_tags_validator)
```
Then for use in task sdk, add a similar factory as needed. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
