potiuk commented on a change in pull request #20847:
URL: https://github.com/apache/airflow/pull/20847#discussion_r783720678
##########
File path: airflow/www/gunicorn_config.py
##########
@@ -37,3 +42,18 @@ def on_starting(server):
# Load providers before forking workers
ProvidersManager().connection_form_widgets
+
+
+def post_fork(server, worker):
+ """
+ Needed to workoround open-telemetry not working well with gunicorn fork
model.
+
+ See more:
https://opentelemetry-python.readthedocs.io/en/latest/examples/fork-process-model/README.html
+ """
+ server.log.info("Worker spawned (pid: %s)", worker.pid)
+
+ resource = Resource.create(attributes={"service.name": "airflow-service"})
+
+ trace.set_tracer_provider(TracerProvider(resource=resource))
+ span_processor = BatchSpanProcessor(JaegerExporter())
Review comment:
This one is a bit tricky - because we have to figure out which exporter
to start. Currently we only use Jaeger and I "hard-coded" it there, but Ideally
we should take `OTEL_TRACES_EXPORTER` environment variable and run the right
exporter depending on the value of this variable.
And I think we should simply look how opentelemetry-python does it.
It does not seem complex:
https://github.com/open-telemetry/opentelemetry-python/blob/8d4a804b49ac36d1cc629931960168dacde4d2ab/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py#L175
We should be able to copy part of the code that retrieves and initializes
the exporter from there.
That might be a nice exercise for you to do some more python coding here :).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
