Hello! I am a very new engineer at my company and I have been asked to set up Airflow with Celery. We have existing Redis resources for the message broker, but I am not allowed to connect to redis directly, I must do it through redis-sentinel. Celery 4 can use redis-sentinel but I don't see how to pass the configuration through Airflow—you have to send it a dictionary called 'broker_transport_options' with the name of the Redis service you are looking for. Has anybody gotten the CeleryExecutor working with redis-sentinel? My teammate says I should contribute a PR to Airflow (which I am happy to try) but I want to check in case anybody has suggestions. Thank you!
-- Desiree Cox Associate Software Engineer Zendesk
