I like the generic "keyval" provider direction. To build on this discussion, I'm planning two separate experiments:
1. Valkey as Celery backend: Testing if Valkey can drop-in replace Redis as the message broker for Celery executor 2. Valkey with existing Redis provider: Testing if the current apache-airflow-providers-redis works when connected to Valkey instead of Redis These should give us concrete data on both use cases to inform whether a generic "keyval" provider approach makes sense and how seamless the interchangeability really is. On 2025-08-14, 10:54 AM, "Jarek Potiuk" <ja...@potiuk.com <mailto:ja...@potiuk.com>> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque. Ah.. Yes. Sorry - thanks Jed - you are right. I mis-read it.. What my proposal would be is to keep redis to do what it does but have optional `[redis]' (alongside ['rabbitmq'' on celery provider: "apache-airflow-providers-celery[redis]` - and it would **not** install redis but install redis as dependency. And also what we could add is a combo "celery-redis" that would install "apache-airflow-providers-celery[redis]". However - I would propose something else to keep the consistency. Why don't we already rename redis provider to "keyval" or something else and have it work for both redis and valkey ? Even if we do not want to implement anything specific for "valkey". For me that would be quite a good solution for now - no "two" providers separately - just one for now. THEN we could indeed change "redis" dependency to point to just redis dependencies - as there will be no "redis" provider. J. On Thu, Aug 14, 2025 at 7:24 PM Jed Cunningham <jedcunning...@apache.org <mailto:jedcunning...@apache.org>> wrote: > It feels like a slippery slope to depart from the "apache-airflow[x]" -> > "apache-airflow-providers-x" pattern when x is a valid provider name. Right > now it's easy to know what it'll do before you run, but if we did this > you'd either have to look it up or try it. >
