Hi Jarek, Understood. That's a clear picture of the trade-offs from the community's side, and I'll work out the right path from here.
Best, Virag On Sat, Jul 11, 2026 at 1:57 PM Jarek Potiuk <[email protected]> wrote: > > I do want to gently correct one thing, because it matters. This did not > start as a commercial play. I opened the original discussion as an > individual contributor, following the exact path you recommended, which was > to raise it on the dev list before writing code > > I perfectly understand that—my intention was not to paint your effort > as a "commercial venture hidden as an OSS contribution". If you read > it this way - this was not my intention.\ > > But we had one very good example from the past where a similar path > burned us. We have MsSQL support added - by individual contributor - > over an extended period (more than 6 months) - and while it was a > substantially more complex change, it's been also better testable - > because we could - and we did - run all the unit tests and integration > tests with MsSQL. While it seemed to be working fine, it backfired > significantly. We've always explained it as "experimental," which > allowed us to remove it eventually. However, it caused so many > problems and slowed us down so much that we had to drop it, even > though some users strongly wanted us to keep MsSQL support, and we had > to tell them "no". > > Having Cockroach DB "supported" by Airflow means that we literally > have to test **every single PR** that touches DB with Cocroach DB. > This is quite impossible to do in our CI, even with a generous in-kind > donation from Cockroach Labs, purely because of the additional > complexity involved. Furthermore, any behaviour different from regular > Postgres might require additional testing effort and extra re-testing > time and some of those issues might silently accumulate over time. Any > of our contributors (we have 1000s of them) when trying to add a > database feature involving migration, or possible performance or > behaviour penalty would have to be able to run it against CocroachDB - > no exception, which means that if we go that direction CocroachLabs > would have to take on their shoulders management and disctribution of > access to CocroachDB for - possibly - big number of contributoirs - we > have **no** time and **noone** who would be willing to spend their > volunteer time for communication and setup of this. > > And every time we prepare a minor release it means that people - > mostly Astronomer folks - have to run 50% more of the complete > integration test suite. They already run Postgres and MySQL suites > every time we make a minor release. > > And no - we cannot "skip" that part. The moment our users start > thinking "Community version supports CocroachDB" - and start using it > regardless—they will look to us to fix any resulting issues. > > So - it's not that i suspect you wanted to "sneak" something. It's > just that there is a real cost in volunteer time, and Astronomer's > time that needs to be spent "extra"—and it's not a small one. And it's > very difficult to balance that time with any donation or money - > because time is something we can only lose; we cannot regain it, > especially volunteer time. > > So this is really pure economy: > > Case A: Cocroach DN changes in community: > > * Cocroach Labs - gains potential customers > * Community - looses time for testing - even if helpd and with > donations from CocroachLabs > > Case B: Cocroach Labs apply the changes and releases their fork: > > * Cocroach Labs - gains potential customers > * Cocroach Labs - bears the same costs as community woudl (re) > applying patches and testing and solving CocroachDB issues > * Community does not loose time > > It's really pure math - nothing personal or "sneaky". Case A simply > does not add up. > > J. > > > On Sat, Jul 11, 2026 at 6:37 PM Virag Tripathi via dev > <[email protected]> wrote: > > > > Hi Ash, > > > > Thanks for the quick and candid response, I genuinely prefer a clear > answer > > to a maybe. > > > > One thing worth separating out. #69556 has no CockroachDB code in it at > > all. It is a pure PostgreSQL fix for the scheduler crashing under > > SERIALIZABLE, handling SQLSTATE 40001 and 40P01 in the critical section > the > > same way 55P03 is already handled. I'd love for that one to be judged on > > its own PostgreSQL merits rather than folded into the dialect discussion. > > > > Best, Virag > > > > On Thu, Jul 9, 2026 at 11:25 AM Ash Berlin-Taylor <[email protected]> > wrote: > > > > > Is there anyone other than an employee of Cockroach Labs who is asking > for > > > this? > > > > > > The Airflow open source project isn’t in a position to support or test > > > Cockroach so _if_ (and it is still very much up for debate if we do, > much > > > like the last time this came up) we merge these, CockroachDB will be an > > > explicitly unsupported DB, and any issues about it’s support on the > > > apache/airflow repo will be closed as unsupported. And any large PRs > that > > > make general maintenance more complicated to support it will be > similarly > > > closed. > > > > > > For 69260 I’m tempted to say “no” too, even though it’s small, as users > > > can already explicitly set the async connection string, and by having > > > partial support like this in Airflow might give the impression that we > (the > > > project) do support CockroachDB when we are not going to. > > > > > > > > > Sorry to be a grouch about this; the hardest thing a maintainer of a > > > truly open source project has to do is say no to PRs. > > > > > > -ash > > > > > > > On 7 Jul 2026, at 16:35, Virag Tripathi via dev < > [email protected]> > > > wrote: > > > > > > > > Hi all, > > > > > > > > > > > > Following up on the proposal from April, the changes are now open as > PRs: > > > > > > > > 1. https://github.com/apache/airflow/pull/69260 (derive the async > > > > database URI for cockroachdb connection strings) > > > > 2. https://github.com/apache/airflow/pull/69555 (make the > > > task_instance > > > > UUID migration work without pgcrypto) > > > > 3. https://github.com/apache/airflow/pull/69556 (survive > serialization > > > > conflicts in the scheduler critical section; this is the > substantive > > > one > > > > and includes a behavior improvement for PostgreSQL under > SERIALIZABLE) > > > > 4. https://github.com/apache/airflow/pull/69557 (extend the > > > scheduler's > > > > HA advisory lock coordination to the cockroachdb dialect, which > > > implements > > > > the same transaction-level advisory locks as of CockroachDB 26.3) > > > > > > > > Each is independent and intentionally small. Happy to adjust scope or > > > > framing based on any comments here or on the PRs. > > > > > > > > > > > > Thanks & regards, > > > > > > > > Virag > > > > > > > > On Tue, May 19, 2026 at 8:55 PM Virag Tripathi < > > > > [email protected]> wrote: > > > > > > > >> Hi all, > > > >> > > > >> Friendly bump on this thread. It's been about a month since I > posted the > > > >> original proposal and I haven't seen any replies, so I wanted to > check > > > if > > > >> the framing landed okay or if there are concerns I should address > > > >> differently. > > > >> > > > >> Quick update since the original post: > > > >> > > > >> - The sqlalchemy-cockroachdb 2.0.4 release shipped with the > > > >> @compiles(timestampdiff, "cockroachdb") fix (PR #301), so the > > > timestampdiff > > > >> issue no longer requires any change in Airflow itself. That's one > fewer > > > >> thing to worry about. > > > >> > > > >> - The POC repo ( > > > https://github.com/viragtripathi/cockroachdb-airflow-poc) > > > >> is public and running end-to-end on Airflow 3.2.1 + CockroachDB > v25.4 > > > LTS. > > > >> 68 tables migrated, DAG parsing, scheduling, task execution all > working. > > > >> Anyone is welcome to spin it up and poke at it. > > > >> > > > >> To recap, the three proposals are: > > > >> > > > >> 1. Generic 40001 serialization-failure retry in the scheduler (fix > > > >> retry_db_transaction to roll back on any DBAPIError, not just > > > >> OperationalError). This one benefits PostgreSQL users under high > > > >> concurrency too. > > > >> > > > >> 2. Add "cockroachdb": "asyncpg" to the async-driver mapping in > > > >> settings.py. One-line change. > > > >> > > > >> 3. Dialect-aware UUID generation in migration 0042 so non-postgres > > > >> PG-compatible dialects can supply their own UUID generator. > Migration > > > runs > > > >> once, no impact on existing PG installs. > > > >> > > > >> Jarek, since you originally suggested bringing this to the devlist > in > > > >> discussion #65453, I'd especially value your take on whether these > are > > > >> scoped right. Happy to adjust the framing or break anything down > > > further. > > > >> > > > >> I'm ready to draft the PRs as soon as there's maintainer signal > that the > > > >> direction is acceptable. Even a "these look fine, go ahead" or > "change X > > > >> before you PR" would be really helpful. > > > >> > > > >> Thanks for your time. > > > >> > > > >> Virag Tripathi > > > >> > > > >> > > > >> [1] https://github.com/apache/airflow/discussions/65453 > > > >> [2] https://github.com/viragtripathi/cockroachdb-airflow-poc > > > >> [3] https://github.com/cockroachdb/sqlalchemy-cockroachdb/pull/301 > > > >> > > > >> On Sat, Apr 18, 2026 at 10:59 AM Virag Tripathi < > > > >> [email protected]> wrote: > > > >> > > > >>> Hi all, > > > >>> > > > >>> > > > >>> Per Jarek's suggestion in GitHub discussion #65453 [1], I'm > bringing > > > this > > > >>> proposal to the devlist before opening PRs. The full long-form > post is > > > on > > > >>> the discussion; this is a tighter summary for devlist review. > > > >>> > > > >>> > > > >>> I'm working on improving Airflow 3.x's behavior on > > > PostgreSQL-compatible > > > >>> databases (currently focused on CockroachDB, which is wire-protocol > > > >>> compatible with PG). Three small, generically-justifiable > improvements > > > have > > > >>> come out of that work. I want maintainer signal on the proposal > before > > > >>> drafting PRs. > > > >>> > > > >>> > > > >>> == Background (skip if uninterested) == > > > >>> > > > >>> - Integration POC validates Airflow 3.2.0 against CockroachDB > v25.4: > > > >>> airflow db migrate succeeds, DAG parsing/scheduling/task > execution > > > all > > > >>> work, example DAGs run end-to-end. 111 Alembic migrations > audited, 12 > > > >>> findings, all handled. > > > >>> - A patch for sqlalchemy-cockroachdb [3] registers > > > >>> @compiles(timestampdiff, "cockroachdb") so MySQL-style > > > >>> func.timestampdiff(unit, start, end) (the fallback used in > > > >>> airflow-core/src/airflow/models/taskinstance.py and dagrun.py for > > > >>> non-PG/non-SQLite dialects) compiles to a PostgreSQL-style > > > EXTRACT(EPOCH > > > >>> FROM ...) expression. This means Airflow itself needs no change > for > > > the > > > >>> timestampdiff issue once the dialect ships 2.0.4+. > > > >>> > > > >>> So the items below are the only ones that genuinely require small > > > >>> upstream changes. > > > >>> > > > >>> > > > >>> *== Proposed change 1 — Generic 40001 serialization-failure retry > in > > > the > > > >>> scheduler ==* > > > >>> > > > >>> Files: airflow-core/src/airflow/utils/retries.py and > > > >>> airflow-core/src/airflow/jobs/scheduler_job_runner.py > > > >>> > > > >>> Why this is generic, not CRDB-specific: PostgreSQL also raises > > > >>> SerializationFailure (SQLSTATE 40001) under high concurrency on > > > >>> SELECT ... FOR NO KEY UPDATE ... SKIP LOCKED. Today Airflow's > scheduler > > > >>> does not retry these reliably. Issue #40882 [4] (closed) noted that > > > >>> retry_db_transaction only issues session.rollback() for > > > >>> OperationalError, not all DBAPIError subclasses, leaving subsequent > > > retries > > > >>> to fail with InFailedSqlTransaction. > > > >>> > > > >>> Proposal: > > > >>> > > > >>> 1. Fix retry_db_transaction to roll back on any DBAPIError, not > just > > > >>> OperationalError. > > > >>> 2. Apply it (or an equivalent retry loop) to > > > >>> _critical_section_enqueue_task_instances. > > > >>> 3. Use exponential backoff and bound the number of retries. > > > >>> > > > >>> Risk: Medium — touches the scheduler hot path. Should be guarded by > > > tests > > > >>> that fault-inject 40001 and assert progress. Benefit: Improves PG > > > >>> resilience under concurrent scheduler workloads. Side benefit: > makes > > > the > > > >>> scheduler usable on any PG-compatible DB whose isolation model > surfaces > > > >>> 40001. > > > >>> > > > >>> This is the largest of the three changes and the one I most want a > > > steer > > > >>> on. > > > >>> > > > >>> > > > >>> *== Proposed change 2 — Add cockroachdb to the async-driver > mapping ==* > > > >>> > > > >>> File: airflow-core/src/airflow/settings.py:240 > > > >>> > > > >>> def _get_async_conn_uri_from_sync(sync_uri): AIO_LIBS_MAPPING = > > > {"sqlite": > > > >>> "aiosqlite", "postgresql": "asyncpg", "mysql": "aiomysql"} ... > > > >>> > > > >>> Proposal: Add "cockroachdb": "asyncpg" to the mapping. > > > >>> > > > >>> Why standalone-justifiable: This function exists to derive an > async URI > > > >>> when one isn't explicitly configured. Today, anyone using a > > > >>> cockroachdb:// SQLAlchemy URL (the canonical scheme registered by > > > >>> sqlalchemy-cockroachdb since 2017) hits "The asyncio extension > > > requires an > > > >>> async driver. The loaded 'psycopg2' is not async." The workaround > is > > > >>> setting AIRFLOW__DATABASE__SQL_ALCHEMY_CONN_ASYNC explicitly. > One-line > > > >>> fix; no behavior change for postgres/mysql/sqlite users. > > > >>> > > > >>> Risk: Trivial. Touches one dict literal. > > > >>> > > > >>> > > > >>> *== Proposed change 3 — Dialect-aware UUID generation in migration > > > >>> 0042 ==* > > > >>> > > > >>> > > > >>> File: > > > > airflow-core/src/airflow/migrations/versions/0042_3_0_0_add_uuid_primary_key_to_task_instance_.py > > > >>> > > > >>> The migration creates a custom uuid_generate_v7() SQL function via > > > CREATE > > > >>> EXTENSION pgcrypto. CockroachDB doesn't expose pgcrypto extensions > but > > > >>> offers native gen_random_uuid(). > > > >>> > > > >>> Proposal: Add a dialect-name check at the top of the upgrade so > > > >>> non-postgres PG-compatible dialects can supply their own UUID > > > >>> generator (defaulting to gen_random_uuid() for cockroachdb). PG > users > > > see > > > >>> no change. The fallback uses v4 UUIDs (not v7) on the alternate > > > >>> dialect, which is documented as a tradeoff. > > > >>> > > > >>> Risk: Low. Migration runs once, dialect-name-guarded, no impact on > > > >>> existing PG installs. > > > >>> > > > >>> > > > >>> == What I'm asking == > > > >>> > > > >>> 1. Are these acceptable as standalone PRs against main? > > > >>> 2. Any framing or scoping changes you'd like before I draft them? > > > >>> 3. For change 1 specifically: any preference on where the retry > logic > > > >>> lives (decorator vs. explicit loop in the scheduler call site)? > > > >>> > > > >>> == Disclosure == > > > >>> > > > >>> I work at Cockroach Labs. The integration POC and > > > >>> sqlalchemy-cockroachdb#301 are linked below for full transparency. > > > Happy to > > > >>> share any additional context. > > > >>> > > > >>> > > > >>> Thanks for reading. > > > >>> > > > >>> Virag Tripathi > > > >>> > > > >>> > > > >>> [1] https://github.com/apache/airflow/discussions/65453 > > > >>> > > > >>> [2] https://github.com/apache/airflow/issues/46175 > > > >>> > > > >>> [3] https://github.com/cockroachdb/sqlalchemy-cockroachdb/pull/301 > > > >>> > > > >>> [4] https://github.com/apache/airflow/issues/40882 > > > >>> > > > >> > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > >
