viragtripathi commented on PR #69555: URL: https://github.com/apache/airflow/pull/69555#issuecomment-5054210061
> 2.5k TI rows is frankly nothing. This approach (of needing to iterate over every TI row in python) is not suitable. > > I wonder if the ALTER PRIMARY KEY approach performs better on postgres too? Fair point, the python loop was the weak part. Reworked: the branch now creates a plain SQL function that assembles the v7 hex layout (same technique as the mysql branch above) and backfills server side with batched UPDATE ... LIMIT, no per-row round trips. 50k rows take about 3 seconds on a single node; the LIMIT is only there to bound transaction sizes. On whether ALTER PRIMARY KEY would help postgres too: that statement is cockroachdb-specific, postgres has no atomic primary key swap, so the existing drop-and-recreate there is already the only option as far as I know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
