amirmor1 commented on PR #69855: URL: https://github.com/apache/airflow/pull/69855#issuecomment-5727286012
@VladaZakharova Thanks for the review — both inline points are addressed above, and the system test is in as of the latest push. batch_id_prefix is now exercised by example_dataproc_batch.py: a create_batch_with_prefix task submits with the prefix, and the teardown resolves the generated ID from XCom, since that ID is only known at runtime. The docs no longer carry a hand-written snippet — they exampleinclude that same task, so the documented example is the one that actually runs. Since system tests are excluded from PR CI, I ran it against a live Dataproc Serverless project (Airflow 3.2.1, google provider built from this branch). To demonstrate the retry behaviour rather than just a green run, I cleared create_batch_with_prefix and let it make a second attempt: attempt 1 submitted …-b3678bb9 attempt 2 submitted …-0bf374be Same prefix, different generated suffix, both batches created and completed successfully, and both cleaned up by the delete task. On main that second attempt is precisely the case that fails with ALREADY_EXISTS. Screenshots: the run overview with both tasks green, and the attempt 1 / attempt 2 logs showing the two different generated IDs. <img width="1600" height="1000" alt="batch_id_prefix_grid" src="https://github.com/user-attachments/assets/363d3f4e-93b0-4074-a1c1-ebced6939baa"; /> <img width="1600" height="1000" alt="redacted_attempt1_log" src="https://github.com/user-attachments/assets/3a2eacd7-0615-42b0-b31e-d6af42044819"; /> <img width="1600" height="1000" alt="redacted_attempt2_log" src="https://github.com/user-attachments/assets/0deaf147-9eca-40e7-9c7f-943b1dfdedc0"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
