MaksYermak commented on PR #69102: URL: https://github.com/apache/airflow/pull/69102#issuecomment-4841618260
> Thanks for the detailed review, @MaksYermak! > > We haven't run the full Dataflow system tests — we only exercised unit tests and the existing beam hook test suite locally. > > Your point about using the existing `is_dataflow_job_id_exist_callback` to also drive the API poll is a cleaner design than introducing a separate `periodic_callback`, and the note about the fd-blocking concern is well taken. > > Since you're close to finishing your own fix with the correct approach, we're happy to close this PR and defer to yours. We'd rather avoid parallel work that heads in incompatible directions. Just let us know if there's anything from this PR you'd find useful to carry forward, or if you'd prefer we simply close it now. @gingeekrishna here is a PR https://github.com/VladaZakharova/airflow/pull/325 with changes. This PR is blocked by https://github.com/apache/airflow/pull/66952 because in the current time `apache-beam` provider is suspended and firstly we need to un-suspend it for being able to run beam and dataflow unit tests, run dataflow system tests, and developing using breeze. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
