mik-laj commented on pull request #9122: URL: https://github.com/apache/airflow/pull/9122#issuecomment-638321505
I contacted the Cloud Dataflow product team. We should use the following regular expressions. > for java and go: Submitted job: (.*) > for python: Created job with id: \[(.*)\] Searching for Job ID by URL is not a good idea. Can you check my proposition? Moreover, we should prepare system tests that will check if these two versions are still working properly. For Python, this means adding a new task in the sample DAG with a different version. https://github.com/apache/airflow/blob/1c9374d2573483dd66f5c35032e24140864e72c0/airflow/providers/google/cloud/example_dags/example_dataflow.py#L120 For Java, we need to prepare two JAR files. We can do it with Cloud Build so that everyone can easily access it. Here is example: https://github.com/apache/airflow/blob/aa29389cac91f29f5f37b268fa61c0a3eaabd663/tests/providers/google/cloud/operators/test_dataflow_system.py If you don't have time, then you don't have to do system tests for Java, but I would be very happy. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
