rafalh commented on code in PR #28970:
URL: https://github.com/apache/airflow/pull/28970#discussion_r1081250709
##########
tests/providers/google/cloud/hooks/test_dataproc.py:
##########
@@ -1006,17 +1006,9 @@ def test_set_python_main(self):
self.builder.set_python_main(main)
assert main ==
self.builder.job["job"][self.job_type]["main_python_file_uri"]
- @pytest.mark.parametrize(
- "job_name",
- [
- pytest.param("name", id="simple"),
- pytest.param("name_with_dash", id="name with underscores"),
- pytest.param("group.name", id="name with dot"),
- pytest.param("group.name_with_dash", id="name with dot and
underscores"),
- ],
- )
Review Comment:
@eladkal I moved the replacement to `DataprocJobBaseOperator` default value
for `job_name` and added another test that checks if `job_name` generated in
`DataprocJobBaseOperator` has dots replaced by `_` so that issue should still
be fixed. That issue explicitly says that it's the default job name being
broken:
>
[DataprocJobBaseOperator](https://github.com/apache/airflow/blob/05ccfd42f28db7d0a8fe3ed023b0e7a8ec188609/airflow/providers/google/cloud/operators/dataproc.py#L836-L838)
have default of using task_id for job name
And that's exactly what this PR fixes, letting the people who use
`DataProcJobBuilder` directly to pass any job name they want including
templates containing dots.
##########
tests/providers/google/cloud/hooks/test_dataproc.py:
##########
@@ -1006,17 +1006,9 @@ def test_set_python_main(self):
self.builder.set_python_main(main)
assert main ==
self.builder.job["job"][self.job_type]["main_python_file_uri"]
- @pytest.mark.parametrize(
- "job_name",
- [
- pytest.param("name", id="simple"),
- pytest.param("name_with_dash", id="name with underscores"),
- pytest.param("group.name", id="name with dot"),
- pytest.param("group.name_with_dash", id="name with dot and
underscores"),
- ],
- )
Review Comment:
@eladkal I moved the replacement to `DataprocJobBaseOperator` default value
for `job_name` and added another test that checks if `job_name` generated in
`DataprocJobBaseOperator` has dots replaced by `_` so that issue should still
be fixed. That issue explicitly says that it's the default job name being
broken:
>
[DataprocJobBaseOperator](https://github.com/apache/airflow/blob/05ccfd42f28db7d0a8fe3ed023b0e7a8ec188609/airflow/providers/google/cloud/operators/dataproc.py#L836-L838)
have default of using task_id for job name
And that's exactly what this PR fixes, letting the people who use
`DataProcJobBuilder` directly to pass any job name they want including
templates containing dots.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
