mik-laj commented on a change in pull request #12548:
URL: https://github.com/apache/airflow/pull/12548#discussion_r529522345
##########
File path: setup.py
##########
@@ -658,9 +655,10 @@ def write_version(filename: str = os.path.join(*[my_dir,
"airflow", "git_version
EXTRAS_PROVIDERS_PACKAGES: Dict[str, Iterable[str]] = {
'all': list(PROVIDERS_REQUIREMENTS.keys()),
- # this is not 100% accurate with devel_ci definition, but we really want
to have all providers
- # when devel_ci extra is installed!
+ # this is not 100% accurate with devel_ci and devel_all definition, but we
really want
+ # to have all providers when devel_ci extra is installed!
'devel_ci': list(PROVIDERS_REQUIREMENTS.keys()),
+ 'devel_all': list(PROVIDERS_REQUIREMENTS.keys()),
Review comment:
We don't import, but run Python with Beam as a subprocess. By default,
this is the same interpreter as Airflow.
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/airflow/providers/google/cloud/operators/dataflow.py#L755
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/airflow/providers/google/cloud/operators/dataflow.py#L941
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/airflow/providers/google/cloud/hooks/dataflow.py#L818
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/airflow/providers/google/cloud/hooks/dataflow.py#L894
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/airflow/providers/google/cloud/hooks/dataflow.py#L569-L570
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/airflow/providers/google/cloud/hooks/dataflow.py#L437
It works similar to a pyspark.
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/airflow/providers/apache/spark/hooks/spark_submit.py#L418
https://github.com/apache/airflow/blob/936566c586e6cbb155ffa541e89a31f7239f51bb/setup.py#L388
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
