[
https://issues.apache.org/jira/browse/AIRFLOW-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133927#comment-17133927
]
ASF GitHub Bot commented on AIRFLOW-6294:
-----------------------------------------
mik-laj commented on a change in pull request #8809:
URL: https://github.com/apache/airflow/pull/8809#discussion_r439207592
##########
File path: airflow/providers/google/cloud/example_dags/example_dataflow.py
##########
@@ -108,9 +114,16 @@
)
# [END howto_operator_start_python_job]
+ py_to_local = GCSToLocalOperator(
+ task_id="py-to-local",
+ bucket=GCS_PYTHON_BUCKET_NAME,
+ object_name=GCS_PYTHON_OBJECT_NAME,
+ filename="/tmp/dataflow-{{ ds_nodash }}.py",
+ )
+
start_python_job_local = DataflowCreatePythonJobOperator(
task_id="start-python-job-local",
- py_file='apache_beam.examples.wordcount',
+ py_file='/tmp/dataflow-{{ ds_nodash }}.py',
py_options=['-m'],
Review comment:
If you use the -m option, you should specify the module, not the file
name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Create guide for Dataflow service operator
> ------------------------------------------
>
> Key: AIRFLOW-6294
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6294
> Project: Apache Airflow
> Issue Type: New Feature
> Components: documentation, gcp
> Affects Versions: 1.10.6
> Reporter: Kamil Bregula
> Priority: Minor
> Labels: GoodForNewContributors
>
> Hello,
> A guide that describes how to use Dataflow service perators would be useful.
> Other guides are available:
> https://airflow.readthedocs.io/en/latest/howto/operator/gcp/index.html
> If anyone is interested in this task, I am willing to provide all the
> necessary tips and information.
> Best regards,
> Kamil
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
