[
https://issues.apache.org/jira/browse/AIRFLOW-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maximilian Roos updated AIRFLOW-1758:
-------------------------------------
Description:
Currently when there is a failure during a run, it's difficult to see what the
cause was. Could we at least print the python stack trace?
As an example:
{code:python}
[2017-10-26 21:43:38,155] {models.py:1563} ERROR - DataFlow failed with return
code 1
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1461,
in _run_raw_task
result = task_copy.execute(context=context)
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/dataflow_operator.py",
line 192, in execute
self.py_file, self.py_options)
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
line 155, in start_python_dataflow
task_id, variables, dataflow, name, ["python"] + py_options)
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
line 141, in _start_dataflow
_Dataflow(cmd).wait_for_done()
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
line 122, in wait_for_done
self._proc.returncode))
Exception: DataFlow failed with return code 1
{code}
I then need to jump into a repl and attempt simulate the command that airflow
would have run, which is both difficult and error prone. (Or is there a simpler
way of doing this??)
I then get a better stack-trace:
{code:python}
File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line
328, in run
return self.runner.run(self)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/dataflow_runner.py",
line 283, in run
self.dataflow_client.create_job(self.job), self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py",
line 168, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py",
line 423, in create_job
self.create_job_description(job)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py",
line 446, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py",
line 347, in stage_job_resources
build_setup_args)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py",
line 439, in _build_setup_package
os.chdir(os.path.dirname(setup_file))
OSError: [Errno 2] No such file or directory: ''
{code}
Somewhat related to: https://issues.apache.org/jira/browse/AIRFLOW-174
(I'm using the DataFlowPythonOperator at the moment, but I suspect the issue is
wider)
was:
Currently when there is a failure during a run, it's difficult to see what the
cause was. Could we at least print the python stack trace?
As an example:
```
[2017-10-26 21:43:38,155] {models.py:1563} ERROR - DataFlow failed with return
code 1
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1461,
in _run_raw_task
result = task_copy.execute(context=context)
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/dataflow_operator.py",
line 192, in execute
self.py_file, self.py_options)
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
line 155, in start_python_dataflow
task_id, variables, dataflow, name, ["python"] + py_options)
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
line 141, in _start_dataflow
_Dataflow(cmd).wait_for_done()
File
"/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
line 122, in wait_for_done
self._proc.returncode))
Exception: DataFlow failed with return code 1
```
I then need to jump into a repl and attempt simulate the command that airflow
would have run, which is both difficult and error prone. (Or is there a simpler
way of doing this??)
I then get a better stack-trace:
```
File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line
328, in run
return self.runner.run(self)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/dataflow_runner.py",
line 283, in run
self.dataflow_client.create_job(self.job), self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py",
line 168, in wrapper
return fun(*args, **kwargs)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py",
line 423, in create_job
self.create_job_description(job)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py",
line 446, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py",
line 347, in stage_job_resources
build_setup_args)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py",
line 439, in _build_setup_package
os.chdir(os.path.dirname(setup_file))
OSError: [Errno 2] No such file or directory: ''
```
Somewhat related to: https://issues.apache.org/jira/browse/AIRFLOW-174
(I'm using the DataFlowPythonOperator at the moment, but I suspect the issue is
wider)
> Print full traceback on errors
> ------------------------------
>
> Key: AIRFLOW-1758
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1758
> Project: Apache Airflow
> Issue Type: Improvement
> Components: cli
> Reporter: Maximilian Roos
> Priority: Minor
>
> Currently when there is a failure during a run, it's difficult to see what
> the cause was. Could we at least print the python stack trace?
> As an example:
> {code:python}
> [2017-10-26 21:43:38,155] {models.py:1563} ERROR - DataFlow failed with
> return code 1
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1461,
> in _run_raw_task
> result = task_copy.execute(context=context)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/dataflow_operator.py",
> line 192, in execute
> self.py_file, self.py_options)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
> line 155, in start_python_dataflow
> task_id, variables, dataflow, name, ["python"] + py_options)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
> line 141, in _start_dataflow
> _Dataflow(cmd).wait_for_done()
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py",
> line 122, in wait_for_done
> self._proc.returncode))
> Exception: DataFlow failed with return code 1
> {code}
> I then need to jump into a repl and attempt simulate the command that airflow
> would have run, which is both difficult and error prone. (Or is there a
> simpler way of doing this??)
> I then get a better stack-trace:
> {code:python}
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line
> 328, in run
> return self.runner.run(self)
> File
> "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/dataflow_runner.py",
> line 283, in run
> self.dataflow_client.create_job(self.job), self)
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py",
> line 168, in wrapper
> return fun(*args, **kwargs)
> File
> "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py",
> line 423, in create_job
> self.create_job_description(job)
> File
> "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py",
> line 446, in create_job_description
> job.options, file_copy=self._gcs_file_copy)
> File
> "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py",
> line 347, in stage_job_resources
> build_setup_args)
> File
> "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py",
> line 439, in _build_setup_package
> os.chdir(os.path.dirname(setup_file))
> OSError: [Errno 2] No such file or directory: ''
> {code}
> Somewhat related to: https://issues.apache.org/jira/browse/AIRFLOW-174
> (I'm using the DataFlowPythonOperator at the moment, but I suspect the issue
> is wider)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
