ImadYIdrissi opened a new issue #8541:
URL: https://github.com/apache/airflow/issues/8541
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
**Apache Airflow version**:1.10.2
**Kubernetes version (if you are using kubernetes)** (use `kubectl
version`): 1.14.10-gke.27
**Environment**: GC Composer
- **Cloud provider or hardware configuration**: GCP
- **OS** (e.g. from /etc/os-release):
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**: The "clear" option crashes the UI.
**error messages**: TypeError: can't pickle _thread.RLock objects
**What you expected to happen**: I expect airflow to clear the task and its
dependencies, then re-run the task.
**What do you think went wrong?**: When I try to include a
`BigQueryOperator` from `airflow.contrib.operators.bigquery_operator` in my
DAG, the error occurs.
**How to reproduce it**: I am not sure how to reproduce this error in any
other environment. Please find the video explaining the bug in the following
link :
**How often does this problem occur?** Every time I try to add the
`BigQueryOperator` to this particular DAG.
<details><summary>UI Crash Logs</summary> Ooops.
____/ ( ( ) ) \___
/( ( ( ) _ )) ) )\
(( ( )( ) ) ( ) )
((/ ( _( ) ( _) ) ( () ) )
( ( ( (_) (( ( ) .((_ ) . )_
( ( ) ( ( ) ) ) . ) ( )
( ( ( ( ) ( _ ( _) ). ) . ) ) ( )
( ( ( ) ( ) ( )) ) _)( ) ) )
( ( ( \ ) ( (_ ( ) ( ) ) ) ) )) ( )
( ( ( ( (_ ( ) ( _ ) ) ( ) ) )
( ( ( ( ( ) (_ ) ) ) _) ) _( ( )
(( ( )( ( _ ) _) _(_ ( (_ )
(_((__(_(__(( ( ( | ) ) ) )_))__))_)___)
((__) \\||lll|l||/// \_))
( /(/ ( ) ) )\ )
( ( ( ( | | ) ) )\ )
( /(| / ( )) ) ) )) )
( ( ((((_(|)_))))) )
( ||\(|(|)|/|| )
( |(||(||)|||| )
( //|/l|||)|\\ \ )
(/ / // /|//||||\\ \ \ \ _)
-------------------------------------------------------------------------------
Node: 788ee3e5c207
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/opt/python3.6/lib/python3.6/site-packages/flask/app.py", line 1982,
in wsgi_app
response = self.full_dispatch_request()
File "/opt/python3.6/lib/python3.6/site-packages/flask/app.py", line 1614,
in full_dispatch_request
rv = self.handle_user_exception(e)
File "/opt/python3.6/lib/python3.6/site-packages/flask/app.py", line 1517,
in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/opt/python3.6/lib/python3.6/site-packages/flask/_compat.py", line
33, in reraise
raise value
File "/opt/python3.6/lib/python3.6/site-packages/flask/app.py", line 1612,
in full_dispatch_request
rv = self.dispatch_request()
File "/opt/python3.6/lib/python3.6/site-packages/flask/app.py", line 1598,
in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/python3.6/lib/python3.6/site-packages/flask_admin/base.py",
line 69, in inner
return self._run_view(f, *args, **kwargs)
File "/opt/python3.6/lib/python3.6/site-packages/flask_admin/base.py",
line 368, in _run_view
return fn(self, *args, **kwargs)
File "/opt/python3.6/lib/python3.6/site-packages/flask_login/utils.py",
line 258, in decorated_view
return func(*args, **kwargs)
File "/usr/local/lib/airflow/airflow/www/utils.py", line 281, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/airflow/airflow/www/utils.py", line 328, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/airflow/airflow/www/views.py", line 1166, in clear
include_upstream=upstream)
File "/usr/local/lib/airflow/airflow/models.py", line 4166, in sub_dag
dag = copy.deepcopy(self)
File "/opt/python3.6/lib/python3.6/copy.py", line 161, in deepcopy
y = copier(memo)
File "/usr/local/lib/airflow/airflow/models.py", line 4151, in __deepcopy__
setattr(result, k, copy.deepcopy(v, memo))
File "/opt/python3.6/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 161, in deepcopy
y = copier(memo)
File "/usr/local/lib/airflow/airflow/models.py", line 2874, in __deepcopy__
setattr(result, k, copy.deepcopy(v, memo))
File "/opt/python3.6/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/opt/python3.6/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/opt/python3.6/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/opt/python3.6/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 215, in _deepcopy_list
append(deepcopy(a, memo))
File "/opt/python3.6/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/opt/python3.6/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/opt/python3.6/lib/python3.6/copy.py", line 169, in deepcopy
rv = reductor(4)
TypeError: can't pickle _thread.RLock objects</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
