yinbiao opened a new issue #12714:
URL: https://github.com/apache/airflow/issues/12714
**Apache Airflow version**:
1.10.13
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
**Environment**:
Apache Airflow [1.10.13]
Platform: [Linux, x86_64] uname_result(system='Linux',
node='yinbiao327-124', release='3.10.0-862.el7.x86_64', version='#1 SMP Fri Apr
20 16:44:24 UTC 2018', machine='x86_64', processor='x86_64')
Locale: ('en_US', 'UTF-8')
Python Version: [3.6.12 (default, Sep 18 2020, 10:09:30) [GCC 4.8.5
20150623 (Red Hat 4.8.5-39)]]
Python Location: [/data/svn/airflow/airflow_develop/venv/bin/python]
git: [git version 1.8.3.1]
ssh: [OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017]
kubectl: [NOT AVAILABLE]
gcloud: [NOT AVAILABLE]
cloud_sql_proxy: [NOT AVAILABLE]
mysql: [NOT AVAILABLE]
sqlite3: [3.7.17 2013-05-20 00:56:22
118a3b35693b134d56ebd780123b7fd6f1497668]
psql: [NOT AVAILABLE]
Airflow Home: [/data/airflow_data]
System PATH:
[/data/svn/airflow/airflow_develop/src/airflow/bin/:/data/svn/airflow/airflow_develop/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/local/node-v10.16.0-linux-x64/bin/:/root/bin]
Python PATH:
[/data/svn/airflow/airflow_develop/src/airflow/bin:/data/svn/airflow/airflow_develop/src:/usr/local/python-3.6.12/lib/python36.zip:/usr/local/python-3.6.12/lib/python3.6:/usr/local/python-3.6.12/lib/python3.6/lib-dynload:/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages:/data/airflow_data/dags:/data/airflow_data/config:/data/airflow_data/plugins]
airflow on PATH: [True]
Executor: [CeleryExecutor]
SQL Alchemy Conn: [mysql+mysqldb://root:[email protected]/pingan_airflow]
DAGS Folder: [/data/airflow_data/dags]
Plugins Folder: [/data/airflow_data/plugins]
Base Log Folder: [/data/airflow_data/logs]
**What happened**:
i use example dags: example_python_operator.py，it has error when i tigger
it:
[2020-11-30 16:27:28,844: INFO/MainProcess] Received task:
airflow.executors.celery_executor.execute_command[97832c08-3a50-4c44-9387-53f535d2d0e9]
[2020-11-30 16:27:28,846: INFO/ForkPoolWorker-16] Executing command in
Celery: ['airflow', 'run', 'example_python_operator', 'print_the_context',
'2020-11-30T08:27:26.021068+00:00', '--pickle', '99485', '--local', '--pool',
'default_pool', '-sd', '/data/airflow_data/dags/example_python_operator.py']
Loading pickle id 99485
Traceback (most recent call last):
File "/data/svn/airflow/airflow_develop/src/airflow/bin/airflow", line 37,
in <module>
args.func(args)
File "/data/svn/airflow/airflow_develop/src/airflow/utils/cli.py", line
80, in wrapper
return f(*args, **kwargs)
File "/data/svn/airflow/airflow_develop/src/airflow/bin/cli.py", line 543,
in run
dag_pickle = session.query(DagPickle).filter(DagPickle.id ==
args.pickle).first()
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py",
line 3397, in first
ret = list(self[0:1])
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py",
line 3171, in __getitem__
return list(res)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py",
line 101, in instances
cursor.close()
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py",
line 69, in __exit__
exc_value, with_traceback=exc_tb,
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py",
line 178, in raise_
raise exception
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py",
line 81, in instances
rows = [proc(row) for row in fetch]
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py",
line 81, in <listcomp>
rows = [proc(row) for row in fetch]
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py",
line 589, in _instance
populators,
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py",
line 726, in _populate_full
dict_[key] = getter(row)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sqlalchemy/sql/sqltypes.py",
line 1710, in process
return loads(value)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/dill/_dill.py",
line 275, in loads
return load(file, ignore, **kwds)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/dill/_dill.py",
line 270, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/dill/_dill.py",
line 473, in load
obj = StockUnpickler.load(self)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/dill/_dill.py",
line 463, in find_class
return StockUnpickler.find_class(self, module, name)
ModuleNotFoundError: No module named
'unusual_prefix_f881c2226b208fc30420f38af1a3121d5507943c_example_python_operator'
[2020-11-30 16:27:31,250: ERROR/ForkPoolWorker-16] execute_command
encountered a CalledProcessError
Traceback (most recent call last):
File
"/data/svn/airflow/airflow_develop/src/airflow/executors/celery_executor.py",
line 78, in execute_command
close_fds=True, env=env)
File "/usr/local/python-3.6.12/lib/python3.6/subprocess.py", line 311, in
check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['airflow', 'run',
'example_python_operator', 'print_the_context',
'2020-11-30T08:27:26.021068+00:00', '--pickle', '99485', '--local', '--pool',
'default_pool', '-sd', '/data/airflow_data/dags/example_python_operator.py']'
returned non-zero exit status 1.
[2020-11-30 16:27:31,252: ERROR/ForkPoolWorker-16] None
[2020-11-30 16:27:31,262: ERROR/ForkPoolWorker-16] Task
airflow.executors.celery_executor.execute_command[97832c08-3a50-4c44-9387-53f535d2d0e9]
raised unexpected: AirflowException('Celery command failed',)
Traceback (most recent call last):
File
"/data/svn/airflow/airflow_develop/src/airflow/executors/celery_executor.py",
line 78, in execute_command
close_fds=True, env=env)
File "/usr/local/python-3.6.12/lib/python3.6/subprocess.py", line 311, in
check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['airflow', 'run',
'example_python_operator', 'print_the_context',
'2020-11-30T08:27:26.021068+00:00', '--pickle', '99485', '--local', '--pool',
'default_pool', '-sd', '/data/airflow_data/dags/example_python_operator.py']'
returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/celery/app/trace.py",
line 412, in trace_task
R = retval = fun(*args, **kwargs)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/celery/app/trace.py",
line 704, in __protected_call__
return self.run(*args, **kwargs)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sentry_sdk/integrations/celery.py",
line 171, in _inner
reraise(*exc_info)
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sentry_sdk/_compat.py",
line 57, in reraise
raise value
File
"/data/svn/airflow/airflow_develop/venv/lib/python3.6/site-packages/sentry_sdk/integrations/celery.py",
line 166, in _inner
return f(*args, **kwargs)
File
"/data/svn/airflow/airflow_develop/src/airflow/executors/celery_executor.py",
line 83, in execute_command
raise AirflowException('Celery command failed')
airflow.exceptions.AirflowException: Celery command failed
**What you expected to happen**:
<!-- What do you think went wrong? -->
**How to reproduce it**:
<!---
As minimally and precisely as possible. Keep in mind we do not have access
to your cluster or dags.
If you are using kubernetes, please attempt to recreate the issue using
minikube or kind.
## Install minikube/kind
- Minikube https://minikube.sigs.k8s.io/docs/start/
- Kind https://kind.sigs.k8s.io/docs/user/quick-start/
If this is a UI bug, please provide a screenshot of the bug or a link to a
youtube video of the bug in action
You can include images using the .md style of
To record a screencast, mac users can use QuickTime and then create an
unlisted youtube video with the resulting .mov file.
--->
**Anything else we need to know**:
<!--
How often does this problem occur? Once? Every time etc?
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
