tnk-dev opened a new issue #19467:
URL: https://github.com/apache/airflow/issues/19467
### Apache Airflow version
2.2.1 (latest released)
### Operating System
Docker python:3.8-slim image
### Versions of Apache Airflow Providers
```pip freeze
alembic==1.7.4
anyio==3.3.4
apache-airflow==2.2.1
apache-airflow-providers-amazon==1.4.0
apache-airflow-providers-apache-hive==2.0.3
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-http==2.0.1
apache-airflow-providers-imap==2.0.1
apache-airflow-providers-jdbc==2.0.1
apache-airflow-providers-postgres==2.3.0
apache-airflow-providers-sqlite==2.0.1
apache-airflow-providers-ssh==2.3.0
apispec==3.3.2
argcomplete==1.12.3
astroid==2.6.6
atomicwrites==1.4.0
attrs==20.3.0
awscli==1.19.97
Babel==2.9.1
bcrypt==3.2.0
bleach==4.1.0
blinker==1.4
boto3==1.17.97
botocore==1.20.97
cattrs==1.5.0
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.7
click==7.1.2
clickclick==20.10.2
colorama==0.4.3
colorlog==5.0.1
commonmark==0.9.1
croniter==1.0.15
cryptography==35.0.0
defusedxml==0.7.1
dill==0.3.4
dnspython==2.1.0
docutils==0.15.2
email-validator==1.1.3
Flask==1.1.4
Flask-AppBuilder==3.3.4
Flask-Babel==2.0.0
Flask-Caching==1.10.1
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.3.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
future==0.18.2
graphviz==0.18
gunicorn==20.1.0
h11==0.12.0
hmsclient==0.1.1
httpcore==0.13.7
httpx==0.20.0
idna==3.3
importlib-metadata==4.8.1
importlib-resources==5.4.0
inflection==0.5.1
iso8601==1.0.0
isodate==0.6.0
isort==5.10.0
itsdangerous==1.1.0
JayDeBeApi==1.2.3
Jinja2==2.11.3
jmespath==0.10.0
JPype1==1.3.0
jsonschema==3.2.0
lazy-object-proxy==1.6.0
lockfile==0.12.2
Mako==1.1.5
Markdown==3.3.4
MarkupSafe==1.1.1
marshmallow==3.14.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
mccabe==0.6.1
more-itertools==8.10.0
mypy==0.812
mypy-extensions==0.4.3
numpy==1.21.4
openapi-schema-validator==0.1.5
openapi-spec-validator==0.3.1
packaging==21.2
pandas==1.3.4
paramiko==2.8.0
pendulum==2.1.2
pluggy==0.13.1
prison==0.2.1
psutil==5.8.0
psycopg2-binary==2.9.1
pure-sasl==0.6.2
py==1.11.0
pyasn1==0.4.8
pycparser==2.21
Pygments==2.10.0
PyHive==0.6.4
PyJWT==1.7.1
pylint==2.7.4
PyNaCl==1.4.0
pyparsing==2.4.7
pyrsistent==0.18.0
pysftp==0.2.9
pytest==5.2.2
pytest-responses==0.5.0
python-daemon==2.3.0
python-dateutil==2.8.2
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2021.3
pytzdata==2020.1
PyYAML==5.4.1
requests==2.26.0
requests-toolbelt==0.9.1
responses==0.15.0
rfc3986==1.5.0
rich==10.13.0
rsa==4.7.2
s3transfer==0.4.2
sasl==0.3.1
setproctitle==1.2.2
six==1.16.0
sniffio==1.2.0
SQLAlchemy==1.3.23
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.37.9
sshtunnel==0.4.0
swagger-ui-bundle==0.0.9
tableau-api-lib==0.1.14
tabulate==0.8.9
tenacity==8.0.1
termcolor==1.1.0
text-unidecode==1.3
thrift==0.15.0
thrift-sasl==0.4.3
toml==0.10.2
typed-ast==1.4.3
typeguard==2.13.0
typing-extensions==3.10.0.2
unicodecsv==0.14.1
urllib3==1.26.7
watchtower==0.7.3
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
wrapt==1.12.1
WTForms==3.0.0
zipp==3.6.0
```
### Deployment
Virtualenv installation
### Deployment details
Hey Airflow,
while developing new dags etc today without changing any underlying
infrastructure the following happened when running:
```bash
airflow db init
airflow db upgrade
```
Our Dockerfile
```yaml
FROM python:3.8-slim
# Airflow
ARG ENV_FILE
ARG PRIVATE_SSH_KEY_FILE
ENV PIP_VERSION=20.2.4
ENV LANGUAGE="en_US.UTF-8"
ENV LANG="en_US.UTF-8"
ENV LC_ALL="en_US.UTF-8"
ENV LC_CTYPE="en_US.UTF-8"
ENV LC_MESSAGES="en_US.UTF-8"
ENV AWS_DEFAULT_REGION="eu-central-1"
RUN apt-get update -y
RUN apt-get install -y locales
RUN sed -i 's/^# en_US.UTF-8 UTF-8$/en_US.UTF-8 UTF-8/g' /etc/locale.gen \
&& locale-gen \
&& update-locale LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8
ENV AIRFLOW_HOME=/usr/local/around-airflow
RUN mkdir -p ${AIRFLOW_HOME}
COPY requirements.txt ${AIRFLOW_HOME}/requirements.txt
RUN apt-get install -y build-essential ssh jq libffi-dev libssl-dev
libsasl2-dev && \
pip3 install --upgrade pip==${PIP_VERSION}&& \
pip3 install --no-cache -r ${AIRFLOW_HOME}/requirements.txt && \
apt-get purge -y build-essential && \
apt-get autoremove -y --purge && \
apt-get clean
RUN useradd -d ${AIRFLOW_HOME} airflow
COPY scripts ${AIRFLOW_HOME}/scripts
COPY tests ${AIRFLOW_HOME}/tests
COPY generated_airflow.cfg ${AIRFLOW_HOME}/airflow.cfg
COPY dags ${AIRFLOW_HOME}/dags
COPY $ENV_FILE ${AIRFLOW_HOME}/.env
COPY mypy.ini .pylintrc ${AIRFLOW_HOME}/
RUN mkdir -p ${AIRFLOW_HOME}/.ssh
COPY $PRIVATE_SSH_KEY_FILE ${AIRFLOW_HOME}/.ssh/id_rsa
RUN chmod 400 ${AIRFLOW_HOME}/.ssh/id_rsa
COPY .airflowignore ${AIRFLOW_HOME}/.airflowignore
RUN chown -R airflow ${AIRFLOW_HOME}
EXPOSE 8080
USER airflow
WORKDIR ${AIRFLOW_HOME}
ENTRYPOINT ["/bin/bash", "-l", "-c"]
CMD ["bash scripts/entrypoint.sh"]
```
This is unexpected behaviour since we didn't change any pip package versions
nor added any packages requirements.txt
### What happened
```
DB: sqlite:////usr/local/around-airflow/airflow.db
300[2021-11-08 12:27:48,868] {{db.py:674}} INFO - Creating tables
301INFO [alembic.runtime.migration] Context impl SQLiteImpl.
302INFO [alembic.runtime.migration] Will assume non-transactional DDL.
303Traceback (most recent call last):
304 File "/usr/local/bin/airflow", line 8, in <module>
305 sys.exit(main())
306 File "/usr/local/lib/python3.8/site-packages/airflow/__main__.py", line
40, in main
307 args.func(args)
308 File
"/usr/local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in
command
309 return func(*args, **kwargs)
310 File
"/usr/local/lib/python3.8/site-packages/airflow/cli/commands/db_command.py",
line 31, in initdb
311 db.initdb()
312 File "/usr/local/lib/python3.8/site-packages/airflow/utils/db.py", line
549, in initdb
313 upgradedb()
314 File "/usr/local/lib/python3.8/site-packages/airflow/utils/db.py", line
684, in upgradedb
315 command.upgrade(config, 'heads')
316 File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line
320, in upgrade
317 script.run_env()
318 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py",
line 563, in run_env
319 util.load_python_file(self.dir, "env.py")
320 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py",
line 92, in load_python_file
321 module = load_module_py(module_id, path)
322 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py",
line 108, in load_module_py
323 spec.loader.exec_module(module) # type: ignore
324 File "<frozen importlib._bootstrap_external>", line 783, in exec_module
325 File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed
326 File
"/usr/local/lib/python3.8/site-packages/airflow/migrations/env.py", line 108,
in <module>
327 run_migrations_online()
328 File
"/usr/local/lib/python3.8/site-packages/airflow/migrations/env.py", line 102,
in run_migrations_online
329 context.run_migrations()
330 File "<string>", line 8, in run_migrations
331 File
"/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line
851, in run_migrations
332 self.get_context().run_migrations(**kw)
333 File
"/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line
608, in run_migrations
334 for step in self._migrations_fn(heads, self):
335 File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line
309, in upgrade
336 return script._upgrade_revs(revision, rev)
337 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py",
line 439, in _upgrade_revs
338 for script in reversed(list(revs))
339 File
"/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 793,
in iterate_revisions
340 revisions, heads = fn(
341 File
"/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 1393,
in _collect_upgrade_revisions
342 targets: Collection["Revision"] = self._parse_upgrade_target(
343 File
"/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 1193,
in _parse_upgrade_target
344 return self.get_revisions(target)
345 File
"/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 527,
in get_revisions
346 resolved_id, branch_label = self._resolve_revision_number(
347 File
"/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 747,
in _resolve_revision_number
348 self._revision_map
349 File
"/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line
893, in __get__
350 obj.__dict__[self.__name__] = result = self.fget(obj)
351 File
"/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 189,
in _revision_map
352 for revision in self._generator():
353 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py",
line 136, in _load_revisions
354 script = Script._from_filename(self, vers, file_)
355 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py",
line 999, in _from_filename
356 module = util.load_python_file(dir_, filename)
357 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py",
line 92, in load_python_file
358 module = load_module_py(module_id, path)
359 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py",
line 108, in load_module_py
360 spec.loader.exec_module(module) # type: ignore
361 File "<frozen importlib._bootstrap_external>", line 783, in exec_module
362 File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed
363 File
"/usr/local/lib/python3.8/site-packages/airflow/migrations/versions/82b7c48c147f_remove_can_read_permission_on_config_.py",
line 28, in <module>
364 from airflow.www.app import create_app
365 File "/usr/local/lib/python3.8/site-packages/airflow/www/app.py", line
23, in <module>
366 from flask_appbuilder import SQLA
367 File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/__init__.py", line 6,
in <module>
368 from .base import AppBuilder # noqa: F401
369 File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/base.py",
line 8, in <module>
370 from .api.manager import OpenApiManager
371 File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/manager.py", line
7, in <module>
372 from flask_appbuilder.baseviews import BaseView
373 File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/baseviews.py", line
21, in <module>
374 from .forms import GeneralModelConverter
375 File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/forms.py", line 16, in
<module>
376 from .fields import EnumField, QuerySelectField,
QuerySelectMultipleField
377 File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/fields.py", line 6, in
<module>
378 from wtforms.compat import string_types, text_type
379ModuleNotFoundError: No module named 'wtforms.compat'
```
### What you expected to happen
I expected to build airflow as usual
### How to reproduce
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
