-1 (binding) Sorry Bolke for not checking this earlier. In rc3 we've replaced some of the reserved keywords. But still I'm unable to run a simple dag in the 1.10 rc4 release under Python 3.7:
MacBook-Pro-van-Fokko:sdh-api-pobt fokkodriesprong$ docker run -e SLUGIFY_USES_TEXT_UNIDECODE=yes -t -i python:3.7 /bin/bash -c "pip install https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc4/apache-airflow-1.10.0rc4+incubating-bin.tar.gz && airflow initdb && airflow run example_bash_operator runme_0 2017-07-01" Collecting https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc4/apache-airflow-1.10.0rc4+incubating-bin.tar.gz Downloading https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc4/apache-airflow-1.10.0rc4+incubating-bin.tar.gz (4.4MB) 100% |████████████████████████████████| 4.4MB 2.5MB/s Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-req-build-91ci7xlu/setup.py", line 124 async = [ ^ SyntaxError: invalid syntax ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-91ci7xlu/ You are using pip version 10.0.1, however version 18.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command. I think we should cherry-pick these three commits on to 1.10 branch: *- Remove the async from setup.py* https://github.com/apache/incubator-airflow/commit/e38a4e5d3064980abd10b8afa6918ab9f10dd8a2 *- Upgrade lxml to >4.0 to let it compile with Python 3.7* https://github.com/apache/incubator-airflow/commit/5290688ee0576ad167d9622c96cdeb08e9965a20 lxml is needed for Python 3.7: https://github.com/apache/incubator-airflow/pull/3583 *- Bump tenacy from 4.8.0 to 4.12.0 * https://github.com/apache/incubator-airflow/pull/3723/commits/271ea663df72c16aa105017ed5cc87a639846777 The 4.8 version of Tenacy contains reserved keywords https://github.com/apache/incubator-airflow/pull/3723 After this I'm able to run an example dag using Python3.7: docker run -e SLUGIFY_USES_TEXT_UNIDECODE=yes -t -i python:3.7 /bin/bash -c "pip install git+https://github.com/Fokko/incubator-airflow.git@v1-10-stable && airflow initdb && airflow run example_bash_operator runme_0 2017-07-01" [2018-08-08 21:25:57,944] {__init__.py:51} INFO - Using executor SequentialExecutor [2018-08-08 21:25:58,069] {models.py:258} INFO - Filling up the DagBag from /root/airflow/dags [2018-08-08 21:25:58,112] {example_kubernetes_operator.py:54} WARNING - Could not import KubernetesPodOperator: No module named 'kubernetes' [2018-08-08 21:25:58,112] {example_kubernetes_operator.py:55} WARNING - Install kubernetes dependencies with: pip install airflow['kubernetes'] [2018-08-08 21:25:58,155] {cli.py:492} INFO - Running <TaskInstance: example_bash_operator.runme_0 2017-07-01T00:00:00+00:00 [None]> on host 31ec1d1554b7 [2018-08-08 21:25:58,739] {__init__.py:51} INFO - Using executor SequentialExecutor [2018-08-08 21:25:58,915] {models.py:258} INFO - Filling up the DagBag from /root/airflow/dags/example_dags/example_bash_operator.py [2018-08-08 21:25:58,987] {example_kubernetes_operator.py:54} WARNING - Could not import KubernetesPodOperator: No module named 'kubernetes' [2018-08-08 21:25:58,987] {example_kubernetes_operator.py:55} WARNING - Install kubernetes dependencies with: pip install airflow['kubernetes'] [2018-08-08 21:25:59,060] {cli.py:492} INFO - Running <TaskInstance: example_bash_operator.runme_0 2017-07-01T00:00:00+00:00 [None]> on host 31ec1d1554b7 https://github.com/Fokko/incubator-airflow/commits/v1-10-stable Still no hard guarantees that 3.7 will be fully supported, but at least it runs :-) Cheers, Fokko 2018-08-08 20:21 GMT+02:00 Bolke de Bruin <[email protected]>: > Hey all, > > I have cut Airflow 1.10.0 RC4. This email is calling a vote on the release, > which will last for 72 hours. Consider this my (binding) +1. > > Airflow 1.10.0 RC 4 is available at: > > https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc4/ < > https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc4/> > > apache-airflow-1.10.0rc4+incubating-source.tar.gz is a source release that > comes with INSTALL instructions. > apache-airflow-1.10.0rc4+incubating-bin.tar.gz is the binary Python > "sdist" > release. > > Public keys are available at: > > https://dist.apache.org/repos/dist/release/incubator/airflow/ < > https://dist.apache.org/repos/dist/release/incubator/airflow/> > > The amount of JIRAs fixed is over 700. Please have a look at the > changelog. > Since RC3 the following has been fixed: > > [AIRFLOW-2870] Use abstract TaskInstance for migration > [AIRFLOW-2859] Implement own UtcDateTime > [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook > [AIRFLOW-2869] Remove smart quote from default config > [AIRFLOW-2857] Fix Read the Docs env > > Please note that the version number excludes the `rcX` string as well > as the "+incubating" string, so it's now simply 1.10.0. This will allow us > to rename the artifact without modifying the artifact checksums when we > actually release. > > WARNING: Due to licensing requirements you will need to set > SLUGIFY_USES_TEXT_UNIDECODE=yes in your environment when > installing or upgrading. We will try to remove this requirement for the > next release. > > Cheers, > Bolke
