That said I can't reproduce it myself (I will take discussion to the Jira ticket)
-ash > On 12 Jul 2018, at 13:36, Ash Berlin-Taylor <ash_airflowl...@firemirror.com> > wrote: > > Possible blocking regression here - .airflowignore doesn't seem to be > respected anymore > > https://issues.apache.org/jira/browse/AIRFLOW-2729 > <https://issues.apache.org/jira/browse/AIRFLOW-2729> > > -ash >> On 12 Jul 2018, at 08:45, Bolke de Bruin <bdbr...@gmail.com> wrote: >> >> Hi Jakob, >> >> Thanks. We do include an INSTALL document that explains how to well install >> airflow and is quite a standard location for install instructions. Or did >> the install instruction needed to be included in the Vote? >> >> B. >> >> >> >> Verstuurd vanaf mijn iPad >> >>> Op 12 jul. 2018 om 07:54 heeft Jakob Homan <jgho...@gmail.com> het volgende >>> geschreven: >>> >>> +1 (binding) >>> >>> * Sigs look good >>> * Artifact has incubating in name >>> * LICENSE/NOTICE/DISCLAIMER look good >>> - nit: DISCLAIMER is not word wrapped to 80 chars as I've seen in >>> other projects. >>> * Nit: Last release Sebb called out "Copyright 2016 and onwards" in >>> NOTICE as being imprecise. This language remains. >>> * Spot check on license headers looks ok. >>> * Nit: Last release there was a request for instructions on how to >>> build the code. This isn't included. >>> >>> -Jakob >>> >>>> On 11 July 2018 at 19:50, Sid Anand <san...@apache.org> wrote: >>>> FYI! >>>> I just installed the release candidate. The first thing I noticed is a >>>> missing tool tip for the Null State in the Recent Tasks column on the >>>> landing page. Since the null globe is new to this UI, users will likely >>>> hover over it to inquire what it means... and will be left wanting. Of >>>> course, they could click on the globe, which will take them to >>>> http://localhost:8080/admin/taskinstance/?flt1_dag_id_equals=example_bash_operator&flt2_state_equals=null, >>>> which will always show an empty list, leaving them a bit more confused. >>>> >>>> -s >>>> >>>> On Wed, Jul 11, 2018 at 3:13 PM Carl Johan Gustavsson >>>> <carl.jo...@tictail.com.invalid> wrote: >>>> >>>>> Hi Bolke, >>>>> >>>>> (Switching email to avoid moderation on my emails.) >>>>> >>>>> The normal Airflow test suite does not fail as it uses a LC_ALL set to >>>>> utf-8. >>>>> >>>>> I think it is a proper test though, it is a minimal reproducible version >>>>> of >>>>> the code that fails. And the only difference in behaviour is at 3.7 which >>>>> we don’t support anyway so I’m fairly sure it is broken for all supported >>>>> Python 3 versions. >>>>> >>>>> I now tried running the tests in docker using 3.5 with the LC_ALL/LANG >>>>> unset and I see the same failure. >>>>> >>>>> I don’t think this is a big thing though and we could release it without >>>>> the fix I made. I think most people run it with a sane LC_ALL, but >>>>> apparently we didn’t. >>>>> Here’s the log for the test: >>>>> >>>>>> docker run -t -i -v `pwd`:/airflow/ python:3.5 bash >>>>> root@b99b297df111:/# locale >>>>> LANG=C.UTF-8 >>>>> LANGUAGE= >>>>> LC_CTYPE="C.UTF-8" >>>>> LC_NUMERIC="C.UTF-8" >>>>> LC_TIME="C.UTF-8" >>>>> LC_COLLATE="C.UTF-8" >>>>> LC_MONETARY="C.UTF-8" >>>>> LC_MESSAGES="C.UTF-8" >>>>> LC_PAPER="C.UTF-8" >>>>> LC_NAME="C.UTF-8" >>>>> LC_ADDRESS="C.UTF-8" >>>>> LC_TELEPHONE="C.UTF-8" >>>>> LC_MEASUREMENT="C.UTF-8" >>>>> LC_IDENTIFICATION="C.UTF-8" >>>>> LC_ALL= >>>>>> unset LANG >>>>> root@b99b297df111:/# locale >>>>> LANG= >>>>> LANGUAGE= >>>>> LC_CTYPE="POSIX" >>>>> LC_NUMERIC="POSIX" >>>>> LC_TIME="POSIX" >>>>> LC_COLLATE="POSIX" >>>>> LC_MONETARY="POSIX" >>>>> LC_MESSAGES="POSIX" >>>>> LC_PAPER="POSIX" >>>>> LC_NAME="POSIX" >>>>> LC_ADDRESS="POSIX" >>>>> LC_TELEPHONE="POSIX" >>>>> LC_MEASUREMENT="POSIX" >>>>> LC_IDENTIFICATION="POSIX" >>>>> LC_ALL= >>>>> root@b99b297df111:/# pip install -e .[devel] >>>>> root@b99b297df111:/airflow# ./run_unit_tests.sh >>>>> + export AIRFLOW_HOME=/root/airflow >>>>> + AIRFLOW_HOME=/root/airflow >>>>> + export AIRFLOW__CORE__UNIT_TEST_MODE=True >>>>> + AIRFLOW__CORE__UNIT_TEST_MODE=True >>>>> + export AIRFLOW__TESTSECTION__TESTKEY=testvalue >>>>> + AIRFLOW__TESTSECTION__TESTKEY=testvalue >>>>> + export AIRFLOW_USE_NEW_IMPORTS=1 >>>>> + AIRFLOW_USE_NEW_IMPORTS=1 >>>>> +++ dirname ./run_unit_tests.sh >>>>> ++ cd . >>>>> ++ pwd >>>>> + DIR=/airflow >>>>> + export PYTHONPATH=:/airflow/tests/test_utils >>>>> + PYTHONPATH=:/airflow/tests/test_utils >>>>> + nose_args= >>>>> + which airflow >>>>> + echo 'Initializing the DB' >>>>> Initializing the DB >>>>> + airflow resetdb >>>>> + yes >>>>> Traceback (most recent call last): >>>>> File "/usr/local/bin/airflow", line 6, in <module> >>>>> exec(compile(open(__file__).read(), __file__, 'exec')) >>>>> File "/airflow/airflow/bin/airflow", line 21, in <module> >>>>> from airflow import configuration >>>>> File "/airflow/airflow/__init__.py", line 35, in <module> >>>>> from airflow import configuration as conf >>>>> File "/airflow/airflow/configuration.py", line 106, in <module> >>>>> DEFAULT_CONFIG = f.read() >>>>> File "/usr/local/lib/python3.5/encodings/ascii.py", line 26, in decode >>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>> 21082: >>>>> ordinal not in range(128) >>>>> + '[' '' ']' >>>>> + '[' -z '' ']' >>>>> + nose_args='--with-coverage --cover-erase --cover-html >>>>> --cover-package=airflow --cover-html-dir=airflow/www/static/coverage >>>>> --with-ignore-docstrings --rednose --with-timer -s -v >>>>> --logging-level=DEBUG ' >>>>> + echo 'Starting the unit tests with the following nose arguments: >>>>> --with-coverage' --cover-erase --cover-html --cover-package=airflow >>>>> --cover-html-dir=airflow/www/static/coverage --with-ignore-docstrings >>>>> --rednose --with-timer -s -v --logging-level=DEBUG >>>>> Starting the unit tests with the following nose arguments: --with-coverage >>>>> --cover-erase --cover-html --cover-package=airflow >>>>> --cover-html-dir=airflow/www/static/coverage --with-ignore-docstrings >>>>> --rednose --with-timer -s -v --logging-level=DEBUG >>>>> + nosetests --with-coverage --cover-erase --cover-html >>>>> --cover-package=airflow --cover-html-dir=airflow/www/static/coverage >>>>> --with-ignore-docstrings --rednose --with-timer -s -v >>>>> --logging-level=DEBUG >>>>> nose.plugins.cover: ERROR: Coverage not available: unable to import >>>>> coverage module >>>>> Failure: UnicodeDecodeError ('ascii' codec can't decode byte 0xe2 in >>>>> position 21082: ordinal not in range(128)) ... ERROR >>>>> Failure: UnicodeDecodeError ('ascii' codec can't decode byte 0xe2 in >>>>> position 21082: ordinal not in range(128)) ... ERROR >>>>> ====================================================================== >>>>> 1) ERROR: Failure: UnicodeDecodeError ('ascii' codec can't decode byte >>>>> 0xe2 >>>>> in position 21082: ordinal not in range(128)) >>>>> ---------------------------------------------------------------------- >>>>> Traceback (most recent call last): >>>>> /usr/local/lib/python3.5/site-packages/nose/failure.py line 39 in >>>>> runTest >>>>> raise self.exc_val.with_traceback(self.tb) >>>>> /usr/local/lib/python3.5/site-packages/nose/loader.py line 418 in >>>>> loadTestsFromName >>>>> addr.filename, addr.module) >>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 47 in >>>>> importFromPath >>>>> return self.importFromDir(dir_path, fqname) >>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 94 in >>>>> importFromDir >>>>> mod = load_module(part_fqname, fh, filename, desc) >>>>> /usr/local/lib/python3.5/imp.py line 245 in load_module >>>>> return load_package(name, filename) >>>>> /usr/local/lib/python3.5/imp.py line 217 in load_package >>>>> return _load(spec) >>>>> <frozen importlib._bootstrap> line 693 in _load >>>>> >>>>> <frozen importlib._bootstrap> line 673 in _load_unlocked >>>>> >>>>> <frozen importlib._bootstrap_external> line 697 in exec_module >>>>> >>>>> <frozen importlib._bootstrap> line 222 in _call_with_frames_removed >>>>> >>>>> airflow/__init__.py line 35 in <module> >>>>> from airflow import configuration as conf >>>>> airflow/configuration.py line 106 in <module> >>>>> DEFAULT_CONFIG = f.read() >>>>> /usr/local/lib/python3.5/encodings/ascii.py line 26 in decode >>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>> 21082: ordinal not in range(128) >>>>> ====================================================================== >>>>> 2) ERROR: Failure: UnicodeDecodeError ('ascii' codec can't decode byte >>>>> 0xe2 >>>>> in position 21082: ordinal not in range(128)) >>>>> ---------------------------------------------------------------------- >>>>> Traceback (most recent call last): >>>>> /usr/local/lib/python3.5/site-packages/nose/failure.py line 39 in >>>>> runTest >>>>> raise self.exc_val.with_traceback(self.tb) >>>>> /usr/local/lib/python3.5/site-packages/nose/loader.py line 418 in >>>>> loadTestsFromName >>>>> addr.filename, addr.module) >>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 47 in >>>>> importFromPath >>>>> return self.importFromDir(dir_path, fqname) >>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 94 in >>>>> importFromDir >>>>> mod = load_module(part_fqname, fh, filename, desc) >>>>> /usr/local/lib/python3.5/imp.py line 245 in load_module >>>>> return load_package(name, filename) >>>>> /usr/local/lib/python3.5/imp.py line 217 in load_package >>>>> return _load(spec) >>>>> <frozen importlib._bootstrap> line 693 in _load >>>>> >>>>> <frozen importlib._bootstrap> line 673 in _load_unlocked >>>>> >>>>> <frozen importlib._bootstrap_external> line 697 in exec_module >>>>> >>>>> <frozen importlib._bootstrap> line 222 in _call_with_frames_removed >>>>> >>>>> tests/__init__.py line 25 in <module> >>>>> from .configuration import * >>>>> tests/configuration.py line 28 in <module> >>>>> from airflow import configuration >>>>> airflow/__init__.py line 35 in <module> >>>>> from airflow import configuration as conf >>>>> airflow/configuration.py line 106 in <module> >>>>> DEFAULT_CONFIG = f.read() >>>>> /usr/local/lib/python3.5/encodings/ascii.py line 26 in decode >>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>> 21082: ordinal not in range(128) >>>>> >>>>> [error] 100.00% nose.failure.Failure.runTest: 0.0004s >>>>> >>>>> ----------------------------------------------------------------------------- >>>>> 2 tests run in 0.060 seconds. >>>>> 2 errors (0 tests passed) >>>>> >>>>> >>>>> -- >>>>> Carl Johan Gustavsson >>>>> >>>>> On 11 July 2018 at 23:24:01, Bolke de Bruin (bdbr...@gmail.com) wrote: >>>>> >>>>> Hi Carl, >>>>> >>>>> That is not a real check, ie. Does Airflow have the same issue clean >>>>> install on 3.5? Travis’ tests run on 3.5. >>>>> >>>>> B. >>>>> >>>>> Verstuurd vanaf mijn iPad >>>>> >>>>>> Op 10 jul. 2018 om 15:10 heeft Carl Johan Gustavsson < >>>>> carl.j.gustavs...@gmail.com> het volgende geschreven: >>>>>> >>>>>> Hi Bolke, >>>>>> >>>>>> >>>>>> I did a quick test on 3.5.5, 3.4.0 and 3.7.0 on OS X now, all but 3.7 >>>>> breaks >>>>>> >>>>>> Quick repro: >>>>>> >>>>>> ➜ ~ pyenv local 3.5.5 >>>>>> ➜ ~ locale >>>>>> LANG= >>>>>> LC_COLLATE="C" >>>>>> LC_CTYPE="C" >>>>>> LC_MESSAGES="C" >>>>>> LC_MONETARY="C" >>>>>> LC_NUMERIC="C" >>>>>> LC_TIME="C" >>>>>> LC_ALL= >>>>>> ➜ ~ cat testweird.txt >>>>>> ’ >>>>>> ➜ ~ python >>>>>> Python 3.5.5 (default, Jul 7 2018, 17:00:56) >>>>>> [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin >>>>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>>>> open('testweird.txt').read() >>>>>> Traceback (most recent call last): >>>>>> File "<stdin>", line 1, in <module> >>>>>> File "/Users/cjg/.pyenv/versions/3.5.5/lib/python3.5/encodings/ascii.py", >>>>> line 26, in decode >>>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: >>>>> ordinal not in range(128) >>>>>>>>> >>>>>> >>>>>> >>>>>> Maybe not a blocking change but it is a breaking change from 1.9 I guess. >>>>>> >>>>>> >>>>>> / Carl Johan >>>>>> >>>>>> >>>>>>> On 10 July 2018 at 14:48:47, Bolke de Bruin (bdbr...@gmail.com) wrote: >>>>>>> >>>>>>> Hi Carl, >>>>>>> >>>>>>> Did you this on python 3.5 as well? 3.6 is not an officially supported >>>>> (yet). As a workaround is available I won’t consider this blocking btw. >>>>>>> >>>>>>> Bolke >>>>>>> >>>>>>> Verstuurd vanaf mijn iPad >>>>>>> >>>>>>> Op 10 jul. 2018 om 11:53 heeft Carl Johan Gustavsson < >>>>> carl.j.gustavs...@gmail.com> het volgende geschreven: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> First of all, thank you for all the work with the release management. >>>>>>>> >>>>>>>> I ran in to a weird issue testing the RC1, running under Python 3.6.0 / >>>>> Ubuntu 14.04.5, upgrading from a master build from February. >>>>>>>> >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 Traceback >>>>> (most recent call last): >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>> "/opt/virtualenv/tictail/pipeline/bin/airflow", line 21, in <module> >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 from airflow >>>>> import configuration >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>> >>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-packages/airflow/__init__.py", >>>>> line 35, in <module> >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 from airflow >>>>> import configuration as conf >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>> >>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-packages/airflow/configuration.py", >>>>> line 106, in <module> >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 >>>>> DEFAULT_CONFIG = f.read() >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/encodings/ascii.py", line >>>>> 26, in decode >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 return >>>>> codecs.ascii_decode(input, self.errors)[0] >>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 >>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>> 20770: >>>>> ordinal not in range(128) >>>>>>>> >>>>>>>> Removing the ’ on >>>>> >>>>> https://github.com/apache/incubator-airflow/blob/master/airflow/config_templates/default_airflow.cfg#L613 >>>>> solved the issue for me, and digging a bit deeper it seems Airflow now >>>>> requires setting LC_ALL=en_US.UTF-8 in the environment or similar to force >>>>> Python to read the file as utf-8 and not ascii. (I think this was changed >>>>> in to default to utf-8 in Python 3.7). >>>>>>>> >>>>>>>> I see 3 solutions for this >>>>>>>> 1. Document that Airflow need to run with LC_ALL=en_US.UTF-8 or >>>>> similar. >>>>>>>> 2. Change the default config file to not contain non-ascii characters. >>>>>>>> 3. Always read the file as unicode regardless of the LC_ALL >>>>> environment, by the encoding='utf-8’ parameter to open(). >>>>>>>> >>>>>>>> I think 3 is the best solution, and I can prepare a PR for that if >>>>> necessary . >>>>>>>> >>>>>>>> I guess this counts as -1 (non-binding) >>>>>>>> >>>>>>>> All the best >>>>>>>> >>>>>>>> Carl Johan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 8 July 2018 at 22:02:33, Bolke de Bruin (bdbr...@gmail.com) wrote: >>>>>>>>> >>>>>>>>> Hey all, >>>>>>>>> >>>>>>>>> I have cut Airflow 1.10.0 RC1. This email is calling a vote on the >>>>> release, >>>>>>>>> which will last for 72 hours. Consider this my (binding) +1. >>>>>>>>> >>>>>>>>> Airflow 1.10.0 RC 1 is available at: >>>>>>>>> >>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc1/ < >>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc1/> >>>>>>>>> >>>>>>>>> apache-airflow-1.10.0rc1+incubating-source.tar.gz is a source release >>>>> that >>>>>>>>> comes with INSTALL instructions. >>>>>>>>> apache-airflow-1.10.0rc1+incubating-bin.tar.gz is the binary Python >>>>> "sdist" >>>>>>>>> release. >>>>>>>>> >>>>>>>>> Public keys are available at: >>>>>>>>> >>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/airflow/ < >>>>> https://dist.apache.org/repos/dist/release/incubator/airflow/> >>>>>>>>> >>>>>>>>> The amount of JIRAs fixed is over 700. Please have a look at the >>>>> changelog. >>>>>>>>> >>>>>>>>> Please note that the version number excludes the `rcX` string as well >>>>>>>>> as the "+incubating" string, so it's now simply 1.10.0. This will >>>>> allow us >>>>>>>>> to rename the artifact without modifying the artifact checksums when >>>>> we >>>>>>>>> actually release. >>>>>>>>> >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Bolke >>>>> >
