https://github.com/apache/incubator-airflow/pull/3602 
<https://github.com/apache/incubator-airflow/pull/3602>

This fixes the ignore file being ignored in dirs, and fixes AIRFLOW-1729 
correctly to not walk into dirs that have been ignored. (The previous PR 
targeted at that Jira didn't change that behaviour)

It doesn't contain unittests as we don't have anything in place to test that. 
I'm not happy with that situation, but it doesn't make anything worse. It 
removes duplication between models.py and utils/dag_processing.py too.

(sent again from my correct email so it doesn't need moderation. Sorry)

-ash 


> On 12 Jul 2018, at 21:54, Bolke de Bruin <bdbr...@gmail.com> wrote:
> 
> If a PR is quick, I’m willing to restart the vote for a RC2 (we are 3 +1, 0.5 
> -1).
> 
> B.
> 
> Verstuurd vanaf mijn iPad
> 
>> Op 12 jul. 2018 om 15:39 heeft James Meickle 
>> <jmeic...@quantopian.com.INVALID> het volgende geschreven:
>> 
>> Looking at that diff, it seems like the function as a whole needs some
>> love, even if that commit were reverted. The use of os.walk means it's
>> going to crawl the entire tree every time, and use accumulated patterns to
>> check against each file in each folder. The behavior should be to use
>> patterns to also exclude entire folders from consideration, which would
>> likely dramatically speed up the function for most use cases.
>> 
>> On Thu, Jul 12, 2018 at 9:14 AM, Ash Berlin-Taylor <
>> ash_airflowl...@firemirror.com> wrote:
>> 
>>> I wasn't paying enough attention. I can reproduce it.
>>> 
>>> I'm not sure it was intentional but the bug was introduced in
>>> https://github.com/apache/incubator-airflow/pull/3171 <
>>> https://github.com/apache/incubator-airflow/pull/3171>
>>> 
>>> I'd rather not release with a regression since 1.9, so updating my vote to
>>> -0.5 (binding), but not blocking/vetoing release if others are okay with it.
>>> 
>>> -ash
>>> 
>>>> On 12 Jul 2018, at 13:46, Ash Berlin-Taylor <ash_airflowlist@firemirror.
>>> com> wrote:
>>>> 
>>>> That said I can't reproduce it myself (I will take discussion to the
>>> Jira ticket)
>>>> 
>>>> -ash
>>>> 
>>>>> On 12 Jul 2018, at 13:36, Ash Berlin-Taylor <
>>> ash_airflowl...@firemirror.com> wrote:
>>>>> 
>>>>> Possible blocking regression here - .airflowignore doesn't seem to be
>>> respected anymore
>>>>> 
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-2729 <
>>> https://issues.apache.org/jira/browse/AIRFLOW-2729>
>>>>> 
>>>>> -ash
>>>>>> On 12 Jul 2018, at 08:45, Bolke de Bruin <bdbr...@gmail.com> wrote:
>>>>>> 
>>>>>> Hi Jakob,
>>>>>> 
>>>>>> Thanks. We do include an INSTALL document that explains how to well
>>> install airflow and is quite a standard location for install instructions.
>>> Or did the install instruction needed to be included in the Vote?
>>>>>> 
>>>>>> B.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Verstuurd vanaf mijn iPad
>>>>>> 
>>>>>>> Op 12 jul. 2018 om 07:54 heeft Jakob Homan <jgho...@gmail.com> het
>>> volgende geschreven:
>>>>>>> 
>>>>>>> +1 (binding)
>>>>>>> 
>>>>>>> * Sigs look good
>>>>>>> * Artifact has incubating in name
>>>>>>> * LICENSE/NOTICE/DISCLAIMER look good
>>>>>>> - nit: DISCLAIMER is not word wrapped to 80 chars as I've seen in
>>>>>>> other projects.
>>>>>>> * Nit: Last release Sebb called out "Copyright 2016 and onwards" in
>>>>>>> NOTICE as being imprecise.  This language remains.
>>>>>>> * Spot check on license headers looks ok.
>>>>>>> * Nit: Last release there was a request for instructions on how to
>>>>>>> build the code.  This isn't included.
>>>>>>> 
>>>>>>> -Jakob
>>>>>>> 
>>>>>>>> On 11 July 2018 at 19:50, Sid Anand <san...@apache.org> wrote:
>>>>>>>> FYI!
>>>>>>>> I just installed the release candidate. The first thing I noticed is
>>> a
>>>>>>>> missing tool tip for the Null State in the Recent Tasks column on the
>>>>>>>> landing page. Since the null globe is new to this UI, users will
>>> likely
>>>>>>>> hover over it to inquire what it means... and will be left wanting.
>>> Of
>>>>>>>> course, they could click on the globe, which will take them to
>>>>>>>> http://localhost:8080/admin/taskinstance/?flt1_dag_id_
>>> equals=example_bash_operator&flt2_state_equals=null,
>>>>>>>> which will always show an empty list, leaving them a bit more
>>> confused.
>>>>>>>> 
>>>>>>>> -s
>>>>>>>> 
>>>>>>>> On Wed, Jul 11, 2018 at 3:13 PM Carl Johan Gustavsson
>>>>>>>> <carl.jo...@tictail.com.invalid> wrote:
>>>>>>>> 
>>>>>>>>> Hi Bolke,
>>>>>>>>> 
>>>>>>>>> (Switching email to avoid moderation on my emails.)
>>>>>>>>> 
>>>>>>>>> The normal Airflow test suite does not fail as it uses a LC_ALL set
>>> to
>>>>>>>>> utf-8.
>>>>>>>>> 
>>>>>>>>> I think it is a proper test though, it is a minimal reproducible
>>> version of
>>>>>>>>> the code that fails. And the only difference in behaviour is at 3.7
>>> which
>>>>>>>>> we don’t support anyway so I’m fairly sure it is broken for all
>>> supported
>>>>>>>>> Python 3 versions.
>>>>>>>>> 
>>>>>>>>> I now tried running the tests in docker using 3.5 with the
>>> LC_ALL/LANG
>>>>>>>>> unset and I see the same failure.
>>>>>>>>> 
>>>>>>>>> I don’t think this is a big thing though and we could release it
>>> without
>>>>>>>>> the fix I made. I think most people run it with a sane LC_ALL, but
>>>>>>>>> apparently we didn’t.
>>>>>>>>> Here’s the log for the test:
>>>>>>>>> 
>>>>>>>>>> docker run -t -i -v `pwd`:/airflow/ python:3.5 bash
>>>>>>>>> root@b99b297df111:/# locale
>>>>>>>>> LANG=C.UTF-8
>>>>>>>>> LANGUAGE=
>>>>>>>>> LC_CTYPE="C.UTF-8"
>>>>>>>>> LC_NUMERIC="C.UTF-8"
>>>>>>>>> LC_TIME="C.UTF-8"
>>>>>>>>> LC_COLLATE="C.UTF-8"
>>>>>>>>> LC_MONETARY="C.UTF-8"
>>>>>>>>> LC_MESSAGES="C.UTF-8"
>>>>>>>>> LC_PAPER="C.UTF-8"
>>>>>>>>> LC_NAME="C.UTF-8"
>>>>>>>>> LC_ADDRESS="C.UTF-8"
>>>>>>>>> LC_TELEPHONE="C.UTF-8"
>>>>>>>>> LC_MEASUREMENT="C.UTF-8"
>>>>>>>>> LC_IDENTIFICATION="C.UTF-8"
>>>>>>>>> LC_ALL=
>>>>>>>>>> unset LANG
>>>>>>>>> root@b99b297df111:/# locale
>>>>>>>>> LANG=
>>>>>>>>> LANGUAGE=
>>>>>>>>> LC_CTYPE="POSIX"
>>>>>>>>> LC_NUMERIC="POSIX"
>>>>>>>>> LC_TIME="POSIX"
>>>>>>>>> LC_COLLATE="POSIX"
>>>>>>>>> LC_MONETARY="POSIX"
>>>>>>>>> LC_MESSAGES="POSIX"
>>>>>>>>> LC_PAPER="POSIX"
>>>>>>>>> LC_NAME="POSIX"
>>>>>>>>> LC_ADDRESS="POSIX"
>>>>>>>>> LC_TELEPHONE="POSIX"
>>>>>>>>> LC_MEASUREMENT="POSIX"
>>>>>>>>> LC_IDENTIFICATION="POSIX"
>>>>>>>>> LC_ALL=
>>>>>>>>> root@b99b297df111:/# pip install -e .[devel]
>>>>>>>>> root@b99b297df111:/airflow# ./run_unit_tests.sh
>>>>>>>>> + export AIRFLOW_HOME=/root/airflow
>>>>>>>>> + AIRFLOW_HOME=/root/airflow
>>>>>>>>> + export AIRFLOW__CORE__UNIT_TEST_MODE=True
>>>>>>>>> + AIRFLOW__CORE__UNIT_TEST_MODE=True
>>>>>>>>> + export AIRFLOW__TESTSECTION__TESTKEY=testvalue
>>>>>>>>> + AIRFLOW__TESTSECTION__TESTKEY=testvalue
>>>>>>>>> + export AIRFLOW_USE_NEW_IMPORTS=1
>>>>>>>>> + AIRFLOW_USE_NEW_IMPORTS=1
>>>>>>>>> +++ dirname ./run_unit_tests.sh
>>>>>>>>> ++ cd .
>>>>>>>>> ++ pwd
>>>>>>>>> + DIR=/airflow
>>>>>>>>> + export PYTHONPATH=:/airflow/tests/test_utils
>>>>>>>>> + PYTHONPATH=:/airflow/tests/test_utils
>>>>>>>>> + nose_args=
>>>>>>>>> + which airflow
>>>>>>>>> + echo 'Initializing the DB'
>>>>>>>>> Initializing the DB
>>>>>>>>> + airflow resetdb
>>>>>>>>> + yes
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "/usr/local/bin/airflow", line 6, in <module>
>>>>>>>>> exec(compile(open(__file__).read(), __file__, 'exec'))
>>>>>>>>> File "/airflow/airflow/bin/airflow", line 21, in <module>
>>>>>>>>> from airflow import configuration
>>>>>>>>> File "/airflow/airflow/__init__.py", line 35, in <module>
>>>>>>>>> from airflow import configuration as conf
>>>>>>>>> File "/airflow/airflow/configuration.py", line 106, in <module>
>>>>>>>>> DEFAULT_CONFIG = f.read()
>>>>>>>>> File "/usr/local/lib/python3.5/encodings/ascii.py", line 26, in
>>> decode
>>>>>>>>> return codecs.ascii_decode(input, self.errors)[0]
>>>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in
>>> position 21082:
>>>>>>>>> ordinal not in range(128)
>>>>>>>>> + '[' '' ']'
>>>>>>>>> + '[' -z '' ']'
>>>>>>>>> + nose_args='--with-coverage     --cover-erase     --cover-html
>>>>>>>>> --cover-package=airflow     --cover-html-dir=airflow/www/
>>> static/coverage
>>>>>>>>> --with-ignore-docstrings     --rednose     --with-timer     -s
>>> -v
>>>>>>>>> --logging-level=DEBUG '
>>>>>>>>> + echo 'Starting the unit tests with the following nose arguments:
>>>>>>>>> --with-coverage' --cover-erase --cover-html --cover-package=airflow
>>>>>>>>> --cover-html-dir=airflow/www/static/coverage
>>> --with-ignore-docstrings
>>>>>>>>> --rednose --with-timer -s -v --logging-level=DEBUG
>>>>>>>>> Starting the unit tests with the following nose arguments:
>>> --with-coverage
>>>>>>>>> --cover-erase --cover-html --cover-package=airflow
>>>>>>>>> --cover-html-dir=airflow/www/static/coverage
>>> --with-ignore-docstrings
>>>>>>>>> --rednose --with-timer -s -v --logging-level=DEBUG
>>>>>>>>> + nosetests --with-coverage --cover-erase --cover-html
>>>>>>>>> --cover-package=airflow --cover-html-dir=airflow/www/
>>> static/coverage
>>>>>>>>> --with-ignore-docstrings --rednose --with-timer -s -v
>>> --logging-level=DEBUG
>>>>>>>>> nose.plugins.cover: ERROR: Coverage not available: unable to import
>>>>>>>>> coverage module
>>>>>>>>> Failure: UnicodeDecodeError ('ascii' codec can't decode byte 0xe2 in
>>>>>>>>> position 21082: ordinal not in range(128)) ... ERROR
>>>>>>>>> Failure: UnicodeDecodeError ('ascii' codec can't decode byte 0xe2 in
>>>>>>>>> position 21082: ordinal not in range(128)) ... ERROR
>>>>>>>>> ============================================================
>>> ==========
>>>>>>>>> 1) ERROR: Failure: UnicodeDecodeError ('ascii' codec can't decode
>>> byte 0xe2
>>>>>>>>> in position 21082: ordinal not in range(128))
>>>>>>>>> ------------------------------------------------------------
>>> ----------
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/failure.py line 39 in
>>>>>>>>> runTest
>>>>>>>>> raise self.exc_val.with_traceback(self.tb)
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/loader.py line 418 in
>>>>>>>>> loadTestsFromName
>>>>>>>>> addr.filename, addr.module)
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 47 in
>>>>>>>>> importFromPath
>>>>>>>>> return self.importFromDir(dir_path, fqname)
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 94 in
>>>>>>>>> importFromDir
>>>>>>>>> mod = load_module(part_fqname, fh, filename, desc)
>>>>>>>>> /usr/local/lib/python3.5/imp.py line 245 in load_module
>>>>>>>>> return load_package(name, filename)
>>>>>>>>> /usr/local/lib/python3.5/imp.py line 217 in load_package
>>>>>>>>> return _load(spec)
>>>>>>>>> <frozen importlib._bootstrap> line 693 in _load
>>>>>>>>> 
>>>>>>>>> <frozen importlib._bootstrap> line 673 in _load_unlocked
>>>>>>>>> 
>>>>>>>>> <frozen importlib._bootstrap_external> line 697 in exec_module
>>>>>>>>> 
>>>>>>>>> <frozen importlib._bootstrap> line 222 in _call_with_frames_removed
>>>>>>>>> 
>>>>>>>>> airflow/__init__.py line 35 in <module>
>>>>>>>>> from airflow import configuration as conf
>>>>>>>>> airflow/configuration.py line 106 in <module>
>>>>>>>>> DEFAULT_CONFIG = f.read()
>>>>>>>>> /usr/local/lib/python3.5/encodings/ascii.py line 26 in decode
>>>>>>>>> return codecs.ascii_decode(input, self.errors)[0]
>>>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
>>>>>>>>> 21082: ordinal not in range(128)
>>>>>>>>> ============================================================
>>> ==========
>>>>>>>>> 2) ERROR: Failure: UnicodeDecodeError ('ascii' codec can't decode
>>> byte 0xe2
>>>>>>>>> in position 21082: ordinal not in range(128))
>>>>>>>>> ------------------------------------------------------------
>>> ----------
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/failure.py line 39 in
>>>>>>>>> runTest
>>>>>>>>> raise self.exc_val.with_traceback(self.tb)
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/loader.py line 418 in
>>>>>>>>> loadTestsFromName
>>>>>>>>> addr.filename, addr.module)
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 47 in
>>>>>>>>> importFromPath
>>>>>>>>> return self.importFromDir(dir_path, fqname)
>>>>>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 94 in
>>>>>>>>> importFromDir
>>>>>>>>> mod = load_module(part_fqname, fh, filename, desc)
>>>>>>>>> /usr/local/lib/python3.5/imp.py line 245 in load_module
>>>>>>>>> return load_package(name, filename)
>>>>>>>>> /usr/local/lib/python3.5/imp.py line 217 in load_package
>>>>>>>>> return _load(spec)
>>>>>>>>> <frozen importlib._bootstrap> line 693 in _load
>>>>>>>>> 
>>>>>>>>> <frozen importlib._bootstrap> line 673 in _load_unlocked
>>>>>>>>> 
>>>>>>>>> <frozen importlib._bootstrap_external> line 697 in exec_module
>>>>>>>>> 
>>>>>>>>> <frozen importlib._bootstrap> line 222 in _call_with_frames_removed
>>>>>>>>> 
>>>>>>>>> tests/__init__.py line 25 in <module>
>>>>>>>>> from .configuration import *
>>>>>>>>> tests/configuration.py line 28 in <module>
>>>>>>>>> from airflow import configuration
>>>>>>>>> airflow/__init__.py line 35 in <module>
>>>>>>>>> from airflow import configuration as conf
>>>>>>>>> airflow/configuration.py line 106 in <module>
>>>>>>>>> DEFAULT_CONFIG = f.read()
>>>>>>>>> /usr/local/lib/python3.5/encodings/ascii.py line 26 in decode
>>>>>>>>> return codecs.ascii_decode(input, self.errors)[0]
>>>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
>>>>>>>>> 21082: ordinal not in range(128)
>>>>>>>>> 
>>>>>>>>> [error] 100.00% nose.failure.Failure.runTest: 0.0004s
>>>>>>>>> 
>>>>>>>>> ------------------------------------------------------------
>>> -----------------
>>>>>>>>> 2 tests run in 0.060 seconds.
>>>>>>>>> 2 errors (0 tests passed)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Carl Johan Gustavsson
>>>>>>>>> 
>>>>>>>>> On 11 July 2018 at 23:24:01, Bolke de Bruin (bdbr...@gmail.com)
>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Carl,
>>>>>>>>> 
>>>>>>>>> That is not a real check, ie. Does Airflow have the same issue clean
>>>>>>>>> install on 3.5? Travis’ tests run on 3.5.
>>>>>>>>> 
>>>>>>>>> B.
>>>>>>>>> 
>>>>>>>>> Verstuurd vanaf mijn iPad
>>>>>>>>> 
>>>>>>>>>> Op 10 jul. 2018 om 15:10 heeft Carl Johan Gustavsson <
>>>>>>>>> carl.j.gustavs...@gmail.com> het volgende geschreven:
>>>>>>>>>> 
>>>>>>>>>> Hi Bolke,
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> I did a quick test on 3.5.5, 3.4.0 and 3.7.0 on OS X now, all but
>>> 3.7
>>>>>>>>> breaks
>>>>>>>>>> 
>>>>>>>>>> Quick repro:
>>>>>>>>>> 
>>>>>>>>>> ➜ ~ pyenv local 3.5.5
>>>>>>>>>> ➜ ~ locale
>>>>>>>>>> LANG=
>>>>>>>>>> LC_COLLATE="C"
>>>>>>>>>> LC_CTYPE="C"
>>>>>>>>>> LC_MESSAGES="C"
>>>>>>>>>> LC_MONETARY="C"
>>>>>>>>>> LC_NUMERIC="C"
>>>>>>>>>> LC_TIME="C"
>>>>>>>>>> LC_ALL=
>>>>>>>>>> ➜ ~ cat testweird.txt
>>>>>>>>>> ’
>>>>>>>>>> ➜ ~ python
>>>>>>>>>> Python 3.5.5 (default, Jul 7 2018, 17:00:56)
>>>>>>>>>> [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on
>>> darwin
>>>>>>>>>> Type "help", "copyright", "credits" or "license" for more
>>> information.
>>>>>>>>>>>>> open('testweird.txt').read()
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File "<stdin>", line 1, in <module>
>>>>>>>>>> File "/Users/cjg/.pyenv/versions/3.5.5/lib/python3.5/encodings/
>>> ascii.py",
>>>>>>>>> line 26, in decode
>>>>>>>>>> return codecs.ascii_decode(input, self.errors)[0]
>>>>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in
>>> position 0:
>>>>>>>>> ordinal not in range(128)
>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Maybe not a blocking change but it is a breaking change from 1.9 I
>>> guess.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> / Carl Johan
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On 10 July 2018 at 14:48:47, Bolke de Bruin (bdbr...@gmail.com)
>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Carl,
>>>>>>>>>>> 
>>>>>>>>>>> Did you this on python 3.5 as well? 3.6 is not an officially
>>> supported
>>>>>>>>> (yet). As a workaround is available I won’t consider this blocking
>>> btw.
>>>>>>>>>>> 
>>>>>>>>>>> Bolke
>>>>>>>>>>> 
>>>>>>>>>>> Verstuurd vanaf mijn iPad
>>>>>>>>>>> 
>>>>>>>>>>> Op 10 jul. 2018 om 11:53 heeft Carl Johan Gustavsson <
>>>>>>>>> carl.j.gustavs...@gmail.com> het volgende geschreven:
>>>>>>>>>>> 
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> 
>>>>>>>>>>>> First of all, thank you for all the work with the release
>>> management.
>>>>>>>>>>>> 
>>>>>>>>>>>> I ran in to a weird issue testing the RC1, running under Python
>>> 3.6.0 /
>>>>>>>>> Ubuntu 14.04.5, upgrading from a master build from February.
>>>>>>>>>>>> 
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01
>>> Traceback
>>>>>>>>> (most recent call last):
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File
>>>>>>>>> "/opt/virtualenv/tictail/pipeline/bin/airflow", line 21, in
>>> <module>
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 from
>>> airflow
>>>>>>>>> import configuration
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File
>>>>>>>>> 
>>>>>>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-
>>> packages/airflow/__init__.py",
>>>>>>>>> line 35, in <module>
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 from
>>> airflow
>>>>>>>>> import configuration as conf
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File
>>>>>>>>> 
>>>>>>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-
>>> packages/airflow/configuration.py",
>>>>>>>>> line 106, in <module>
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01
>>>>>>>>> DEFAULT_CONFIG = f.read()
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File
>>>>>>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/encodings/ascii.py",
>>> line
>>>>>>>>> 26, in decode
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 return
>>>>>>>>> codecs.ascii_decode(input, self.errors)[0]
>>>>>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01
>>>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in
>>> position 20770:
>>>>>>>>> ordinal not in range(128)
>>>>>>>>>>>> 
>>>>>>>>>>>> Removing the ’ on
>>>>>>>>> 
>>>>>>>>> https://github.com/apache/incubator-airflow/blob/master/
>>> airflow/config_templates/default_airflow.cfg#L613
>>>>>>>>> solved the issue for me, and digging a bit deeper it seems Airflow
>>> now
>>>>>>>>> requires setting LC_ALL=en_US.UTF-8 in the environment or similar
>>> to force
>>>>>>>>> Python to read the file as utf-8 and not ascii. (I think this was
>>> changed
>>>>>>>>> in to default to utf-8 in Python 3.7).
>>>>>>>>>>>> 
>>>>>>>>>>>> I see 3 solutions for this
>>>>>>>>>>>> 1. Document that Airflow need to run with LC_ALL=en_US.UTF-8 or
>>>>>>>>> similar.
>>>>>>>>>>>> 2. Change the default config file to not contain non-ascii
>>> characters.
>>>>>>>>>>>> 3. Always read the file as unicode regardless of the LC_ALL
>>>>>>>>> environment, by the encoding='utf-8’ parameter to open().
>>>>>>>>>>>> 
>>>>>>>>>>>> I think 3 is the best solution, and I can prepare a PR for that
>>> if
>>>>>>>>> necessary .
>>>>>>>>>>>> 
>>>>>>>>>>>> I guess this counts as -1 (non-binding)
>>>>>>>>>>>> 
>>>>>>>>>>>> All the best
>>>>>>>>>>>> 
>>>>>>>>>>>> Carl Johan
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> On 8 July 2018 at 22:02:33, Bolke de Bruin (bdbr...@gmail.com)
>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I have cut Airflow 1.10.0 RC1. This email is calling a vote on
>>> the
>>>>>>>>> release,
>>>>>>>>>>>>> which will last for 72 hours. Consider this my (binding) +1.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Airflow 1.10.0 RC 1 is available at:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.
>>> 10.0rc1/ <
>>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc1/
>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> apache-airflow-1.10.0rc1+incubating-source.tar.gz is a source
>>> release
>>>>>>>>> that
>>>>>>>>>>>>> comes with INSTALL instructions.
>>>>>>>>>>>>> apache-airflow-1.10.0rc1+incubating-bin.tar.gz is the binary
>>> Python
>>>>>>>>> "sdist"
>>>>>>>>>>>>> release.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Public keys are available at:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/airflow/ <
>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/airflow/>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The amount of JIRAs fixed is over 700. Please have a look at the
>>>>>>>>> changelog.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Please note that the version number excludes the `rcX` string
>>> as well
>>>>>>>>>>>>> as the "+incubating" string, so it's now simply 1.10.0. This
>>> will
>>>>>>>>> allow us
>>>>>>>>>>>>> to rename the artifact without modifying the artifact checksums
>>> when
>>>>>>>>> we
>>>>>>>>>>>>> actually release.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Bolke
>>>>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 

Reply via email to