Yes, that's true. But I expect the 1.x to be a subset of the 2.0 migrations, so somewhere it went south.
Right now we have 3 heads in 1.10.6rc2: MacBook-Pro-van-Fokko:airflow fokkodriesprong$ git checkout 1.10.6rc2 MacBook-Pro-van-Fokko:airflow fokkodriesprong$ alembic heads a56c9515abdc (head) 004c1210f153 (head) 74effc47d867 (head) Full history: c8ffec048a3b -> a56c9515abdc (head), Remove dag_stat table 939bb1e647c8 -> 004c1210f153 (head), increase queue name size limit 6e96a59344a4 -> 74effc47d867 (head), change datetime to datetime2(6) on MSSQL tables 939bb1e647c8 -> 6e96a59344a4, Make TaskInstance.pool not nullable dd4ecb8fbee3 -> 939bb1e647c8 (branchpoint), task reschedule fk on cascade delete c8ffec048a3b -> dd4ecb8fbee3, Add schedule interval to dag 41f5f12752f8 -> c8ffec048a3b (branchpoint), add fields to dag 03bc53e68815 -> 41f5f12752f8, add superuser field 0a2a5b66e19d, bf00311e1990 -> 03bc53e68815 (mergepoint), merge_heads_2 9635ae0956e7 -> 0a2a5b66e19d, add task_reschedule table dd25f486b8ea -> bf00311e1990, add index to taskinstance 9635ae0956e7 -> dd25f486b8ea, add idx_log_dag 856955da8476 -> 9635ae0956e7 (branchpoint), index-faskfail f23433877c24 -> 856955da8476, fix sqlite foreign key 05f30312d566 -> f23433877c24, fix mysql not null constraint 86770d1215c0, 0e2a74e0fc9f -> 05f30312d566 (mergepoint), merge heads 27c6a30d7c24 -> 86770d1215c0, add kubernetes scheduler uniqueness 33ae817a1ff4 -> 27c6a30d7c24, kubernetes_resource_checkpointing d2ae31099d61 -> 33ae817a1ff4, kubernetes_resource_checkpointing d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness 947454bf1dff -> d2ae31099d61 (branchpoint), Increase text size for MySQL (not relevant for other DBs' text types) bdaa763e6c56 -> 947454bf1dff, add ti job_id index cc1e65623dc7 -> bdaa763e6c56, Make xcom value column a large binary 127d2bf2dfa7 -> cc1e65623dc7, add max tries column to task instance 5e7d17757c7a -> 127d2bf2dfa7, Add dag_id/state index on dag_run table 8504051e801b -> 5e7d17757c7a, add pid field to TaskInstance 4addfa1236f1 -> 8504051e801b, xcom dag task indices f2ca10b85618 -> 4addfa1236f1, Add fractional seconds to mysql tables 64de9cddf6c9 -> f2ca10b85618, add dag_stats table 211e584da130 -> 64de9cddf6c9, add task fails journal table 2e82aab8ef20 -> 211e584da130, add TI state index 1968acfc09e3 -> 2e82aab8ef20, rename user table bba5a7cfc896 -> 1968acfc09e3, add is_encrypted column to variable table bbc73705a13e -> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field in connection 4446e08588 -> bbc73705a13e, Add notification_sent column to sla_miss 561833c1c74b -> 4446e08588, dagrun start end 40e67319e3a9 -> 561833c1c74b, add password column to user 2e541a1dcfed -> 40e67319e3a9, dagrun_config 1b38cef5b76e -> 2e541a1dcfed, task_duration 502898887f84 -> 1b38cef5b76e, add dagrun 52d714495f0 -> 502898887f84, Adding extra to Log 338e90f54d61 -> 52d714495f0, job_id indices 13eb55f81627 -> 338e90f54d61, More logging into task_instance 1507a7289a2f -> 13eb55f81627, maintain history for compatibility with earlier migrations e3a246e0dc1 -> 1507a7289a2f, create is_encrypted <base> -> e3a246e0dc1, current schema Cheers, Fokko Op zo 27 okt. 2019 om 16:48 schreef Kaxil Naik <kaxiln...@gmail.com>: > As 2.0 is not released, if migration are broken, we can still fix them on > the mater itself. > > > > On Sun, Oct 27, 2019, 15:29 Driesprong, Fokko <fo...@driesprong.frl> > wrote: > > > The problem is that if you upgrade from the 1.10 branch to 2.0, the > > migrations don't work. To probably the migrations in 1.10 and 2.0 have > > diverged somewhere. > > > > The following can be used to reproduce this: > > git pull upstream master # ensure we have latest > > git checkout v-10-stable # 1.10.6rc2 > > rm ~/airflow/airflow.db > > airflow initdb > > git checkout master > > airflow db upgrade > > > > > > > > Op zo 27 okt. 2019 om 15:53 schreef Ash Berlin-Taylor <a...@apache.org>: > > > > > That PR you mentioned isn't yet in a release, so can we not just edit > > that > > > migration in place? > > > > > > I'm not quite following (I didn't get much sleep) but I don't see how > > > broken code on master affects this release? > > > > > > -a > > > > > > > On 27 Oct 2019, at 14:28, Driesprong, Fokko <fo...@driesprong.frl> > > > wrote: > > > > > > > > I'm having second thoughts on my +1 vote. > > > > > > > > It turns out that the database migrations are broken from 1.10.6 to > > > 2.0.0: > > > > > > > > > > https://github.com/apache/airflow/pull/6370/files/c9b9312a60f891475d3072584171c2af56246829#r339287344 > > > > > > > > So we either need to release a 1.10.7 and force users to migrate to > > that > > > > version first, before going to 2.0.0, otherwise, it won't work. I've > > > opened > > > > up a PR to fix this for 2.0.0: > > > https://github.com/apache/airflow/pull/6442 > > > > > > > > Cheers, Fokko > > > > > > > > > > > > Op za 26 okt. 2019 om 09:23 schreef Jarek Potiuk < > > > jarek.pot...@polidea.com>: > > > > > > > >> +1 (binding) > > > >> > > > >> Tested on Python 3.7/Python 2.7 using local installation + double > > > checked > > > >> sources with RAT licence tool (All good this time!). > > > >> > > > >> Regarding why AIRFLOW-5746 ( > > https://github.com/apache/airflow/pull/6434 > > > ) > > > >> is > > > >> being reverted (or rather being improved on). I don't think it's a > > > blocker > > > >> for this release. > > > >> > > > >> It is just a test change - the tests how they were written, broke > some > > > >> development environments (the airflow resetdb was failing in case > > > >> PYTHONPATH was not specially crafted). The current test does not > > really > > > >> test what it should (passing PYTHONPATH to impersonated tasks) but > the > > > >> functionality (impersonation + path) has not actually changed and it > > > works. > > > >> The test worked fine before the change was added - it's just imports > > in > > > >> tests that have been changed. > > > >> > > > >> Kamil (thanks!) already submitted a fix for that, that solves it > > > >> permanently https://github.com/apache/airflow/pull/6436/ - will > > review > > > and > > > >> merge soon. But it's not a blocker IMHO. > > > >> > > > >> J. > > > >> > > > >> On Sat, Oct 26, 2019 at 2:10 AM Kaxil Naik <kaxiln...@gmail.com> > > wrote: > > > >> > > > >>> +1 (binding) > > > >>> > > > >>> On Sat, Oct 26, 2019 at 12:05 AM Driesprong, Fokko > > > <fo...@driesprong.frl > > > >>> > > > >>> wrote: > > > >>> > > > >>>> +1 binding from my side > > > >>>> > > > >>>> Ran an example DAGs with Docker using Python 3.7. > > > >>>> > > > >>>> We might need to check why AIRFLOW-5746 is being reverted: > > > >>>> https://github.com/apache/airflow/pull/6434 > > > >>>> > > > >>>> If there is another RC, I'd like to request to cherry-pick > > > >>>> https://github.com/apache/airflow/pull/6370 onto the 1.10 branch. > > > >>>> > > > >>>> Cheers, Fokko > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>>> Op vr 25 okt. 2019 om 23:04 schreef Ash Berlin-Taylor < > > a...@apache.org > > > >>> : > > > >>>> > > > >>>>> Hey all, > > > >>>>> > > > >>>>> I have cut Airflow 1.10.6 RC2. This email is calling a vote on > the > > > >>>>> release, which will last for 72 hours, until Monday 28, October > > 22nd > > > >> at > > > >>>>> 21:05 UTC. > > > >>>>> > > > >>>>> Consider this my (binding) +1. > > > >>>>> > > > >>>>> Airflow 1.10.6 RC1 is available at: < > > > >>>>> https://dist.apache.org/repos/dist/dev/airflow/1.10.6rc2/> > > > >>>>> > > > >>>>> *apache-airflow-1.10.6rc2-source.tar.gz* is a source release that > > > >> comes > > > >>>>> with INSTALL instructions. > > > >>>>> *apache-airflow-1.10.6rc2-bin.tar.gz* is the binary Python > "sdist" > > > >>>> release. > > > >>>>> *apache_airflow-1.10.6rc2-py2.py3-none-any.whl* is the binary > > Python > > > >>>>> "wheel" release. > > > >>>>> > > > >>>>> Public keys are available at: < > > > >>>>> https://dist.apache.org/repos/dist/release/airflow/KEYS> > > > >>>>> > > > >>>>> As per normal the rc1 is available for testing from PyPi. > > > >>>>> > > > >>>>> Only votes from PMC members are binding, but members of the > > community > > > >>> are > > > >>>>> encouraged to test the release and vote with "(non-binding)". > > > >>>>> > > > >>>>> Please note that the version number excludes the `rcX` string, so > > > >> it's > > > >>>> now > > > >>>>> simply 1.10.6. This will allow us to rename the artifact without > > > >>>> modifying > > > >>>>> the artifact checksums when we actually release. > > > >>>>> > > > >>>>> The changes since RC1 are to fix License issues, ensure tests are > > > >>> running > > > >>>>> on Py2 (they weren't, but the only py3 bits that crept in were in > > the > > > >>>> test > > > >>>>> files luckily.). > > > >>>>> > > > >>>>> Changelog since 1.10.6rc1: > > > >>>>> > > > >>>>> * 73bf71835 [AIRFLOW-XXX] Update date in changelog [Ash > > > >> Berlin-Taylor] > > > >>>>> * 143b43151 [AIRFLOW-5750] Licence check is done also for > > > >>> non-executable > > > >>>>> .sh (#6425) [Jarek Potiuk] > > > >>>>> * 544f2b336 [AIRFLOW-5754] Improved RAT checking (#6429) [Jarek > > > >> Potiuk] > > > >>>>> * 7904669ca [AIRFLOW-5755] Fixed most problems with py27 [Jarek > > > >> Potiuk] > > > >>>>> * d601752c4 [AIRFLOW-5748] Remove python auto-detection (#6423) > > > >> [Jarek > > > >>>>> Potiuk] > > > >>>>> * 71e20417f [AIRFLOW-5746] Fix problems with static checks > (#6420) > > > >>> [Jarek > > > >>>>> Potiuk] > > > >>>>> * 7a6adad60 [AIRFLOW-5746] move FakeDateTime into the only place > it > > > >> is > > > >>>>> used (#6416) [Michael R. Crusoe] > > > >>>>> * e30fb85ca [AIRFLOW-5745] Breeze complete has now licence > (#6415) > > > >>> [Jarek > > > >>>>> Potiuk] > > > >>>>> > > > >>>>> Files changes since rc1: > > > >>>>> > > > >>>>> airflow ❯ git diff --stat 1.10.6rc1...1.10.6rc2 > > > >>>>> .pre-commit-config.yaml | 1 - > > > >>>>> .rat-excludes | 1 + > > > >>>>> .travis.yml | 47 > > > >>>>> +++++++++++++++++++++++++++++++++++++++-------- > > > >>>>> CHANGELOG.txt | 2 +- > > > >>>>> Dockerfile-checklicence | 2 +- > > > >>>>> airflow/models/dag.py | 5 ++++- > > > >>>>> breeze | 1 + > > > >>>>> breeze-complete | 17 > +++++++++++++++++ > > > >>>>> common/_autodetect_variables.sh | 49 > > > >>>>> +++++++++++++------------------------------------ > > > >>>>> files/x | 0 > > > >>>>> files/y | 0 > > > >>>>> scripts/ci/_utils.sh | 16 > ++++++++++++++-- > > > >>>>> scripts/ci/ci_check_license.sh | 1 + > > > >>>>> scripts/ci/ci_run_airflow_testing.sh | 2 +- > > > >>>>> scripts/ci/in_container/run_check_licence.sh | 2 +- > > > >>>>> tests/contrib/hooks/test_ssh_hook.py | 3 ++- > > > >>>>> tests/dags/test_impersonation_custom.py | 12 +++++++++++- > > > >>>>> tests/models/test_baseoperator.py | 9 ++++----- > > > >>>>> tests/models/test_dag.py | 4 ++-- > > > >>>>> tests/test_sentry.py | 2 +- > > > >>>>> tests/test_utils/fake_datetime.py | 29 > > > >>>>> ----------------------------- > > > >>>>> tests/test_utils/system_tests_class.py | 6 +++++- > > > >>>>> tests/www/test_views.py | 6 +++--- > > > >>>>> 23 files changed, 122 insertions(+), 95 deletions(-) > > > >>>>> > > > >>>>> > > > >>>> > > > >>> > > > >> > > > >> > > > >> -- > > > >> > > > >> Jarek Potiuk > > > >> Polidea <https://www.polidea.com/> | Principal Software Engineer > > > >> > > > >> M: +48 660 796 129 <+48660796129> > > > >> [image: Polidea] <https://www.polidea.com/> > > > >> > > > > > > > > >
