Could we make a push to close some of the open PRs and get them into a 1.10 release before focusing on 2.0?
I can make some time to review, if there are other folks that can follow up and merge the PRs. --George On Tue, Dec 19, 2017 at 2:42 PM Chris Riccomini <[email protected]> wrote: > > It does make sense to keep SQLite on the other hand a Docker image with > all the components might just be as convenient? > > I formed an actual opinion. :) IMO, SQLite is very convenient and should be > left in. > > > Dropping stuff would lessen the burden of maintenance. There is a lot of > cruft that is not used. This reduces the surface for bugs and makes it > easier to do the optimizations you refer too. However, I’m not married to a > 2.0 release now, but it does look fresh ;-). > > What do you guys think about doing a 1.10 in January/Feb that has timezone > and RBAC (new UI), and targeting 2.0 after that? > > On Mon, Dec 18, 2017 at 11:50 AM, Bolke de Bruin <[email protected]> > wrote: > > > Hi George, > > > > It does make sense to keep SQLite on the other hand a Docker image with > > all the components might just be as convenient? > > > > Dropping stuff would lessen the burden of maintenance. There is a lot of > > cruft that is not used. This reduces the surface for bugs and makes it > > easier to do the optimizations you refer too. However, I’m not married > to a > > 2.0 release now, but it does look fresh ;-). > > > > Bolke > > > > Verstuurd vanaf mijn iPad > > > > > Op 18 dec. 2017 om 19:48 heeft George Leslie-Waksman < > > [email protected]> het volgende geschreven: > > > > > > I really do not think we should drop sqlite support. We use sqlite for > > > local testing/development; I used it when doing my initial evaluation > of > > > Airflow; and there are regular comments in this group about people > using > > > sqlite locally for their workflows. It feels like a critical feature > for > > a > > > lot of use cases. > > > > > > I also feel like we're kind of jumping the gun on pushing for 2.0 (is > it > > > just because 9 is the last single digit?). There is a lot of the code > > base > > > that's pretty hairy and a number of things that are fairly > > non-performant / > > > buggy. I don't think that we NEED to break backward compatibility to > > clean > > > things up and fix things. I don't particularly like pickle or the old > > > import style but I don't feel like supporting them is terribly onerous > (I > > > could be wrong). > > > > > > New webserver, releasing the API, and even timezones are all things > that > > > can be released in a non-breaking fashion. > > > > > > --George > > > > > >> On Thu, Dec 14, 2017 at 2:20 PM Alek Storm <[email protected]> > > wrote: > > >> > > >> We use sqlite for developing Airflow DAGs locally, and doing basic > > checks > > >> for syntax/import errors in our CI/CD pipeline. > > >> > > >> Alek > > >> > > >> On Thu, Dec 14, 2017 at 4:19 PM, Chris Riccomini < > [email protected] > > > > > >> wrote: > > >> > > >>> Agree with Bolke that it's a good idea, but major work. I will push > > back > > >> on > > >>> this for 2.0 mostly due to time concerns. I don't want 2.0 to take > > months > > >>> and months to get out. > > >>> > > >>>> Btw: what about dropping sqlite support? > > >>> > > >>> I'm fine with this, but isn't this really useful for demos? Seems to > me > > >>> that the out of the box experience from airflow is one of the things > > that > > >>> make sit so compelling. > > >>> > > >>> On Thu, Dec 14, 2017 at 2:12 PM, Bolke de Bruin <[email protected]> > > >> wrote: > > >>> > > >>>> That will take quite some work. It is a a good idea but also a major > > >>>> change. Not sure if we should target that. > > >>>> > > >>>> Btw: what about dropping sqlite support? > > >>>> > > >>>> Verstuurd vanaf mijn iPad > > >>>> > > >>>>> Op 14 dec. 2017 om 21:19 heeft Gael Magnan <[email protected]> > > >> het > > >>>> volgende geschreven: > > >>>>> > > >>>>> Hi, > > >>>>> > > >>>>> haven't been following much lately but on the import side of > things, > > >>>> isn't > > >>>>> Airflow 2 the best moment to change to a pip plugin system for > > >> imports > > >>> of > > >>>>> third party stuff? > > >>>>> I.E being able to add a new type of credentials, operator etc.. > > >> without > > >>>>> touching to the airflow code itself or having them in a special > > >> folder. > > >>>>> > > >>>>> Regards > > >>>>> Gael > > >>>>> > > >>>>> > > >>>>> > > >>>>> Le jeu. 14 déc. 2017 à 14:17, Chris Riccomini < > [email protected] > > >>> > > >>> a > > >>>>> écrit : > > >>>>> > > >>>>>> I'm fine with sensor refactor. Added to Wiki. > > >>>>>> > > >>>>>> On Thu, Dec 14, 2017 at 11:16 AM, Chris Riccomini < > > >>>> [email protected]> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> @Bolke, > > >>>>>>> > > >>>>>>>> Should we, before 2.0, start the graduation from the incubator? > > >>>>>>> > > >>>>>>> No, I'd rather keep them separate. We can certainly start > > >> graduation, > > >>>> but > > >>>>>>> I don't want to block 2.0. Can pursue them in parallel. > > >>>>>>> > > >>>>>>> On Thu, Dec 14, 2017 at 11:15 AM, Andy Hadjigeorgiou < > > >>>>>> [email protected] > > >>>>>>>> wrote: > > >>>>>>> > > >>>>>>>> Does it make sense to include sensors.py refactor in 2.0, so we > > >> can > > >>>>>>>> retire the old import structure easily and support the new > sensors > > >>>>>> package > > >>>>>>>> import structure? > > >>>>>>>> > > >>>>>>>> - Andy > > >>>>>>>> > > >>>>>>>> On Thu, Dec 14, 2017 at 2:12 PM, Driesprong, Fokko > > >>>> <[email protected] > > >>>>>>> > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> Hi all, > > >>>>>>>>> > > >>>>>>>>> Good initiative. I would be happy to refactor the sensors > > >> package. > > >>> I > > >>>>>>>>> started on it but it changes a lot, all the imports will break. > > >>>>>>>>> > > >>>>>>>>> https://github.com/apache/incubator-airflow/pull/2875 > > >>>>>>>>> > > >>>>>>>>> What do you guys think? > > >>>>>>>>> > > >>>>>>>>> Cheers, Fokko > > >>>>>>>>> > > >>>>>>>>> 2017-12-14 20:09 GMT+01:00 Chris Riccomini < > > >> [email protected] > > >>>> : > > >>>>>>>>> > > >>>>>>>>>> I have created a wiki here: > > >>>>>>>>>> > > >>>>>>>>>> > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0 > > >>>>>>>>>> > > >>>>>>>>>> To track features and progress. > > >>>>>>>>>> > > >>>>>>>>>> On Thu, Dec 14, 2017 at 11:08 AM, Chris Riccomini < > > >>>>>>>>> [email protected]> > > >>>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>>> Re: #2: Is there a current ticket out for removing the > legacy > > >>>>>>>>> import > > >>>>>>>>>>> style? > > >>>>>>>>>>> > > >>>>>>>>>>> No, I don't think so, but you can create one! :) > > >>>>>>>>>>> > > >>>>>>>>>>> On Thu, Dec 14, 2017 at 11:06 AM, Andy Hadjigeorgiou < > > >>>>>>>>>> [email protected] > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> This sounds great, something I'd like to see updated for 2.0 > > >>>>>>>>> release (or > > >>>>>>>>>>>> before) is the Airflow documentation > > >>>>>>>>>>>> <http://airflow.readthedocs.io/en/latest/installation.html> > ( > > >>>>>>>>>>>> http://airflow.readthedocs.io/en/latest/installation.html). > > >> It > > >>>>>>>>> seems > > >>>>>>>>>> that > > >>>>>>>>>>>> updating the repo does not update this site - and given that > > >> we > > >>>>>>>>> will be > > >>>>>>>>>>>> removing certain deprecated features I imagine the docs will > > >>>>>> change > > >>>>>>>>>>>> substantially. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Re: #2: Is there a current ticket out for removing the > legacy > > >>>>>> import > > >>>>>>>>>>>> style? > > >>>>>>>>>>>> I'm happy to help drive that forward. > > >>>>>>>>>>>> > > >>>>>>>>>>>> - Andy > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Thu, Dec 14, 2017 at 1:55 PM, Chris Riccomini < > > >>>>>>>>> [email protected] > > >>>>>>>>>>> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Hey all, > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> With 1.9.0 wrapping up soon (hopefully), there's been some > > >>>>>>>>> discussion > > >>>>>>>>>> on > > >>>>>>>>>>>>> the having the next release be Airflow 2.0 (rather than > > >> 1.10). > > >>>>>>>>> This > > >>>>>>>>>>>> would > > >>>>>>>>>>>>> allow us to break compatibility, and clean up some stuff. > > >>>>>> Proposed > > >>>>>>>>>>>> things > > >>>>>>>>>>>>> to include in 2.0 are: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> 1) New webserver that Joy Gao has been working on. > > >>>>>>>>>>>>> 2) Remove the legacy import style that's been deprecated > > >> since > > >>>>>> at > > >>>>>>>>>> least > > >>>>>>>>>>>> 1.8 > > >>>>>>>>>>>>> 3) New timzone feature > > >>>>>>>>>>>>> 4) Move API out of experimental > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I want to keep the list fairly tight, preferably to things > > >> that > > >>>>>>>>> have > > >>>>>>>>>>>>> already been done, so that we can ship it fairly quickly > (in > > >>> the > > >>>>>>>>> next > > >>>>>>>>>>>>> couple of months). > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Does this sound like a good list? > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Cheers, > > >>>>>>>>>>>>> Chris > > >>>>>> > > >>>> > > >>> > > >> > > >
