> It does make sense to keep SQLite on the other hand a Docker image with all the components might just be as convenient?
I formed an actual opinion. :) IMO, SQLite is very convenient and should be left in. > Dropping stuff would lessen the burden of maintenance. There is a lot of cruft that is not used. This reduces the surface for bugs and makes it easier to do the optimizations you refer too. However, I’m not married to a 2.0 release now, but it does look fresh ;-). What do you guys think about doing a 1.10 in January/Feb that has timezone and RBAC (new UI), and targeting 2.0 after that? On Mon, Dec 18, 2017 at 11:50 AM, Bolke de Bruin <[email protected]> wrote: > Hi George, > > It does make sense to keep SQLite on the other hand a Docker image with > all the components might just be as convenient? > > Dropping stuff would lessen the burden of maintenance. There is a lot of > cruft that is not used. This reduces the surface for bugs and makes it > easier to do the optimizations you refer too. However, I’m not married to a > 2.0 release now, but it does look fresh ;-). > > Bolke > > Verstuurd vanaf mijn iPad > > > Op 18 dec. 2017 om 19:48 heeft George Leslie-Waksman < > [email protected]> het volgende geschreven: > > > > I really do not think we should drop sqlite support. We use sqlite for > > local testing/development; I used it when doing my initial evaluation of > > Airflow; and there are regular comments in this group about people using > > sqlite locally for their workflows. It feels like a critical feature for > a > > lot of use cases. > > > > I also feel like we're kind of jumping the gun on pushing for 2.0 (is it > > just because 9 is the last single digit?). There is a lot of the code > base > > that's pretty hairy and a number of things that are fairly > non-performant / > > buggy. I don't think that we NEED to break backward compatibility to > clean > > things up and fix things. I don't particularly like pickle or the old > > import style but I don't feel like supporting them is terribly onerous (I > > could be wrong). > > > > New webserver, releasing the API, and even timezones are all things that > > can be released in a non-breaking fashion. > > > > --George > > > >> On Thu, Dec 14, 2017 at 2:20 PM Alek Storm <[email protected]> > wrote: > >> > >> We use sqlite for developing Airflow DAGs locally, and doing basic > checks > >> for syntax/import errors in our CI/CD pipeline. > >> > >> Alek > >> > >> On Thu, Dec 14, 2017 at 4:19 PM, Chris Riccomini <[email protected] > > > >> wrote: > >> > >>> Agree with Bolke that it's a good idea, but major work. I will push > back > >> on > >>> this for 2.0 mostly due to time concerns. I don't want 2.0 to take > months > >>> and months to get out. > >>> > >>>> Btw: what about dropping sqlite support? > >>> > >>> I'm fine with this, but isn't this really useful for demos? Seems to me > >>> that the out of the box experience from airflow is one of the things > that > >>> make sit so compelling. > >>> > >>> On Thu, Dec 14, 2017 at 2:12 PM, Bolke de Bruin <[email protected]> > >> wrote: > >>> > >>>> That will take quite some work. It is a a good idea but also a major > >>>> change. Not sure if we should target that. > >>>> > >>>> Btw: what about dropping sqlite support? > >>>> > >>>> Verstuurd vanaf mijn iPad > >>>> > >>>>> Op 14 dec. 2017 om 21:19 heeft Gael Magnan <[email protected]> > >> het > >>>> volgende geschreven: > >>>>> > >>>>> Hi, > >>>>> > >>>>> haven't been following much lately but on the import side of things, > >>>> isn't > >>>>> Airflow 2 the best moment to change to a pip plugin system for > >> imports > >>> of > >>>>> third party stuff? > >>>>> I.E being able to add a new type of credentials, operator etc.. > >> without > >>>>> touching to the airflow code itself or having them in a special > >> folder. > >>>>> > >>>>> Regards > >>>>> Gael > >>>>> > >>>>> > >>>>> > >>>>> Le jeu. 14 déc. 2017 à 14:17, Chris Riccomini <[email protected] > >>> > >>> a > >>>>> écrit : > >>>>> > >>>>>> I'm fine with sensor refactor. Added to Wiki. > >>>>>> > >>>>>> On Thu, Dec 14, 2017 at 11:16 AM, Chris Riccomini < > >>>> [email protected]> > >>>>>> wrote: > >>>>>> > >>>>>>> @Bolke, > >>>>>>> > >>>>>>>> Should we, before 2.0, start the graduation from the incubator? > >>>>>>> > >>>>>>> No, I'd rather keep them separate. We can certainly start > >> graduation, > >>>> but > >>>>>>> I don't want to block 2.0. Can pursue them in parallel. > >>>>>>> > >>>>>>> On Thu, Dec 14, 2017 at 11:15 AM, Andy Hadjigeorgiou < > >>>>>> [email protected] > >>>>>>>> wrote: > >>>>>>> > >>>>>>>> Does it make sense to include sensors.py refactor in 2.0, so we > >> can > >>>>>>>> retire the old import structure easily and support the new sensors > >>>>>> package > >>>>>>>> import structure? > >>>>>>>> > >>>>>>>> - Andy > >>>>>>>> > >>>>>>>> On Thu, Dec 14, 2017 at 2:12 PM, Driesprong, Fokko > >>>> <[email protected] > >>>>>>> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi all, > >>>>>>>>> > >>>>>>>>> Good initiative. I would be happy to refactor the sensors > >> package. > >>> I > >>>>>>>>> started on it but it changes a lot, all the imports will break. > >>>>>>>>> > >>>>>>>>> https://github.com/apache/incubator-airflow/pull/2875 > >>>>>>>>> > >>>>>>>>> What do you guys think? > >>>>>>>>> > >>>>>>>>> Cheers, Fokko > >>>>>>>>> > >>>>>>>>> 2017-12-14 20:09 GMT+01:00 Chris Riccomini < > >> [email protected] > >>>> : > >>>>>>>>> > >>>>>>>>>> I have created a wiki here: > >>>>>>>>>> > >>>>>>>>>> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0 > >>>>>>>>>> > >>>>>>>>>> To track features and progress. > >>>>>>>>>> > >>>>>>>>>> On Thu, Dec 14, 2017 at 11:08 AM, Chris Riccomini < > >>>>>>>>> [email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>>> Re: #2: Is there a current ticket out for removing the legacy > >>>>>>>>> import > >>>>>>>>>>> style? > >>>>>>>>>>> > >>>>>>>>>>> No, I don't think so, but you can create one! :) > >>>>>>>>>>> > >>>>>>>>>>> On Thu, Dec 14, 2017 at 11:06 AM, Andy Hadjigeorgiou < > >>>>>>>>>> [email protected] > >>>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> This sounds great, something I'd like to see updated for 2.0 > >>>>>>>>> release (or > >>>>>>>>>>>> before) is the Airflow documentation > >>>>>>>>>>>> <http://airflow.readthedocs.io/en/latest/installation.html> ( > >>>>>>>>>>>> http://airflow.readthedocs.io/en/latest/installation.html). > >> It > >>>>>>>>> seems > >>>>>>>>>> that > >>>>>>>>>>>> updating the repo does not update this site - and given that > >> we > >>>>>>>>> will be > >>>>>>>>>>>> removing certain deprecated features I imagine the docs will > >>>>>> change > >>>>>>>>>>>> substantially. > >>>>>>>>>>>> > >>>>>>>>>>>> Re: #2: Is there a current ticket out for removing the legacy > >>>>>> import > >>>>>>>>>>>> style? > >>>>>>>>>>>> I'm happy to help drive that forward. > >>>>>>>>>>>> > >>>>>>>>>>>> - Andy > >>>>>>>>>>>> > >>>>>>>>>>>> On Thu, Dec 14, 2017 at 1:55 PM, Chris Riccomini < > >>>>>>>>> [email protected] > >>>>>>>>>>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hey all, > >>>>>>>>>>>>> > >>>>>>>>>>>>> With 1.9.0 wrapping up soon (hopefully), there's been some > >>>>>>>>> discussion > >>>>>>>>>> on > >>>>>>>>>>>>> the having the next release be Airflow 2.0 (rather than > >> 1.10). > >>>>>>>>> This > >>>>>>>>>>>> would > >>>>>>>>>>>>> allow us to break compatibility, and clean up some stuff. > >>>>>> Proposed > >>>>>>>>>>>> things > >>>>>>>>>>>>> to include in 2.0 are: > >>>>>>>>>>>>> > >>>>>>>>>>>>> 1) New webserver that Joy Gao has been working on. > >>>>>>>>>>>>> 2) Remove the legacy import style that's been deprecated > >> since > >>>>>> at > >>>>>>>>>> least > >>>>>>>>>>>> 1.8 > >>>>>>>>>>>>> 3) New timzone feature > >>>>>>>>>>>>> 4) Move API out of experimental > >>>>>>>>>>>>> > >>>>>>>>>>>>> I want to keep the list fairly tight, preferably to things > >> that > >>>>>>>>> have > >>>>>>>>>>>>> already been done, so that we can ship it fairly quickly (in > >>> the > >>>>>>>>> next > >>>>>>>>>>>>> couple of months). > >>>>>>>>>>>>> > >>>>>>>>>>>>> Does this sound like a good list? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>> Chris > >>>>>> > >>>> > >>> > >> >
