I created AIP-21 https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths This is the same document as above, I just deleted my opinions (summary sections) and work plan.
On Fri, Apr 19, 2019 at 2:18 PM Jarek Potiuk <[email protected]> wrote: > Thanks Kamil! Excellent analysis! > > My preferences: > > Case 1: (contrib vs not) > B (move all that are "well" tested): I think having "incubator" kind of > approach for not super tested/documented but still usable code is useful. > Though we would have to establish some objective criteria and describe a > "growing from incubation" process. > > Case 2 airflow.operators.foo_operator.FooOperator could become > A: airflow.operators.foo.FooOperator > > Case 3: airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator > could become > C: airflow.integration.gcp.operators.bigtable.BigTableOperator > > Case 4: (separate packages/namesapces) > B: no namespaces - i think we are very far from modularisation to happen > (and how) and it might get never implemented so there are no real benefits > I see now. > > Case 5: "Operator" suffix on class names > B: I prefer to keep B as this will be much easier when you search for class > using your IDE. > > Case 7: Zope deprecation or not > B: I prefer to use Kamil's in-house hack than zope, but I feel we could > improve it slightly to remove a bit of boilerplate - it looks like the > warning.warn() message should be very similar in all deprecation cases and > possibly we can define it in one place (providing that DepracationWarning > will continue to work well with IDEs) > > > > > On Fri, Apr 19, 2019 at 12:36 AM Kamil Breguła <[email protected]> > wrote: > > > Inline comment > > > > On Fri, Apr 19, 2019 at 12:23 AM Arthur Wiedmer < > [email protected]> > > wrote: > > > > > > > > > > > > Case #5 Are we talking about the class name or the file name? > The > > > > class > > > > > > > name is fine to me, but we could remove _operator from the file > > > name. > > > > > > > > > > > > > > > > > > > Case #2 describes the change of file names > > > > > > Case #5 describes the change of class names > > > > > > > > > > > > I added examples to two cases to better describe the changes. > > > > > > > > > > > > > > > > Thank you. I guess I see it for file names, but I am wondering > about > > > the > > > > > operators and name collisions. > > > > > Say I need both the HiveOperator and I inline a custom operator for > > > > which I > > > > > need a hive hook. > > > > > # Would you recommmend the following? Or does case #5 only apply to > > > > > Operators? > > > > > from airflow.operators.hive import Hive as HiveOperator > > > > > from airfow.hooks.hive import Hive as HiveHook > > > > > > > > > > I was just thinking it might be nice to be able to import what I > need > > > > > without renaming and not worry too much about names shadowing > others. > > > > > Especially if I am new-ish to Apache Airflow. > > > > > > > > > > > > > > I think that the operator should describe the behavior(verb), hook > > should > > > > describe a service(noun). These are other parts of speech. > > > > In my opinion, HiveOperator should be named HiveExecuteQuery. Hook > will > > > be > > > > called Hive. Then there will be no name conflicts. > > > > > > > > > > But we will still provide the backwards compatibility for a while with > > > aliases to the old names, correct? > > > > > > > Yes. It is possible to provide backward compatibility also in this case. > > > > > > -- > > > > Kamil Breguła > > Polidea <https://www.polidea.com/> | Software Engineer > > > > M: +48 505 458 451 <+48505458451> > > E: [email protected] > > [image: Polidea] <https://www.polidea.com/> > > > > We create human & business stories through technology. > > Check out our projects! <https://www.polidea.com/our-work> > > [image: Github] <https://github.com/Polidea> [image: Facebook] > > <https://www.facebook.com/Polidea.Software> [image: Twitter] > > <https://twitter.com/polidea> [image: Linkedin] > > <https://www.linkedin.com/company/polidea> [image: Instagram] > > <https://instagram.com/polidea> [image: Behance] > > <https://www.behance.net/polidea> > > > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > E: [email protected] > -- Kamil Breguła Polidea <https://www.polidea.com/> | Software Engineer M: +48 505 458 451 <+48505458451> E: [email protected] [image: Polidea] <https://www.polidea.com/> We create human & business stories through technology. Check out our projects! <https://www.polidea.com/our-work> [image: Github] <https://github.com/Polidea> [image: Facebook] <https://www.facebook.com/Polidea.Software> [image: Twitter] <https://twitter.com/polidea> [image: Linkedin] <https://www.linkedin.com/company/polidea> [image: Instagram] <https://instagram.com/polidea> [image: Behance] <https://www.behance.net/polidea>
