Thanks Kamil! Excellent analysis! My preferences:
Case 1: (contrib vs not) B (move all that are "well" tested): I think having "incubator" kind of approach for not super tested/documented but still usable code is useful. Though we would have to establish some objective criteria and describe a "growing from incubation" process. Case 2 airflow.operators.foo_operator.FooOperator could become A: airflow.operators.foo.FooOperator Case 3: airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator could become C: airflow.integration.gcp.operators.bigtable.BigTableOperator Case 4: (separate packages/namesapces) B: no namespaces - i think we are very far from modularisation to happen (and how) and it might get never implemented so there are no real benefits I see now. Case 5: "Operator" suffix on class names B: I prefer to keep B as this will be much easier when you search for class using your IDE. Case 7: Zope deprecation or not B: I prefer to use Kamil's in-house hack than zope, but I feel we could improve it slightly to remove a bit of boilerplate - it looks like the warning.warn() message should be very similar in all deprecation cases and possibly we can define it in one place (providing that DepracationWarning will continue to work well with IDEs) On Fri, Apr 19, 2019 at 12:36 AM Kamil Breguła <[email protected]> wrote: > Inline comment > > On Fri, Apr 19, 2019 at 12:23 AM Arthur Wiedmer <[email protected]> > wrote: > > > > > > > > > Case #5 Are we talking about the class name or the file name? The > > > class > > > > > > name is fine to me, but we could remove _operator from the file > > name. > > > > > > > > > > > > > > > > Case #2 describes the change of file names > > > > > Case #5 describes the change of class names > > > > > > > > > > I added examples to two cases to better describe the changes. > > > > > > > > > > > > > Thank you. I guess I see it for file names, but I am wondering about > > the > > > > operators and name collisions. > > > > Say I need both the HiveOperator and I inline a custom operator for > > > which I > > > > need a hive hook. > > > > # Would you recommmend the following? Or does case #5 only apply to > > > > Operators? > > > > from airflow.operators.hive import Hive as HiveOperator > > > > from airfow.hooks.hive import Hive as HiveHook > > > > > > > > I was just thinking it might be nice to be able to import what I need > > > > without renaming and not worry too much about names shadowing others. > > > > Especially if I am new-ish to Apache Airflow. > > > > > > > > > > > I think that the operator should describe the behavior(verb), hook > should > > > describe a service(noun). These are other parts of speech. > > > In my opinion, HiveOperator should be named HiveExecuteQuery. Hook will > > be > > > called Hive. Then there will be no name conflicts. > > > > > > > But we will still provide the backwards compatibility for a while with > > aliases to the old names, correct? > > > > Yes. It is possible to provide backward compatibility also in this case. > > > -- > > Kamil Breguła > Polidea <https://www.polidea.com/> | Software Engineer > > M: +48 505 458 451 <+48505458451> > E: [email protected] > [image: Polidea] <https://www.polidea.com/> > > We create human & business stories through technology. > Check out our projects! <https://www.polidea.com/our-work> > [image: Github] <https://github.com/Polidea> [image: Facebook] > <https://www.facebook.com/Polidea.Software> [image: Twitter] > <https://twitter.com/polidea> [image: Linkedin] > <https://www.linkedin.com/company/polidea> [image: Instagram] > <https://instagram.com/polidea> [image: Behance] > <https://www.behance.net/polidea> > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> E: [email protected]
