I agree with Jarek's suggestion to add another module. Personally I don't
like having too many on the airflow._ level, there are quite a few already.
Also, I would also go for providers. Mostly because we have hook*s* and
operator*s*. When there are a lot of {operators,hooks,sensors} it makes
sense to package these in a separate module. I could imagine that other
companies that provide managed services
<https://airflow.readthedocs.io/en/latest/operators-and-hooks-ref.html#service-integrations>
might
evolve to a separate package, but maybe we should come up with some
criteria.
Cheers, Fokko
Op vr 4 okt. 2019 om 19:00 schreef Jarek Potiuk <[email protected]>:
> Not that I want to open Pandora's box :). But I think the longer name - the
> worse. Provider is quite nice and short.
>
> I agree with plural 'providers' (as we have hooks and operators). So for
> consistency it should be indeed plural.
>
> J.
>
> pt., 4 paź 2019, 18:48 użytkownik Chris Palmer <[email protected]>
> napisał:
>
> > My question about Oracle/MySql wasn't a serious one, but I forget
> sometimes
> > that sarcasm doesn't come across well on email.
> >
> > I guess my objection is that I don't think that 'provider' adds anything
> of
> > value. I'm not convinced that there needs to be a level between 'airflow'
> > and 'google' but if going that route I would advocate for at least the
> > plural form 'providers' or the more descriptive 'cloud_providers'.
> >
> > Chris
> >
> > On Fri, Oct 4, 2019 at 11:57 AM Kamil Breguła <[email protected]
> >
> > wrote:
> >
> > > Hello,
> > >
> > > We think that only cloud providers should be separated from others,
> > > because these services are integrated with each other. Very often,
> > > when you use one cloud provider, you use many services of a given
> > > provider. Using a single provider solution provides a uniform way of
> > > authorization, etc. A large number of problems and mechanisms are
> > > common to one provider. You can see the amount of integration from
> > > Microsoft, Azure, Google on the reference list.
> > > https://airflow.readthedocs.io/en/latest/operators-and-hooks-ref.html
> > > In this list, cloud reference providers have been placed in separate
> > > tables because they have a very large number of services. If Oracle
> > > will have many integrations, it is worth emphasizing this fact and
> > > moving these integrations to a separate place, so that it is easier to
> > > find them. and use. Keeping all possible files in one place makes it
> > > very difficult to use them.
> > >
> > > Best regards,
> > >
> > > On Fri, Oct 4, 2019 at 5:32 PM Chris Palmer <[email protected]>
> wrote:
> > > >
> > > > This seems unnecessary to me.
> > > >
> > > > Is everything going to be under some 'provider' or just certain sets
> of
> > > > operators, and if so what differentiates when something should be
> > under a
> > > > provider or not? For example, are the mysql operators going to go
> under
> > > > 'provider/oracle/'?
> > > >
> > > > Chris
> > > >
> > > > On Fri, Oct 4, 2019 at 9:21 AM Jarek Potiuk <
> [email protected]>
> > > > wrote:
> > > >
> > > > > Agree with Ash.
> > > > >
> > > > > After doing the gcp move and seeing the result we agreed that
> > > 'provider' is
> > > > > better as additional prefix.
> > > > >
> > > > > If no-one objects (Lazy Consensus
> > > > > <https://community.apache.org/committers/lazyConsensus.html>) till
> > > Monday
> > > > > 3.20 CEST, we will update AIP-21 and move the gcp operators to
> > > > > *provider/google/[gcp,gsuite]*.
> > > > >
> > > > > J.
> > > > >
> > >
> >
>
