In my opinion the source of the transfer operation is what matters - without 
knowing the source you don‘t need to know where it can be transferred to.

So I prefer to put those „cross transfer“ operators to its source. For example: 
GoogleApiToS3 -> gcp

Best Regards,
Felix

Sent from ProtonMail Mobile

On Sat, Sep 21, 2019 at 21:52, Jarek Potiuk <jarek.pot...@polidea.com> wrote:

> I have a question: Should we put all transfer operators between into
> separate "cross_transfer" package ?
>
> *Context:*
>
> We had one unresolved point when we decided about AIP-21 - where to put
> transfer operators between service providers. In the middle of implementing
> it, it turned out that we need to make some decisions as it has some
> undesirable side effects if we just move the transfer operators to core
> without any structure. Detailed discussion in this PR:
> https://github.com/apache/airflow/pull/6147
>
> We can solve it easily by choosing "cross_transfer" package for all
> transfer operators that are crossing "service provider" boundary.
>
> This way we will have "gcp" (or maybe even "alphabet" soon), "aws", "azure"
> etc. and "cross_transfer" for all the S3->GCP, AWS->S3 etc.
>
> What do you think? Anyone strongly against this? Or maybe we can follow
> lazy consensus rule for this? Or maybe someone can come up with a better
> name :) ?
>
> J.
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>

Reply via email to