Hello all, in Polidea we have come up with an idea for a generic transfer operator that would be able to transport data between two destinations of various types (file, database, storage, etc.) - please find the link with a short doc with POC <https://docs.google.com/document/d/1o7Ph7RRNqLWkTbe7xkWjb100eFaK1Apjv27LaqHgNkE/edit?usp=sharing> where we can discuss the design initially. Once we come to the initial conclusion I can create an AIP on cWiki - can I ask for permission to do so (my id is 'kamil.olszewski')? I believe that during the discussion we should definitely aim for this feature to be released only after Airflow 2.0 is out.
What do you think about this idea? Would you find such an operator helpful in your pipelines? Maybe you already use a similar solution or know packages that could be used to implement it? Best regards, -- Kamil Olszewski Polidea <https://www.polidea.com> | Software Engineer M: +48 503 361 783 E: [email protected] Unique Tech Check out our projects! <https://www.polidea.com/our-work>
