o-nikolas commented on PR #73301: URL: https://github.com/apache/airflow/pull/73301#issuecomment-5767654625
Hey @ashb, thanks for staying engaged on this one! > @o-nikolas A discussion thread is just that, a discussion, and not a detailed design discussion. Doubly so as in that discuss list you said nothing about "AwsDuckDBOperator". > > The main thing I don't like about this is, as a user, why do I need to go "okay, I've developed this against local files and it works, now I need to swap it to a different operator to deploy to prod." It really feels like it should be "I just change the connection". > > Why can't it work this way? It's a much nicer user experience. I think both are a minor changes to a dag and I'm not sure that one could argue one is much worse than the other realistically. Also it's a false premise, any operator based off the vendor neutral operator works with all the local development. If you knew you wanted to work with S3, you could begin your local testing with the `AwsDuckDBOperator` and then switch it to S3 with a connection change there. In all unbiased honesty I think using an operator to signal which behaviour you want is more obvious and self documenting than a connection change which feels a bit subtle and you have to have read the docs to know that's what happens. But any who, I assume you will still disagree. So I pushed the changes to update the `DuckDBExecuteQueryOperator` to pull the hook given the connection type provided. Please have a look if you can, I'm hoping to get the changes in for this provider release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
