jplauri edited a comment on pull request #19596: URL: https://github.com/apache/airflow/pull/19596#issuecomment-970480520
> @jplauri regarding your question about merge vs insert, i think merge is the right call. That list of issues is pretty ancient now. Merge was implemented in 2008 and by now, I think most of the meaningful kinks have been ironed out. I would bet that the rowcount approach would be slow since it limits you to row-by-row (where as with merge you could, e.g. load into a temp table and merge into target with deduping logic too) > > But I don't think you need to tackle that in this PR. One thing at a time. Can do this limited case first. Thanks! And yes, I would be fine with this limited solution too. I mean, GenericTransfer is meant to be used for small datasets anyway as it works in-mem, so I can imagine there are cases where doing say truncate & insert all is just fine instead of replace. Perhaps there are other use cases where replace is useful, but anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
