pingzh commented on PR #21301: URL: https://github.com/apache/airflow/pull/21301#issuecomment-1223116438
> @pingzh I think the reason why we kept PipeCallbackSink was backward compatibility - in non-standalone DagProcessor mode we don't need DB (as we can rely on pipes). Note that using DatabaseCallbackSink may have some impact - on performance (not sure how much) and database usage (additional requests). But I think we could discuss it. > > cc: @potiuk @ashb @uranusjr @kaxil @mhenc thanks for the response. I agree with the concerns. Does it still create the table when it uses non-standalone DagProcessor? If yes, I think it is still backward-compatible as callbacks are still invoked. Using pipe vs db is more like a implementation choice. My concern here is supporting both cases make the code more complex and testing harder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
