mobuchowski commented on PR #71312: URL: https://github.com/apache/airflow/pull/71312#issuecomment-5226729729
In general I like this approach, but maybe this should be configurable more broadly: - not narrowed down to hook-level lineage, but operator lineage too - more options than just regex - going even forward with it, maybe it could be just generalizable "preprocessor" for this data? This could take a form of generalized filters (python function) that are being passed context and lineage and return lineage output. We could have ones that are build in, and one of those could be "filter hook level lineage using regex"; but we could pass more using Airflow plugins if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
