Sabbir02 opened a new issue, #45856:
URL: https://github.com/apache/airflow/issues/45856

   ### Description
   
   This issue adds OpenLineage support for the PostgresOperator to enable 
metadata tracking for PostgreSQL operations. The feature aims to enhance data 
pipeline observability by integrating OpenLineage capabilities with Apache 
Airflow's PostgreSQL provider.
   
   Current Behavior
   PostgresOperator currently lacks built-in OpenLineage support for tracking 
metadata.
   Users cannot define both the database and schema for PostgreSQL connections 
directly, limiting flexibility.
   Expected Behavior
   Add support for OpenLineage in the PostgresOperator to enable metadata 
tracking for queries and tasks.
   Allow users to specify both the database and schema explicitly when using 
PostgreSQL connections in Airflow.
   
   ### Use case/motivation
   
   This feature improves Apache Airflow by adding OpenLineage support to the 
PostgresOperator, making it easier to track and understand how data flows 
through your workflows. With this integration, users can capture detailed 
metadata about SQL queries and their inputs and outputs, enhancing visibility 
and compliance in data pipelines.
   
   It also simplifies connection configurations by letting users explicitly set 
the database and schema in PostgreSQL connections. This is especially useful in 
complex setups with multiple databases or schemas.
   
   By adopting OpenLineage, Airflow aligns with modern data engineering 
standards, helping teams monitor, debug, and scale their workflows with 
confidence. This feature makes managing and maintaining data pipelines more 
transparent and user-friendly.
   
   ### Related issues
   
   This feature builds on PR #31398, which lays the groundwork for integrating 
OpenLineage with Apache Airflow. To ensure everything works smoothly, thorough 
testing is needed to verify that lineage tracking and database/schema 
configurations are correctly implemented.
   
   
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to