claudevdm commented on PR #34398:
URL: https://github.com/apache/beam/pull/34398#issuecomment-2754612696
Thank you for your contribution to the Cloud SQL Enrichment Handler. I'd
like to share some thoughts about potential improvements:
- Query Flexibility: We should consider expanding the interface to allow
users to:
1. Select specific columns rather than always returning all columns
2. Apply multiple filtering conditions (e.g., filtering on both
customer_id AND region)
3. Support more complex query conditions beyond simple equality checks
- SQLAlchemy Integration: Given that we're supporting three different
database types (MySQL, PostgreSQL, and SQL Server), leveraging SQLAlchemy would:
1. Provide database-agnostic query building
2. Handle dialect-specific syntax differences automatically
3. Offer better protection against SQL injection vulnerabilities
- Batch Processing Support: For performance optimization, we should
implement batched queries similar to the BigQuery handler
https://github.com/apache/beam/blob/6426f72a59c00ea1561250af9a3594340a5ee373/sdks/python/apache_beam/transforms/enrichment_handlers/bigquery.py#L180
Could @liferoad and @svetakvsundhar share their thoughts on the user
interface design and SQLAlchemy integration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
