Kaileshwar16 opened a new issue, #853: URL: https://github.com/apache/wayang/issues/853
In `wayang-jdbc-template`, I found a couple of issues in `SqlToStreamOperator` when streaming query results from JDBC. - JDBC resources are not always closed In `SqlToStreamOperator.evaluate()`, a JDBC connection is created and used to read query results. However, `ResultSetIterator` only closes the `ResultSet` when iteration reaches the end. The underlying `Statement` and `Connection` are not closed, and if a downstream operator stops reading early, even the `ResultSet` may remain open. This could leave database resources hanging around longer than necessary. - Fetch size configuration is commented out In `ResultSetIterator`, `Statement.setFetchSize()` is currently commented out. For large queries, this can cause some JDBC drivers to load too much data into memory instead of fetching rows in batches. Is there any particular reason it is commented out? It would be useful to support a configurable fetch size, so users can tune how many rows are fetched at a time. Some drivers, such as PostgreSQL, may also require additional settings, like disabling auto-commit, for cursor-based fetching to work properly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
