Kaileshwar16 commented on code in PR #866:
URL: https://github.com/apache/wayang/pull/866#discussion_r4538354838


##########
wayang-platforms/wayang-jdbc-template/src/main/java/org/apache/wayang/jdbc/operators/SqlToStreamOperator.java:
##########
@@ -97,30 +99,50 @@ public Tuple<Collection<ExecutionLineageNode>, 
Collection<ChannelInstance>> eval
         final StreamChannel.Instance output = (StreamChannel.Instance) 
outputs[0];
 
         JdbcPlatformTemplate producerPlatform = (JdbcPlatformTemplate) 
input.getChannel().getProducer().getPlatform();
+        final String sqlQuery = input.getSqlQuery();
         final Connection connection = producerPlatform
                 .createDatabaseDescriptor(executor.getConfiguration())
                 .createJdbcConnection();
 
-        Iterator<Record> resultSetIterator = new ResultSetIterator(connection, 
input.getSqlQuery());
-        Spliterator<Record> resultSetSpliterator = 
Spliterators.spliteratorUnknownSize(resultSetIterator, 0);
-        Stream<Record> resultSetStream = 
StreamSupport.stream(resultSetSpliterator, false);
+        ResultSetIterator resultSetIterator = new 
ResultSetIterator(connection, sqlQuery);
+        boolean success = false;
+        try {
+            Spliterator<Record> resultSetSpliterator = 
Spliterators.spliteratorUnknownSize(resultSetIterator, 0);
+            Stream<Record> resultSetStream = 
StreamSupport.stream(resultSetSpliterator, false)
+                    .onClose(resultSetIterator::close);
 
-        output.accept(resultSetStream);
+            // Terminal stream operations do not close streams. Keep a 
job-level fallback for
+            // short-circuiting consumers, without changing StreamChannel's 
ownership semantics.
+            CrossPlatformExecutor crossPlatformExecutor = 
executor.getCrossPlatformExecutor();

Review Comment:
   I added this so JDBC resources get cleaned up when the job ends, even if 
downstream processing stops early. Stream operations don’t automatically close 
the stream, and disposing the channel only clears its reference.
     
   I also tried registering cleanup with JavaExecutor alone, but live channels 
can prevent it from being disposed after a downstream failure. Both new 
regression cases failed with that approach. I will add tests and have the 
comment more precise. is it justifed to add it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to