Raghu Angadi created SPARK-44433: ------------------------------------ Summary: Implement termination of Python process for foreachBatch & streaming listener Key: SPARK-44433 URL: https://issues.apache.org/jira/browse/SPARK-44433 Project: Spark Issue Type: Task Components: Connect, Structured Streaming Affects Versions: 3.4.1 Reporter: Raghu Angadi Fix For: 3.5.0
In the first implementation of Python support for foreachBatch, the python process termination is not handled correctly. See the long TODO in [https://github.com/apache/spark/blob/master/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/StreamingForeachBatchHelper.scala] about an outline of the feature to terminate the runners by registering StreamingQueryListners. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org