Hi, I see the following comment in FlatMapGroupsInPandasExec.scala <https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala#L47> : "It's possible to further split one group into multiple record batches to reduce the memory footprint on the Java side, this is left as future work."
I checked the JIRA but could not find anything related to this. Is there a plan to support this scenario? Thanks, Terry