cloud-fan commented on code in PR #57375:
URL: https://github.com/apache/spark/pull/57375#discussion_r3630150771
##########
sql/core/src/test/scala/org/apache/spark/sql/ApproxTopKSuite.scala:
##########
@@ -575,6 +578,20 @@ class ApproxTopKSuite extends SharedSparkSession {
"CAST('13:00:00.123' AS TIME(3))"))
)
+ test("SPARK-58069: serialize an empty approx_top_k_combine buffer") {
Review Comment:
Addressed in 41eecbac20e. The round-trip test now covers both a concrete
maxItemsTracked value and VOID_MAX_ITEMS_TRACKED, and asserts that nullCount is
restored. I also added an end-to-end SQL test that repartitions 10 input
sketches across 20 partitions before approx_top_k_combine, guaranteeing empty
shuffle partitions while exercising the unspecified-size production path.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxTopKAggregates.scala:
##########
Review Comment:
Thanks for identifying the related path. This PR remains scoped to
serializing placeholder buffers from empty intermediate partitions when other
partitions provide real input. The entirely-empty final-input case, including
the eval fallback and resolving VOID_MAX_ITEMS_TRACKED, is explicitly deferred
to #57289 to avoid duplicating that fix. Commit 41eecbac20e adds an end-to-end
mixed empty/non-empty shuffle test for this PRs scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
