hvanhovell commented on code in PR #40796:
URL: https://github.com/apache/spark/pull/40796#discussion_r1183946902
##########
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala:
##########
@@ -664,7 +665,53 @@ class SparkConnectPlanner(val session: SparkSession) {
input: proto.Relation,
groupingExprs: java.util.List[proto.Expression],
sortingExprs: java.util.List[proto.Expression]):
UntypedKeyValueGroupedDataset = {
- val logicalPlan = transformRelation(input)
+ apply(transformRelation(input), groupingExprs, sortingExprs)
+ }
+
+ private def apply(
+ logicalPlan: LogicalPlan,
+ groupingExprs: java.util.List[proto.Expression],
+ sortingExprs: java.util.List[proto.Expression]):
UntypedKeyValueGroupedDataset = {
+ if (groupingExprs.size() == 1) {
+ createFromGroupByKeyFunc(logicalPlan, groupingExprs, sortingExprs)
+ } else if (groupingExprs.size() > 1) {
Review Comment:
This feels a bit dirty. I understand that the handling the grouping function
seems a bit different, but I wonder if we need the special single function
result flattening behavior in the grand scheme of things, especially when you
consider we are going to end up with a tupled dataset anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
