jbewing commented on code in PR #14683:
URL: https://github.com/apache/iceberg/pull/14683#discussion_r2560768927
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkWriteRequirements.java:
##########
@@ -48,12 +62,19 @@ public SortOrder[] ordering() {
return ordering;
}
+ public org.apache.iceberg.SortOrder icebergOrdering() {
+ return icebergOrdering;
+ }
Review Comment:
I'm not in love with this approach, but I went through a couple of
iterations of this and plumbing the intended Iceberg SortOrder along through
with the Spark SortOrder ended up being the cleanest in my opinion (open to any
suggestions of course). I initially had an approach which would attempt to
re-guess the Iceberg SortOrder from a Spark SortOrder. It worked pretty well,
but was pretty naive in my opinion—the algorithm was to essentially reverse the
Iceberg -> Spark SortOrder transformation as best as possible and then see if
things matched up. It gets complicated when you start to consider Spark
Orderings for stuff like say a positionDeltaUpdateMergeOrdering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
