szehon-ho commented on a change in pull request #4047:
URL: https://github.com/apache/iceberg/pull/4047#discussion_r806412971
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/SparkDistributionAndOrderingUtil.java
##########
@@ -171,12 +194,16 @@ private static Distribution
buildPositionDeleteUpdateDistribution(DistributionMo
}
}
- public static SortOrder[] buildPositionDeltaOrdering(Table table, Command
command, Distribution distribution) {
+ public static SortOrder[] buildPositionDeltaOrdering(Table table, Command
command) {
// the spec requires position delete files to be sorted by file and pos
+ SortOrder[] deleteOrdering = {SPEC_ID_ORDER, PARTITION_ORDER,
FILE_PATH_ORDER, ROW_POSITION_ORDER};
+
if (command == DELETE || command == UPDATE) {
- return new SortOrder[]{SPEC_ID_ORDER, PARTITION_ORDER, FILE_PATH_ORDER,
ROW_POSITION_ORDER};
+ return deleteOrdering;
} else {
- throw new IllegalArgumentException("Only position deletes and updates
are currently supported");
+ // all metadata columns like _spec_id, _file, _pos will be null for new
data records
Review comment:
Optional: just noticed we throw IllegalArgumentException in other
places where we have an unknown command, so was just wondering would it be
clearer/more consistent to put this in a MERGE case block and throw exception
in unknown command case? Up to you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
