aokolnychyi commented on a change in pull request #4047:
URL: https://github.com/apache/iceberg/pull/4047#discussion_r801000535
##########
File path:
spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/TestSparkDistributionAndOrderingUtil.java
##########
@@ -1567,6 +1567,370 @@ public void
testRangePositionDeltaUpdatePartitionedTable() {
table, UPDATE, expectedDistribution,
SPEC_ID_PARTITION_FILE_POSITION_ORDERING);
}
+ //
==================================================================================
+ // Distribution and ordering for merge-on-read MERGE operations with
position deletes
+ //
==================================================================================
+ //
+ // UNPARTITIONED UNORDERED
+ // -------------------------------------------------------------------------
+ // merge mode is NOT SET -> rely on write distribution and ordering as a
basis
+ // merge mode is NONE -> unspecified distribution + LOCALLY ORDER BY
_spec_id, _partition, _file, _pos
+ // merge mode is HASH -> unspecified distribution + LOCALLY ORDER BY
_spec_id, _partition, _file, _pos
+ // merge mode is RANGE -> unspecified distribution + LOCALLY ORDER BY
_spec_id, _partition, _file, _pos
+ //
+ // UNPARTITIONED ORDERED BY id, data
+ // -------------------------------------------------------------------------
+ // merge mode is NOT SET -> rely on write distribution and ordering as a
basis
+ // merge mode is NONE -> unspecified distribution +
+ // LOCALLY ORDER BY _spec_id, _partition, _file, _pos,
id, data
+ // merge mode is HASH -> unspecified distribution +
+ // LOCALLY ORDER BY _spec_id, _partition, _file, _pos,
id, data
+ // merge mode is RANGE -> RANGE DISTRIBUTE BY _spec_id, _partition, _file,
id, data +
+ // LOCALLY ORDER BY _spec_id, _partition, _file,
_pos, id, data
+ //
+ // PARTITIONED BY date, days(ts) UNORDERED
+ // -------------------------------------------------------------------------
+ // merge mode is NOT SET -> rely on write distribution and ordering as a
basis
+ // merge mode is NONE -> unspecified distribution +
+ // LOCALLY ORDERED BY _spec_id, _partition, _file,
_pos, date, days(ts)
+ // merge mode is HASH -> CLUSTER BY _spec_id, _partition, date, days(ts) +
+ // LOCALLY ORDER BY _spec_id, _partition, _file, _pos,
date, days(ts)
+ // merge mode is RANGE -> RANGE DISTRIBUTE BY _spec_id, _partition, _file,
date, days(ts) +
+ // LOCALLY ORDER BY _spec_id, _partition, _file,
_pos, date, days(ts)
+ //
+ // PARTITIONED BY date ORDERED BY id
+ // -------------------------------------------------------------------------
+ // merge mode is NOT SET -> rely on write distribution and ordering as a
basis
+ // merge mode is NONE -> unspecified distribution +
+ // LOCALLY ORDERED BY _spec_id, _partition, _file,
_pos, date, id
+ // merge mode is HASH -> CLUSTER BY _spec_id, _partition, date +
+ // LOCALLY ORDER BY _spec_id, _partition, _file, _pos,
date, id
+ // merge mode is RANGE -> RANGE DISTRIBUTE BY _spec_id, _partition, _file,
date, id
Review comment:
Note that we don't include `_pos` in the distribution. Just in the local
ordering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
