rdblue commented on a change in pull request #2061:
URL: https://github.com/apache/iceberg/pull/2061#discussion_r559051327
##########
File path: api/src/test/java/org/apache/iceberg/TestPartitionSpecValidation.java
##########
@@ -123,6 +123,21 @@ public void
testMultipleDatePartitionsWithDifferentSourceColumns() {
PartitionSpec.builderFor(SCHEMA).hour("d").hour("another_d").build();
}
+ @Test
+ public void testMultipleIdentityPartitions() {
+
PartitionSpec.builderFor(SCHEMA).year("d").identity("id").identity("d").identity("s").build();
Review comment:
This check was introduced because users were accustomed to partitioning
by both date and hour, assuming that hour was in [0-23]. In Iceberg, `hours` is
the granularity, not a repeating value. We had people using
`day("ts").hour("ts")` often, which is redundant so we want to fail cases like
that because it signals the user does not understand what they are using.
When updating partitioning, we allow duplication because there is probably
existing data in the table stored by day. If the user had to drop `ts_day` to
add `ts_hour`, then the metadata tables would also be affected. The result is
that metadata queries to find `max(partition.ts_day)` would start to fail.
Also, there is no harm in populating `ts_day` for data partitioned by `ts_hour`
so we allow it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
