szehon-ho commented on code in PR #4637:
URL: https://github.com/apache/iceberg/pull/4637#discussion_r861387515
##########
spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/source/TestMetadataTablesWithPartitionEvolution.java:
##########
@@ -546,6 +546,156 @@ public void testPartitionsTableSwitchFields() throws
Exception {
PARTITIONS);
}
+ @Test
+ public void testPartitionTableFilterAddRemoveFields() throws ParseException {
+ // Create un-partitioned table
+ sql("CREATE TABLE %s (id bigint NOT NULL, category string, data string)
USING iceberg " +
+ "TBLPROPERTIES ('commit.manifest-merge.enabled' 'false')", tableName);
Review Comment:
Nope , removed them.
##########
core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java:
##########
@@ -550,6 +550,229 @@ public void testDeleteFilesTableSelection() throws
IOException {
Assert.assertEquals(expected, scan.schema().asStruct());
}
+ @Test
+ public void testPartitionSpecEvolutionAdditiveV1() {
Review Comment:
Difference was the way to make PartitionKey. Combined the test using
different formatVersion handling for this part and the asserts.
##########
core/src/main/java/org/apache/iceberg/PartitionsTable.java:
##########
@@ -149,14 +150,15 @@ static CloseableIterable<FileScanTask>
planFiles(StaticTableScan scan) {
Snapshot snapshot = table.snapshot(scan.snapshot().snapshotId());
boolean caseSensitive = scan.isCaseSensitive();
- // use an inclusive projection to remove the partition name prefix and
filter out any non-partition expressions
- Expression partitionFilter = Projections
- .inclusive(transformSpec(scan.schema(), table.spec()), caseSensitive)
- .project(scan.filter());
+ LoadingCache<Integer, ManifestEvaluator> evalCache =
Caffeine.newBuilder().build(specId -> {
Review Comment:
Discussed, at this point we keep all partition specs so this will very
slightly save a cycle if we have some spec without any manifests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
