KnightChess commented on a change in pull request #3459:
URL: https://github.com/apache/iceberg/pull/3459#discussion_r750339012
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java
##########
@@ -298,6 +311,103 @@ public void deleteWhere(Filter[] filters) {
}
}
+ @Override
+ public StructType partitionSchema() {
+ if (lazyPartitionSchema == null) {
+ Table table = table();
+ PartitionsTable partitionsTable =
+ (PartitionsTable)
MetadataTableUtils.createMetadataTableInstance(table,
MetadataTableType.PARTITIONS);
+ Types.NestedField partition =
partitionsTable.schema().findField("partition");
+ if (partition != null) {
+ this.lazyPartitionSchema = (StructType)
SparkSchemaUtil.convert(partition.type());
Review comment:
> I think you are ok here since you are getting the type from the
partitions metadata table. I would just make sure we have a test for it.
ok, and I have a question. for the `partitionSchema` method.
```java
public StructType partitionSchema()
```
If I have a table, partitioned by dt, ht. then I drop ht. So I think the
method should only return partition dt. Is it right?
And in https://github.com/apache/iceberg/pull/3459#discussion_r749869700,
the data from metadata partitions table will contain ht. So I think
`listPartitionIdentifiers` doesn't conflict with `partitionSchema`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
