pvary commented on code in PR #18862:
URL: https://github.com/apache/iceberg/pull/18862#discussion_r4524576946
##########
core/src/main/java/org/apache/iceberg/PartitionData.java:
##########
@@ -37,6 +39,23 @@
public class PartitionData
implements IndexedRecord, StructLike, SpecificData.SchemaConstructable,
Serializable {
+ /**
+ * {@link #equals(Object)}/{@link #hashCode()} ignore {@link #schema}, so
independently created
+ * {@link PartitionData} instances for the same partition type (e.g. from
unrelated scans, or from
+ * Avro reflection when reading different manifests) are value-equal even
when each one initially
+ * carries its own, structurally-equivalent but distinct {@code Schema}
instance. Without
+ * canonicalizing here, {@link #INTERNER} would non-deterministically pick
whichever instance
+ * happened to intern first, so copies sharing a partition value could end
up with a schema object
+ * that didn't come from the caller's own template. Callers like {@code
PartitionsTable} rely on
+ * every partition from one scan sharing the exact same schema instance, so
the schema is cached
+ * per partition type instead and reused everywhere.
+ *
+ * <p>Declared before {@link #EMPTY} since its initializer constructs a
{@link PartitionData} that
+ * reads from this cache.
+ */
+ private static final ConcurrentMap<Types.StructType, Schema> SCHEMA_CACHE =
Review Comment:
Who will clean this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
