xiangfu0 opened a new pull request, #19571: URL: https://github.com/apache/pinot/pull/19571
## What Builds on #19473 to reuse matching table-schema FieldSpecs during final immutable-segment loading, following [the review suggestion](https://github.com/apache/pinot/pull/19473#pullrequestreview-5217340163). Independent segment loads deserialize fresh schemas. A table-scoped cache now reuses the latest equivalent schema after timestamp-index normalization, with recursive complex-child comparisons and synchronized publication. Fresh ZooKeeper reads continue, and changed definitions replace the cached instance without editing previous snapshots. After preprocessing and before index readers are constructed, matching non-complex specs replace the parsed references in both column metadata and the segment's own schema. Different defaults/types, table-only attributes, complex fields, and unsupported metadata keep their parsed specs. #19473's weak interner remains the fallback for unmatched definitions and standalone metadata loads. ## Review scope This draft targets #19473's branch so its diff shows only this additional cache and loading behavior. It is separate for review of whether to keep it as a follow-up or fold it into #19473. The existing downstream stack is not retargeted. The cache lifecycle and preprocessing boundary are additional concerns beyond #19473's parse-time sharing. This PR does not remove the interner or claim a measured heap, startup-time, or query-latency improvement. ## Validation - **62 focused tests passed** (0 failed/skipped) on JDK 25: `TableSchemaCacheTest`, `SchemaFieldSpecReuseTest`, `SegmentMetadataFieldSpecReuseTest`, `ColumnMetadataImplTest`, `SegmentMetadataImplTest`, `IndexLoadingConfigTest`, and `DefaultColumnHandlerTest`. - Coverage includes fresh table-manager fetches, concurrent equal schemas, nested schema changes, timestamp normalization, real segment reads/null markers, autogenerated-default rebuilds, legacy TIME, JSON/order stability, materialized-child name collisions, and interner fallbacks. - Spotless, Checkstyle and license checks passed on `pinot-core`, `pinot-segment-local`, and `pinot-segment-spi`. - Standard reactor compilation passed. The additional strict `-Xlint:all` check stops at the previously observed missing `org.jetbrains.annotations.NotNull` dependency in unchanged `ZstandardDecompressor.java:51`; no dependency change is included. These are correctness and identity checks. Production cache churn, external index/segment-directory plugins, and performance are not validated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
