FrankChen021 opened a new pull request, #5282: URL: https://github.com/apache/calcite/pull/5282
## Jira Link [CALCITE-7805](https://issues.apache.org/jira/browse/CALCITE-7805) ## Changes Proposed `RelDataTypeFactoryImpl` already canonicalizes types through the global weak `DATATYPE_CACHE` interner. However, `Interner.intern` accepts a fully constructed `RelDataType`, so canonicalization prevents equivalent types from being retained but does not prevent temporary `BasicSqlType`, charset wrapper, and type-digest allocations. This change adds two bounded front-cache entries per `SqlTypeName`: 1. `createSqlType(typeName, precision)` retains the most recent canonical base type, keyed by normalized precision and default charset. 2. `createTypeWithCharsetAndCollation(source, charset, collation)` retains the most recent canonical decoration, keyed by source identity, charset equality, and collation identity. Both caches check existing entries before constructing a type. Misses follow the existing construction and canonicalization paths and then atomically publish the new bounded entry. The global weak canonical interner remains authoritative. For a string literal, the two entries form one pipeline: ```text createSqlType(CHAR, n) -> reuse canonical CHAR(n), default charset, IMPLICIT createTypeWithCharsetAndCollation(..., COERCIBLE) -> reuse canonical CHAR(n), default charset, COERCIBLE ``` The decoration cache was also tested independently. Without the base-type cache, each literal alternates the same `CHAR` slot between the raw-to-`IMPLICIT` and canonical-to-`COERCIBLE` transformations. It therefore misses frequently and increased allocation by 3.78% at 100,000 literals and 3.68% at 1,000,000. The base-type cache removes the repeated raw-to-`IMPLICIT` path and stabilizes the source identity, allowing the decoration cache to hit. For this reason, the two cache changes are submitted together. The caches remain strictly bounded by the number of `SqlTypeName` values. They depend on locality: consecutive requests for the same type, precision, charset, and collation hit; a different request for the same `SqlTypeName` replaces the previous entry. ## Isolated Benchmark Druid's `InPlanningBenchmark.queryStringInSqlPlanOnly` compared this PR's exact merge-base (`38413ece6`) with the combined PR head (`8a37f37ae`). The original Druid executor was used in both variants, so the only production differences were the two cache changes in this PR. | String literals | Allocation before | Allocation after | Allocation saved | Change | Time before | Time after | |---:|---:|---:|---:|---:|---:|---:| | 100,000 | 1,470.31 MiB/op | 797.81 MiB/op | 672.50 MiB/op | -45.74% | 743.99 ms/op | 585.09 ms/op | | 1,000,000 | 14.478 GiB/op | 8.007 GiB/op | 6.471 GiB/op | -44.70% | 7.690 s/op | 6.204 s/op | The 99.9% timing confidence intervals do not overlap in either case. The measured mean planning time improved by 21.36% at 100,000 literals and 19.33% at 1,000,000 literals. Allocation is total allocated bytes per planning operation, not retained or peak heap. The benchmark generates ordered numeric strings (`'1'`, `'2'`, ...). Consecutive ranges contain many literals with the same character length, making this a high-locality workload. These measurements describe that workload and should not be generalized to arbitrary string `IN` lists. Fixed-length strings retain high locality regardless of value order; frequently alternating lengths may have a lower hit rate. Benchmark parameters: `inSubQueryThreshold=2147483647`, `rowsPerSegment=500000`; 2 JMH forks, each with 2 one-second warmup iterations and 5 one-second measured iterations; GC profiler; Temurin 25.0.4.1. Matching Calcite core and linq4j artifacts were placed ahead of the Druid benchmark JAR. ## Validation - `./gradlew :core:test --tests org.apache.calcite.sql.type.SqlTypeFactoryTest :core:autostyleJavaCheck :core:checkstyleMain :core:checkstyleTest` (28 completed, 0 failed) - Isolated Druid JMH benchmark at 100,000 and 1,000,000 string literals -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
