FrankChen021 opened a new pull request, #5263: URL: https://github.com/apache/calcite/pull/5263
## Jira Link [CALCITE-7782](https://issues.apache.org/jira/browse/CALCITE-7782) ## Changes Proposed Validation of large ARRAY and MAP constructors may adjust many operands to a common component type. `SqlValidatorUtil.adjustTypeForMultisetConstructor` currently calls `SqlBasicCall.setOperand` separately for every required cast. Each `setOperand` invocation creates a new immutable copy of the complete operand list. When most of `n` operands require casts, this results in approximately O(n²) copying. This change preserves the explicit casts introduced by CALCITE-5948 while installing the adjusted operands efficiently: - add `SqlCall.setOperandList`, with a default implementation that delegates to the existing per-operand setter; - override it in `SqlBasicCall` to create one immutable operand-list copy; - collect all required casts in `adjustTypeForMultisetConstructor` and install the resulting list once; - verify that bulk replacement preserves the existing operand-list snapshot behavior. Other `SqlCall` implementations retain their existing per-operand behavior through the default implementation. ### Reproducer Apache Druid exposes the problem after rewriting a large string `IN` predicate to a scalar function containing an ARRAY constructor: ```sql EXPLAIN PLAN FOR SELECT COUNT(*) FROM foo WHERE long1 = 8 OR LOWER(string1) IN ('1', '2', ..., '1000000') ``` Mixed-width string literals are assigned types such as `CHAR(1)`, `CHAR(2)`, and `CHAR(6)`, then cast to a common `VARCHAR` component type. Consequently, nearly every operand enters the replacement path. ### Benchmark The 81 GB value below is cumulative allocation during one benchmark operation, not peak heap usage. The benchmark used Apache Druid commit `d480d66e7fe52ef5daa46eb1af9f33f954be9b7c`, JDK 25.0.4.1, JMH 1.37, and `InPlanningBenchmark.queryStringFunctionInSql`. Parameters: - `inSubQueryThreshold = 2147483647` - `rowsPerSegment = 500000` - one warmup iteration and three measurement iterations - GC profiler enabled | String literals | Calcite build | Average time | Allocation | |---:|---|---:|---:| | 100,000 | Official 1.42.0 | 5,305.626 ms/op | 81,622,365,189 B/op | | 100,000 | Patched 1.42.0 | 1,042.127 ms/op | 1,620,149,861 B/op | | 1,000,000 | Official 1.42.0 | No completed operation after 20 minutes | Not available | | 1,000,000 | Patched 1.42.0 | 9,996.863 ms/op | 16,187,406,264 B/op | At 100,000 literals, the change is approximately 5.1 times faster and reduces cumulative allocation by approximately 98%. The official 1.42.0 million-element operation was stopped after 20 minutes without producing a JMH score. The patched operation completes in approximately 10 seconds. ### Testing - `SqlCallOperandsTest` - ARRAY constructor tests in `CalciteSqlOperatorTest` - MAP constructor tests in `CalciteSqlOperatorTest` - ARRAY constructor tests in `CoreSqlOperatorTest` - MAP constructor tests in `CoreSqlOperatorTest` - `:core:checkstyleMain` - `:core:checkstyleTest` - GitHub Actions suite on the fork PR All focused tests, style checks, and fork CI passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
