PDGGK opened a new pull request, #19069:
URL: https://github.com/apache/iotdb/pull/19069

   ## Description
   
   In the table model, a predicate on the time column fails when the value is a 
constant TIMESTAMP expression instead of a plain number. On 2.0.11 and on 
current master:
   
   ```sql
   SELECT temperature FROM sensor WHERE time >= CAST(1704067201000 AS TIMESTAMP)
   -- 301: class ...sql.ast.GenericLiteral cannot be cast to class 
...sql.ast.LongLiteral
   ```
   
   `=`, `BETWEEN` and the literal-on-the-left form fail the same way and `IN` 
fails with 305, while `time >= 1704067201000` works, and so does the CAST form 
against a TIMESTAMP *field*.
   
   ### Cause
   
   The constant is folded by `LiteralEncoder` into `GenericLiteral("TIMESTAMP", 
"<long>")`. `GlobalTimePredicateExtractVisitor` only treats a `LongLiteral` as 
a global time filter, so the predicate stays in the scan, where 
`ConvertPredicateToFilterVisitor` hands it to 
`ConvertPredicateToTimeFilterVisitor`. Its `getLongValue` casts the value to 
`LongLiteral` (the server stack ends at 
`ConvertPredicateToTimeFilterVisitor.getLongValue`, line 320 on 2.0.11), and 
`visitInPredicate` checks and casts the same way. The TIMESTAMP-field path 
already accepts this literal (`TypeServices.getTimestampValue`).
   
   ### Change
   
   `ConvertPredicateToTimeFilterVisitor.getLongValue` and `visitInPredicate` 
also accept a `GenericLiteral` of type `TIMESTAMP`, reading its value as the 
long in the current precision, as `TypeServices.getTimestampValue` does. Other 
literal types behave as before.
   
   Not changed here: `GlobalTimePredicateExtractVisitor.checkIsTimeFilter` 
still only extracts `LongLiteral` values, so the CAST form is evaluated as a 
scan filter rather than used for time-partition pruning. The result is correct 
either way; I left that out to keep this change to the failure.
   
   #15662 (open) also edits `getLongValue`, for `DoubleLiteral`; whichever of 
the two lands second needs a small merge there.
   
   ### Verification
   
   - `TablePredicateConversionTest#testTimestampConstantAndTimeColumn` (new) 
converts `>=`, a literal on the left, `IN` and `BETWEEN` with TIMESTAMP 
literals on the time column. All 8 tests in the class pass. With the 
`getLongValue` part reverted the new test fails with the same 
`ClassCastException`; with only the `visitInPredicate` part reverted it fails 
at the `IN` case with `IllegalArgumentException`.
   - With the change, the datanode test classes under 
`queryengine/plan/relational` and `queryengine/plan/planner` that cover 
predicates and time filters (25 classes, 348 tests) pass, and so do 17 
table-model ITs with time predicates in `WHERE`, among them `IoTDBInTableIT`, 
`IoTDBCastFunctionTableIT` and `IoTDBFilterBetweenTableIT` (334 tests, 
`-PTableSimpleIT`). `spotless:check` and the `with-zh-locale` compile pass.
   - On standalone servers built from master 8a5e5ca5444 without and with this 
change, with rows at time 1000, 2000 and 3000:
   
   | query | master | with this change |
   |---|---|---|
   | `time >= CAST(2000 AS TIMESTAMP)` | 301 | 2000, 3000 |
   | `time = CAST(2000 AS TIMESTAMP)` | 301 | 2000 |
   | `time IN (CAST(1000 AS TIMESTAMP), CAST(3000 AS TIMESTAMP))` | 305 | 1000, 
3000 |
   | `time BETWEEN CAST(1500 AS TIMESTAMP) AND CAST(2500 AS TIMESTAMP)` | 301 | 
2000 |
   | `CAST(2000 AS TIMESTAMP) < time` | 301 | 3000 |
   | `time >= 2000` (control) | 2000, 3000 | 2000, 3000 |
   
   `NOT IN` and `NOT BETWEEN` with CAST constants already returned the right 
rows and still do.
   
   I found this through the table-model Flink connector on the 
`flink-iotdb-table-connector` branch of iotdb-extras, which rendered its time 
filters in the CAST form (changed there in apache/iotdb-extras#142); a user 
writing the CAST form by hand hits the same error.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added unit tests or modified existing tests to cover new code paths, 
ensuring the threshold for code coverage.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to