PDGGK opened a new pull request, #19069:
URL: https://github.com/apache/iotdb/pull/19069
## Description
In the table model, a predicate on the time column fails when the value is a
constant TIMESTAMP expression instead of a plain number. On 2.0.11 and on
current master:
```sql
SELECT temperature FROM sensor WHERE time >= CAST(1704067201000 AS TIMESTAMP)
-- 301: class ...sql.ast.GenericLiteral cannot be cast to class
...sql.ast.LongLiteral
```
`=`, `BETWEEN` and the literal-on-the-left form fail the same way and `IN`
fails with 305, while `time >= 1704067201000` works, and so does the CAST form
against a TIMESTAMP *field*.
### Cause
The constant is folded by `LiteralEncoder` into `GenericLiteral("TIMESTAMP",
"<long>")`. `GlobalTimePredicateExtractVisitor` only treats a `LongLiteral` as
a global time filter, so the predicate stays in the scan, where
`ConvertPredicateToFilterVisitor` hands it to
`ConvertPredicateToTimeFilterVisitor`. Its `getLongValue` casts the value to
`LongLiteral` (the server stack ends at
`ConvertPredicateToTimeFilterVisitor.getLongValue`, line 320 on 2.0.11), and
`visitInPredicate` checks and casts the same way. The TIMESTAMP-field path
already accepts this literal (`TypeServices.getTimestampValue`).
### Change
`ConvertPredicateToTimeFilterVisitor.getLongValue` and `visitInPredicate`
also accept a `GenericLiteral` of type `TIMESTAMP`, reading its value as the
long in the current precision, as `TypeServices.getTimestampValue` does. Other
literal types behave as before.
Not changed here: `GlobalTimePredicateExtractVisitor.checkIsTimeFilter`
still only extracts `LongLiteral` values, so the CAST form is evaluated as a
scan filter rather than used for time-partition pruning. The result is correct
either way; I left that out to keep this change to the failure.
#15662 (open) also edits `getLongValue`, for `DoubleLiteral`; whichever of
the two lands second needs a small merge there.
### Verification
- `TablePredicateConversionTest#testTimestampConstantAndTimeColumn` (new)
converts `>=`, a literal on the left, `IN` and `BETWEEN` with TIMESTAMP
literals on the time column. All 8 tests in the class pass. With the
`getLongValue` part reverted the new test fails with the same
`ClassCastException`; with only the `visitInPredicate` part reverted it fails
at the `IN` case with `IllegalArgumentException`.
- With the change, the datanode test classes under
`queryengine/plan/relational` and `queryengine/plan/planner` that cover
predicates and time filters (25 classes, 348 tests) pass, and so do 17
table-model ITs with time predicates in `WHERE`, among them `IoTDBInTableIT`,
`IoTDBCastFunctionTableIT` and `IoTDBFilterBetweenTableIT` (334 tests,
`-PTableSimpleIT`). `spotless:check` and the `with-zh-locale` compile pass.
- On standalone servers built from master 8a5e5ca5444 without and with this
change, with rows at time 1000, 2000 and 3000:
| query | master | with this change |
|---|---|---|
| `time >= CAST(2000 AS TIMESTAMP)` | 301 | 2000, 3000 |
| `time = CAST(2000 AS TIMESTAMP)` | 301 | 2000 |
| `time IN (CAST(1000 AS TIMESTAMP), CAST(3000 AS TIMESTAMP))` | 305 | 1000,
3000 |
| `time BETWEEN CAST(1500 AS TIMESTAMP) AND CAST(2500 AS TIMESTAMP)` | 301 |
2000 |
| `CAST(2000 AS TIMESTAMP) < time` | 301 | 3000 |
| `time >= 2000` (control) | 2000, 3000 | 2000, 3000 |
`NOT IN` and `NOT BETWEEN` with CAST constants already returned the right
rows and still do.
I found this through the table-model Flink connector on the
`flink-iotdb-table-connector` branch of iotdb-extras, which rendered its time
filters in the CAST form (changed there in apache/iotdb-extras#142); a user
writing the CAST form by hand hits the same error.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for code coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]