PDGGK opened a new pull request, #142: URL: https://github.com/apache/iotdb-extras/pull/142
Two timestamp problems I hit while running this branch (f453277) against IoTDB 2.0.11. **1. Time filters and time-keyed lookups fail.** Timestamp literals are rendered as `CAST(<n> AS TIMESTAMP)`. IoTDB 2.0.11 fails a predicate on the time column written that way: ``` SELECT time, temperature FROM sensor WHERE time >= CAST(1704067201000 AS TIMESTAMP) -> 301: class ...GenericLiteral cannot be cast to class ...LongLiteral ``` The same happens with `=`, `IN` and `BETWEEN` on `time`. The plain number (`time >= 1704067201000`) works, and so does comparing a TIMESTAMP field with a number, so `renderTimestampLiteral` now returns the number. Through Flink, `SELECT * FROM t WHERE time >= TIMESTAMP '2024-01-01 00:00:01'` and a lookup join keyed on `time` both failed the job with that 301 error before the change. **2. Times depend on the JVM time zone.** The sink converted `TimestampData` with `toTimestamp()` and the source with `TimestampData.fromTimestamp()`. Both go through `LocalDateTime` in the JVM default zone, while the literals above use the epoch value. `RowDataSinkDataConverterTest.testGetters` already fails on a non-UTC machine (`expected:<1000> but was:<-28799000>` under Asia/Shanghai). Both sides now convert through `Instant`. Writing `2024-01-01 00:00:00/01/02` through the connector and reading back: | JVM zone | stored epoch millis (before) | stored (after) | `time >= '…00:00:01'` rows (after) | |---|---|---|---| | UTC | 1704067200000… | 1704067200000… | 2 | | Asia/Shanghai | 1704038400000… (−8 h) | 1704067200000… | 2 | Flink's own result for the same filter over a VALUES table is the same 2 rows. In a lookup join where `time` is only a join condition (`lookup=[device_id=…]`), the Asia/Shanghai run before the change returned NULL for a row that exists, because the looked-up `time` came back shifted; after the change it returns the value, in both zones and with `iotdb.lookup.async` on and off. Tests: `RowDataSinkDataConverterTest` and the new `RowDataDeserializationSchemaTest` set the default zone to Asia/Shanghai and fail without the respective change; `IoTDBLookupReaderTest` now expects the plain number. All 15 tests in the base module pass on a machine in Australia/Sydney. Note: `spotless:check` on this branch already reports `IoTDBUtils`, `TimestampPrecision` and `IoTDBOptions`; this change adds nothing to that list, and I ran the builds with `-Dspotless.check.skip=true`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
