fengruotj commented on PR #11286:
URL: https://github.com/apache/paimon/pull/11286#issuecomment-7033350707

   > Using the snapshot instant for both system tables addresses the 
JVM-dependent results in 
[#11219](https://github.com/apache/paimon/issues/11219). All 13 Core snapshot 
and tag tests passed on a clean integration with master. I found two reader 
compatibility issues with the type change.
   > 
   > **[P1] Preserve the epoch value with Spark legacy timestamp mapping.** I 
committed a row, created its tag, and joined `$snapshots` to `$tags` on Spark 
3.5.9 with an Asia/Shanghai JVM and a UTC SQL session. With 
`spark.paimon.legacy-timestamp-mapping.enabled=false`, both 
`unix_millis(commit_time)` values matched the stored `1795146612121`. Setting 
it to `true` returned `1795117812121` for both, eight hours early. On master, 
the legacy-mapping query returned the correct instant. 
`AbstractSparkInternalRow.getTimestampMicros` calls the converter without the 
Paimon type. The legacy branch calls `Timestamp.toSQLTimestamp()`, applying the 
JVM zone to the new epoch-backed value. Spark 3.2 and 3.3 always use that 
branch. Please pass through epoch micros for `TIMESTAMP_LTZ` and exercise both 
mapping settings in the SQL regression.
   > 
   > **[P2] Coordinate the StarRocks reader before upgrading its Paimon 
dependency.** StarRocks reads `$snapshots.commit_time` for unpartitioned table 
freshness and converts it using 
`commitTime.toLocalDateTime().atZone(ZoneId.systemDefault()).toInstant()`. I 
ran that conversion with an Asia/Shanghai JVM and both Paimon timestamp 
representations. A commit at `2026-11-17T06:31:24.761Z` produced the same value 
before this change and `2026-11-16T22:31:24.761Z` afterward. The earlier 
timestamp can affect materialized-view refresh and staleness decisions. Please 
document the type transition and track a StarRocks reader update that uses 
epoch milliseconds for LTZ and retains the existing conversion for older Paimon 
schemas. [StarRocks 
consumer](https://github.com/StarRocks/starrocks/blob/ee983a26ea12e033846f53369da3ea941389a020/fe/fe-core/src/main/java/com/starrocks/connector/paimon/PaimonMetadata.java#L1127-L1163)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to