JackieTien97 commented on code in PR #19267:
URL: https://github.com/apache/iotdb/pull/19267#discussion_r4581307365


##########
iotdb-core/calc-commons/src/main/java/org/apache/iotdb/calc/transformation/dag/column/unary/scalar/DateBinFunctionColumnTransformer.java:
##########
@@ -90,19 +90,19 @@ private static LocalDateTime convertToLocalDateTime(long 
timestamp, ZoneId zoneI
   }
 
   private static long convertToTimestamp(LocalDateTime dateTime, ZoneId 
zoneId) {
-    // Converting LocalDateTime to Seconds since epoch
-    long epochMilliSecond = dateTime.atZone(zoneId).toInstant().toEpochMilli();
-    // Get the nanoseconds section
-    long nanoAdjustment = dateTime.getNano();
+    // getEpochSecond() gives the whole seconds and getNano() the nanos inside 
the second, so the
+    // milliseconds inside the second are counted exactly once
+    Instant instant = dateTime.atZone(zoneId).toInstant();
+    long epochSecond = instant.getEpochSecond();
+    long nanoAdjustment = instant.getNano();
 
     switch 
(CommonDescriptor.getInstance().getConfig().getTimestampPrecision()) {
       case "ms":
-        return epochMilliSecond + nanoAdjustment / NANOSECONDS_IN_MILLISECOND;
+        return TimeUnit.SECONDS.toMillis(epochSecond) + nanoAdjustment / 
NANOSECONDS_IN_MILLISECOND;
       case "us":
-        return TimeUnit.MILLISECONDS.toMicros(epochMilliSecond)
-            + nanoAdjustment / NANOSECONDS_IN_MICROSECOND;
+        return TimeUnit.SECONDS.toMicros(epochSecond) + nanoAdjustment / 
NANOSECONDS_IN_MICROSECOND;
       case "ns":
-        return TimeUnit.MILLISECONDS.toNanos(epochMilliSecond) + 
nanoAdjustment;
+        return TimeUnit.SECONDS.toNanos(epochSecond) + nanoAdjustment;

Review Comment:
   Edge case in the rewritten lines: 
`TimeUnit.SECONDS.toMillis/toMicros/toNanos` saturates at `Long.MAX_VALUE`, but 
the `+ nanoAdjustment / ...` after it is an unchecked add. So when a bin 
boundary lies past the end of the `long` range and the origin has a sub-second 
part, the result wraps to a large negative value.
   
   A probe test on this branch (UTC, `1mo`) shows:
   - `ms`, origin `500`: `dateBinStartEnd(Long.MAX_VALUE - 1000)` returns 
`[9223372035446400500, -9223372036854775309]`, and `nextDateBin(1, UTC, 
9223372035446400500)` returns `-9223372036854775309`. Before this PR, 
`toEpochMilli()` threw `ArithmeticException: long overflow` for the same input.
   - `ns`, origin `1500000`: `dateBinStartEnd(Long.MAX_VALUE - 10)` (a time in 
2262-04, which `ns` precision accepts by default) returns an end of 
`-9223372036853275809`. This one already happened before the PR.
   - With a whole-second origin the value only saturates to `Long.MAX_VALUE`.
   
   The gap-fill operators look sensitive to this. From reading the code (not 
run): when `writeCurrentRow` calls `nextTime()` and `currentTime` wraps, 
`hasRemainingGapInPreviousGroup()` (`currentTime < endTime`) stays true, so 
`fillGaps` starts emitting rows from near `Long.MIN_VALUE` up to `endTime`.
   
   `saturatedAdd` is already statically imported here, and it would keep this 
path consistent with the saturating behavior from #17893. With the change 
below, the probe cases return `Long.MAX_VALUE` as the bin end, and 
`DateBinFunctionTest` still passes:
   
   ```suggestion
         case "ms":
           return saturatedAdd(
               TimeUnit.SECONDS.toMillis(epochSecond), nanoAdjustment / 
NANOSECONDS_IN_MILLISECOND);
         case "us":
           return saturatedAdd(
               TimeUnit.SECONDS.toMicros(epochSecond), nanoAdjustment / 
NANOSECONDS_IN_MICROSECOND);
         case "ns":
           return saturatedAdd(TimeUnit.SECONDS.toNanos(epochSecond), 
nanoAdjustment);
   ```
   
   (If you'd rather keep the old throw-on-overflow behavior for `ms`, 
`instant.toEpochMilli()` does that.)



##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/DateBinFunctionTest.java:
##########
@@ -126,4 +133,53 @@ public void testSourceBeforeUnixEpoch() throws 
ParseException {
             zoneId);
     assertEquals(getTimestamp("1969-12-31 00:00:00.000"), result);
   }
+
+  @Test
+  public void testMonthIntervalWithSubSecondOrigin() throws ParseException {

Review Comment:
   Could you add a pre-1970 case as well? That is where the new code relies on 
`getEpochSecond()` rounding down (with `getNano()` always non-negative), and 
where the old code was furthest off: `dateBin(-500, -500, 1, 0, zoneId)` 
returned `0` under `ms`, and `dateBin(-1500, -1500, 1, 0, zoneId)` returned 
`996500` under `us`. Both are correct on this branch, for example:
   
   ```java
   assertEquals(-500L, dateBin(-500L, -500L, 1, 0, zoneId));
   assertEquals(-2678400500L, dateBin(-2678399500L, -500L, 1, 0, zoneId));
   
   setTimestampPrecision("us");
   assertEquals(-1500L, dateBin(-1500L, -1500L, 1, 0, zoneId));
   ```
   
   An IT covering `date_bin`, `date_bin_gapfill` and `GROUP BY date_bin` with a 
sub-second origin would be a nice extra, but I don't think it should block this 
PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to