JackieTien97 commented on code in PR #18589:
URL: https://github.com/apache/iotdb/pull/18589#discussion_r3963872089


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/fill/filter/FixedIntervalFillFilter.java:
##########
@@ -34,6 +36,7 @@ public FixedIntervalFillFilter(long timeInterval) {
   public boolean needFill(long time, long previousTime) {
     // the reason that we use Math.abs is that we may use order by time desc 
which will cause
     // previousTime is larger than time
-    return Math.abs(time - previousTime) <= timeInterval;
+    return timeDifferenceAsDouble(time, previousTime) <= timeInterval
+        && timeDifferenceAsDouble(previousTime, time) <= timeInterval;

Review Comment:
   **[P2] Keep the fill interval comparison exact**
   
   Using a double for this integer threshold comparison accepts gaps that 
exceed the configured interval once the distance is above `2^53`. For example, 
at nanosecond precision, set `timeInterval=15552000000000000L` (180 days), 
`previousTime=1700000000000000000L`, and `time=previousTime+timeInterval+1`. 
`needFill` now returns `true`, whereas the previous implementation returns 
`false`: the 180-day-plus-1-ns gap rounds down to the same double as the 
threshold. This fills a value outside the requested range. Please compare the 
integer distance exactly with overflow handling; the unsigned-distance 
comparison added in `DownSamplingTimeUtils` is one applicable approach.



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/UDTFEqualSizeBucketOutlierSample.java:
##########
@@ -354,14 +356,12 @@ public void outlierSampleLong(RowWindow rowWindow, 
PointCollector collector)
         currentValue = rowWindow.getRow(i).getLong(0);
         nextValue = rowWindow.getRow(i + 1).getLong(0);
 
-        x1 = currentTime - lastTime;
-        x2 = nextTime - currentTime;
-        y1 = currentValue - lastValue;
-        y2 = nextValue - currentValue;
+        x1 = timeDifferenceAsDouble(currentTime, lastTime);
+        x2 = timeDifferenceAsDouble(nextTime, currentTime);
+        y1 = (double) currentValue - (double) lastValue;
+        y2 = (double) nextValue - (double) currentValue;

Review Comment:
   **[P2] Preserve the integer difference before converting INT64 values to 
double**
   
   Casting each operand before subtraction loses small differences between 
large INT64 values. With timestamps `[0, 1, 2, 3, 4]`, values `B + [0, 1, 10, 
1, 0]` where `B = 1L << 60`, and `number=1, proportion=0.2`, both `cos` and 
`prenextdis` select timestamp `1` on this commit instead of the unique central 
spike at timestamp `2`. All five values round to the same double, so the value 
differences become zero. The previous implementations select timestamp `2` for 
both modes. Please use an overflow-safe integer difference before converting to 
double, as `timeDifferenceAsDouble` already does for timestamps, and apply the 
same correction to the INT64 `prenextdis` path.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/DateTimeUtils.java:
##########
@@ -591,32 +592,40 @@ public static long convertDurationStrToLong(
    */
   public static long convertDurationStrToLong(
       long currentTime, String duration, String timestampPrecision, boolean 
convertYearToMonth) {
-    long total = 0;
-    long temp = 0;
-    for (int i = 0; i < duration.length(); i++) {
-      char ch = duration.charAt(i);
-      if (Character.isDigit(ch)) {
-        temp *= 10;
-        temp += (ch - '0');
-      } else {
-        String unit = String.valueOf(duration.charAt(i));
-        // This is to identify units with two letters.
-        if (i + 1 < duration.length() && !Character.isDigit(duration.charAt(i 
+ 1))) {
-          i++;
-          unit += duration.charAt(i);
-        }
-        unit = unit.toLowerCase();
-        if (convertYearToMonth && unit.equals("y")) {
-          temp *= 12;
-          unit = "mo";
+    try {
+      long total = 0;
+      long temp = 0;
+      for (int i = 0; i < duration.length(); i++) {
+        char ch = duration.charAt(i);
+        if (Character.isDigit(ch)) {
+          temp = Math.addExact(Math.multiplyExact(temp, 10), 
Character.digit(ch, 10));
+        } else {
+          String unit = String.valueOf(duration.charAt(i));
+          // This is to identify units with two letters.
+          if (i + 1 < duration.length() && 
!Character.isDigit(duration.charAt(i + 1))) {
+            i++;
+            unit += duration.charAt(i);
+          }
+          unit = unit.toLowerCase();
+          if (convertYearToMonth && unit.equals("y")) {
+            temp = Math.multiplyExact(temp, 12);
+            unit = "mo";
+          }
+          total =
+              Math.addExact(
+                  total,
+                  DateTimeUtils.convertDurationStrToLong(
+                      currentTime == -1 ? -1 : Math.addExact(currentTime, 
total),

Review Comment:
   **[P2] Do not reject a valid subtraction because an unused addition 
overflows**
   
   This method is also called when parsing date subtraction, but every 
component of a duration now checks `currentTime + total`. At nanosecond 
precision with UTC, `select s1 from root.sg.d1 where time > 
2262-04-11T23:47:16.854775806 - 1s1ns` throws `Time duration is out of range.` 
when parsing the second component. The final timestamp is the valid value 
`9223372035854775805`, and the equivalent expression using `- 1000000001ns` 
succeeds. The previous implementation accepts both. For fixed-duration units 
this reference timestamp is not needed; please avoid the unconditional addition 
and let the caller check the final addition/subtraction, while handling any 
calendar-dependent reference calculation according to the actual operation.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/ai/InferenceOperator.java:
##########
@@ -374,4 +378,27 @@ public long ramBytesUsed() {
             ? 0
             : 
targetColumnNames.stream().mapToLong(RamUsageEstimator::sizeOf).sum());
   }
+
+  static long calculateGeneratedTimeInterval(long minTimestamp, long 
maxTimestamp, long totalRow) {
+    try {
+      BigInteger interval =
+          BigInteger.valueOf(maxTimestamp)
+              .subtract(BigInteger.valueOf(minTimestamp))
+              .divide(BigInteger.valueOf(totalRow));
+      if (interval.compareTo(BigInteger.valueOf(Long.MAX_VALUE)) > 0) {
+        throw new ArithmeticException();
+      }
+      return interval.longValue();
+    } catch (ArithmeticException e) {
+      throw new 
ModelInferenceProcessException(GENERATED_TIME_COLUMN_OUT_OF_RANGE_MESSAGE);
+    }
+  }
+
+  static long calculateGeneratedTime(long maxTimestamp, long interval, long 
currentRowIndex) {
+    try {
+      return Math.addExact(maxTimestamp, Math.multiplyExact(interval, 
currentRowIndex));

Review Comment:
   **[P2] Check the complete generated timestamp rather than the intermediate 
product**
   
   A negative `maxTimestamp` can bring an intermediate product larger than 
`Long.MAX_VALUE` back into the valid timestamp range. For two input timestamps 
`-6000000000000000000L` and `-4000000000000000000L`, the interval calculation 
yields `1000000000000000000L`. If the model returns 11 rows, index `10` should 
receive timestamp `6000000000000000000L`, and every generated timestamp is 
representable. The new `multiplyExact` instead throws before adding the 
negative maximum timestamp, causing `fillTimeColumn` to fail with `Generated 
time column is out of range.`; the previous implementation completes this case. 
Please evaluate the entire multiply-and-add expression with `BigInteger` and 
range-check its final result.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to