danzhewuju opened a new issue, #11041:
URL: https://github.com/apache/paimon/issues/11041

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   
   ### Paimon version
   
   Paimon 2.2-SNAPSHOT
   
   ### Compute Engine
   
   Flink 2.2 
   
   ### Minimal reproduce step
   
   A non-null value can become NULL when a decimal cast overflows. For example, 
a nullable INT column containing `1000` is read as `DECIMAL(3, 0)` after a type 
change. Paimon accepts this type conversion, and the decimal cast returns NULL 
on precision overflow.
   The following JUnit test isolates the casted-row and copying behavior. Add 
it as 
`paimon-common/src/test/java/org/apache/paimon/casting/CastedRowDecimalOverflowTest.java`:
   ```java
   package org.apache.paimon.casting;
   import org.apache.paimon.data.GenericRow;
   import org.apache.paimon.data.InternalRow;
   import org.apache.paimon.data.serializer.InternalRowSerializer;
   import org.apache.paimon.types.DataType;
   import org.apache.paimon.types.DataTypes;
   import org.junit.jupiter.api.Test;
   class CastedRowDecimalOverflowTest {
       @Test
       void testDecimalOverflow() {
           DataType sourceType = DataTypes.INT();
           DataType targetType = DataTypes.DECIMAL(3, 0);
           CastedRow row = CastedRow.from(
                   new CastFieldGetter[] {
                       new CastFieldGetter(
                               InternalRow.createFieldGetter(sourceType, 0),
                               CastExecutors.resolve(sourceType, targetType),
                               "amount")
                   }).replaceRow(GenericRow.of(1000));
           System.out.println("decimal=" + row.getDecimal(0, 3, 0));
           System.out.println("isNull=" + row.isNullAt(0));
           new InternalRowSerializer(targetType).copy(row);
       }
   }
   ```
   Run:
   ```bash
   mvn -pl paimon-common -DwildcardSuites=none \
     -Dtest=CastedRowDecimalOverflowTest test
   ```
   The unpatched implementation prints:
   ```text
   decimal=null
   isNull=false
   ```
   Copying then throws `NullPointerException` in `DecimalSerializer.copy`, 
called from `InternalRowSerializer.copyRowData`.
   
   
   ### What doesn't meet your expectations?
   
   `CastedRow.isNullAt(0)` should return `true` when the converted value is 
NULL. Copying the row should preserve that NULL instead of throwing.
   Currently, `isNullAt` reads only the source row's null flag, while 
`getDecimal` returns the converted value. `InternalRowSerializer.copyRowData` 
trusts the incorrect non-null flag and calls `DecimalSerializer.copy(null)`.
   
   ### Anything else?
   
   The normal file reader applies `CastedRow` when a cast mapping is needed. 
The copy failure above has been reproduced with the real cast executors and 
serializer; no end-to-end Spark/Flink SQL failure is claimed.
   A local fix makes the null check account for the converted value, consistent 
with `CastedArray`. Five regression tests cover integer overflow, decimal 
rounding overflow, noncompact decimal overflow, copying, and row reuse. The 
same test command produced 5 failures before the fix, zero after it, 5 after 
reverting only the fix, and zero after restoring it; all 107 selected tests 
pass with the fix.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to