[ 
https://issues.apache.org/jira/browse/NIFI-13593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsihovszki Krisztina updated NIFI-13593:
----------------------------------------
    Description: 
A problem was observed when PutIceberg processor was used with a table that 
contained a column accepting decimal values and the datatype was *decimal(7,2).*

When the following value was present in the incoming record, 
   "cost": 351.00,
the JsonTreeReader trimmed the second zero and returned just "351.0".

That value was not accepted by PutIceberg processor, it reported an error:
{code:java}
java.lang.IllegalArgumentException: Cannot write value as decimal(7,2), wrong 
scale: 351.0{code}
 

Full trace:

{code:java}
2024-07-23 07:23:54,883 ERROR org.apache.nifi.processors.iceberg.PutIceberg: 
PutIceberg[id=ebfa9858-ce4c-3b0a-b9dc-b347b5b7031a] Exception occurred while 
writing iceberg records. Removing uncommitted data files
java.lang.IllegalArgumentException: Cannot write value as decimal(7,2), wrong 
scale: 351.0
    at 
org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:463)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$IntegerDecimalWriter.write(ParquetValueWriters.java:247)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$IntegerDecimalWriter.write(ParquetValueWriters.java:235)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$OptionWriter.write(ParquetValueWriters.java:356)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$StructWriter.write(ParquetValueWriters.java:589)
    at org.apache.iceberg.parquet.ParquetWriter.add(ParquetWriter.java:138)
    at org.apache.iceberg.io.DataWriter.write(DataWriter.java:71)
    at 
org.apache.iceberg.io.BaseTaskWriter$RollingFileWriter.write(BaseTaskWriter.java:362)
    at 
org.apache.iceberg.io.BaseTaskWriter$RollingFileWriter.write(BaseTaskWriter.java:345)
    at 
org.apache.iceberg.io.BaseTaskWriter$BaseRollingWriter.write(BaseTaskWriter.java:277)
    at 
org.apache.iceberg.io.UnpartitionedWriter.write(UnpartitionedWriter.java:42)
    at 
org.apache.nifi.processors.iceberg.PutIceberg.doOnTrigger(PutIceberg.java:305)
 {code}

 

  was:
A problem was observed when PutIceberg processor was used with a table that 
contained a column accepting decimal values and the datatype was *decimal(7,2).*

When the following value was present in the incoming record, 
   "cost": 351.00,
the JsonTreeReader trimmed the second zero and returned just "351.0".

That value was not accepted by PutIceberg processor, it reported an error:
{code:java}
java.lang.IllegalArgumentException: Cannot write value as decimal(7,2), wrong 
scale: 351.0{code}
 

Full trace:


2024-07-23 07:23:54,883 ERROR org.apache.nifi.processors.iceberg.PutIceberg: 
PutIceberg[id=ebfa9858-ce4c-3b0a-b9dc-b347b5b7031a] Exception occurred while 
writing iceberg records. Removing uncommitted data files
java.lang.IllegalArgumentException: Cannot write value as decimal(7,2), wrong 
scale: 351.0
    at 
org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:463)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$IntegerDecimalWriter.write(ParquetValueWriters.java:247)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$IntegerDecimalWriter.write(ParquetValueWriters.java:235)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$OptionWriter.write(ParquetValueWriters.java:356)
    at 
org.apache.iceberg.parquet.ParquetValueWriters$StructWriter.write(ParquetValueWriters.java:589)
    at org.apache.iceberg.parquet.ParquetWriter.add(ParquetWriter.java:138)
    at org.apache.iceberg.io.DataWriter.write(DataWriter.java:71)
    at 
org.apache.iceberg.io.BaseTaskWriter$RollingFileWriter.write(BaseTaskWriter.java:362)
    at 
org.apache.iceberg.io.BaseTaskWriter$RollingFileWriter.write(BaseTaskWriter.java:345)
    at 
org.apache.iceberg.io.BaseTaskWriter$BaseRollingWriter.write(BaseTaskWriter.java:277)
    at 
org.apache.iceberg.io.UnpartitionedWriter.write(UnpartitionedWriter.java:42)
    at 
org.apache.nifi.processors.iceberg.PutIceberg.doOnTrigger(PutIceberg.java:305)
 

 


> PutIceberg issue with decimal scale
> -----------------------------------
>
>                 Key: NIFI-13593
>                 URL: https://issues.apache.org/jira/browse/NIFI-13593
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: Zsihovszki Krisztina
>            Priority: Major
>
> A problem was observed when PutIceberg processor was used with a table that 
> contained a column accepting decimal values and the datatype was 
> *decimal(7,2).*
> When the following value was present in the incoming record, 
>    "cost": 351.00,
> the JsonTreeReader trimmed the second zero and returned just "351.0".
> That value was not accepted by PutIceberg processor, it reported an error:
> {code:java}
> java.lang.IllegalArgumentException: Cannot write value as decimal(7,2), wrong 
> scale: 351.0{code}
>  
> Full trace:
> {code:java}
> 2024-07-23 07:23:54,883 ERROR org.apache.nifi.processors.iceberg.PutIceberg: 
> PutIceberg[id=ebfa9858-ce4c-3b0a-b9dc-b347b5b7031a] Exception occurred while 
> writing iceberg records. Removing uncommitted data files
> java.lang.IllegalArgumentException: Cannot write value as decimal(7,2), wrong 
> scale: 351.0
>     at 
> org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:463)
>     at 
> org.apache.iceberg.parquet.ParquetValueWriters$IntegerDecimalWriter.write(ParquetValueWriters.java:247)
>     at 
> org.apache.iceberg.parquet.ParquetValueWriters$IntegerDecimalWriter.write(ParquetValueWriters.java:235)
>     at 
> org.apache.iceberg.parquet.ParquetValueWriters$OptionWriter.write(ParquetValueWriters.java:356)
>     at 
> org.apache.iceberg.parquet.ParquetValueWriters$StructWriter.write(ParquetValueWriters.java:589)
>     at org.apache.iceberg.parquet.ParquetWriter.add(ParquetWriter.java:138)
>     at org.apache.iceberg.io.DataWriter.write(DataWriter.java:71)
>     at 
> org.apache.iceberg.io.BaseTaskWriter$RollingFileWriter.write(BaseTaskWriter.java:362)
>     at 
> org.apache.iceberg.io.BaseTaskWriter$RollingFileWriter.write(BaseTaskWriter.java:345)
>     at 
> org.apache.iceberg.io.BaseTaskWriter$BaseRollingWriter.write(BaseTaskWriter.java:277)
>     at 
> org.apache.iceberg.io.UnpartitionedWriter.write(UnpartitionedWriter.java:42)
>     at 
> org.apache.nifi.processors.iceberg.PutIceberg.doOnTrigger(PutIceberg.java:305)
>  {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to