wombatu-kun opened a new pull request, #16606: URL: https://github.com/apache/iceberg/pull/16606
Closes #16605 ### Problem `SchemaUtils.inferIcebergType` inferred a decimal column type directly from a `BigDecimal`'s precision and scale. Iceberg requires `0 <= scale <= precision <= 38`, but a `BigDecimal` does not guarantee that: a value < 1 (e.g. `0.001`) has precision 1 and scale 3, and an exponential-form value (e.g. `1E+2`) has a negative scale. `DecimalType.of` only validates `precision <= 38`, so these malformed types were built silently and then failed downstream at write time. ### Solution Normalize precision and scale before building the type: rescale a negative scale to 0, and widen precision to at least the scale. Valid inputs are unchanged (e.g. `12.345` stays `decimal(5, 3)`); `0.001` becomes `decimal(3, 3)` and `1E+2` becomes `decimal(3, 0)`. ### Tests - `TestSchemaUtils.testInferIcebergTypeSmallDecimal` (unit): asserts the inferred type is valid for both the small-fraction and the negative-scale cases. - `TestSinkWriter.testEvolveAddsFractionalDecimalColumn` (end-to-end): with schema evolution enabled, writes a schemaless record with a fractional decimal for a new column. Before the change the write fails with `DataException` caused by `Invalid DECIMAL scale: 3 cannot be greater than precision: 1`; after the change the record is written and the column is `decimal(3, 3)`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
