cshuo opened a new issue, #19521:
URL: https://github.com/apache/hudi/issues/19521

   ### Bug Description
   
   **What happened:**
   
   The Flink `RowData` Parquet writer derives the `FIXED_LEN_BYTE_ARRAY` width 
for decimal fields only from decimal precision. When the authoritative 
Hudi/Avro schema declares a fixed-backed decimal whose size is larger than the 
minimum required for its precision, both the generated Parquet schema and the 
encoded value use the smaller derived width.
   
   For example, an Avro decimal with precision 20 backed by `fixed(10)` is 
written as a 9-byte Parquet decimal. The compact-decimal path also cannot 
safely encode a declared fixed width greater than eight bytes with its existing 
shift-based implementation.
   
   **What you expected:**
   
   When a decimal is backed by an Avro fixed type, the Flink Parquet schema and 
value writer should honor the declared fixed size. Decimal schemas not backed 
by fixed should continue using the minimum width derived from precision.
   
   **Steps to reproduce:**
   
   1. Define a Hudi/Avro schema containing a decimal with precision 20 and a 
declared fixed size of 10 bytes.
   2. Construct the Flink `RowData` Parquet writer with that schema.
   3. Inspect the generated Parquet schema and the bytes passed to the record 
consumer.
   4. Observe that both use 9 bytes instead of the declared 10 bytes.
   
   ### Environment
   
   **Hudi version:** current `master`
   **Query engine:** Flink
   **Relevant configs:** Flink `RowData` Parquet writer with a fixed-backed 
decimal schema
   
   ### Logs and Stack Trace
   
   No exception is required to reproduce the schema and encoded-width mismatch.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to