danny0405 opened a new issue, #20665:
URL: https://github.com/apache/hudi/issues/20665

   ### Bug Description
   
   **What happened:**
   
   `SchemaEvolvingRowDataProjection.createMapProjection()` and 
`TypeConverters.createMapConverter()` call `keyArray()` and `valueArray()` 
inside the entry loop. Flink's `GenericMapData` materializes a complete array 
on each accessor call, so converting an n-entry map performs O(n²) array 
copying and creates 2n temporary arrays.
   
   For a 100-entry map, each accessor is called 100 times. This adds avoidable 
CPU and allocation overhead during Flink schema evolution.
   
   **What you expected:**
   
   Fetch both arrays once per map conversion and reuse them in the loop, 
reducing array copying to O(n) while preserving key/value associations, 
converted values, and null handling.
   
   **Steps to reproduce:**
   
   1. Wrap a `GenericMapData` containing 100 integer-keyed entries in a Mockito 
spy.
   2. Convert its values from INT to BIGINT through 
`TypeConverters.getInstance(fromMapType, toMapType).convert(map)`, or project a 
row containing that map through `SchemaEvolvingRowDataProjection`.
   3. Verify accessor calls: `verify(map).keyArray()` and 
`verify(map).valueArray()` fail on the original implementation because each 
accessor is called 100 times.
   
   ### Environment
   
   **Hudi version:** 1.3.0-SNAPSHOT; affected source inspected at master commit 
7888bf909694f3377c2ea7dc860b78e16509623f.
   **Query engine:** Flink; focused regression tests use Flink 2.2.1 and Java 
17.
   **Relevant configs:** None required; reproducible directly through the 
conversion APIs.
   
   ### Logs and Stack Trace
   
   No runtime exception is required to trigger this performance issue. 
Regression coverage checks accessor counts and conversion results for both 
paths with 0, 1, and 100 entries, including null values.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to