ahmedabu98 opened a new pull request, #40998: URL: https://github.com/apache/beam/pull/40998
Iceberg reads build every Row with `Row.Builder.build()`. Its per-field checks are rather expensive, costing more than the rest of the conversion put together. This switches to `attachValues`, which is how `RowCoder` builds rows when it decodes. Two values that `build()` used to fix up are now converted directly: DATETIME to a joda `Instant`, and `binary` inside lists and maps to `byte[]`. | Local JMH, M4 Pro, JDK 21 | Before | After | |---|---|---| | Convert one flat record (12 columns) | 2,555 ns, 14 KB allocated | 130 ns, 136 B | | Convert one nested record | 3,866 ns, 19.5 KB | 481 ns, 1.2 KB | | Read 200k flat rows from local Parquet | 758 ms | 178 ms | | Read 200k nested rows from local Parquet | 1,055 ms | 360 ms | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
