Copilot commented on code in PR #800:
URL: https://github.com/apache/iceberg-cpp/pull/800#discussion_r3627414694
##########
src/iceberg/avro/avro_data_util.cc:
##########
@@ -497,6 +506,29 @@ Status AppendFieldToBuilder(const ::avro::NodePtr&
avro_node,
} // namespace
+Status AppendDefaultToBuilder(const Literal& literal, ::arrow::ArrayBuilder*
builder) {
+ // The builder's own memory pool is not exposed, so the small scalar buffer
uses the
+ // default pool.
+ ICEBERG_ASSIGN_OR_RAISE(std::shared_ptr<::arrow::Scalar> scalar,
+ arrow::ToArrowScalar(literal,
::arrow::default_memory_pool()));
+
Review Comment:
`AppendDefaultToBuilder` constructs an Arrow scalar (and may cast it) for
every appended row. When a column is missing from the Avro file, this path runs
once per row per defaulted field and can become a hotspot (especially for
string/binary/fixed/uuid/decimal where `ToArrowScalar` allocates buffers and
`CastTo` may allocate).
Consider precomputing the (already-cast) scalar once per projected field
(e.g., stash it in `FieldProjection::attributes` or in the decoder/read
context) and reusing it for each row append, so the per-row work is just
`AppendScalar`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
