hudi-agent commented on code in PR #19483:
URL: https://github.com/apache/hudi/pull/19483#discussion_r3703173793
##########
hudi-trino/src/main/java/io/trino/plugin/hudi/util/HudiAvroSerializer.java:
##########
@@ -524,10 +548,16 @@ private static void writeMap(MapBlockBuilder output,
MapType mapType, Map<?, ?>
static class AvroDecimalConverter
{
private static final Conversions.DecimalConversion
AVRO_DECIMAL_CONVERSION = new Conversions.DecimalConversion();
+ // convert() runs once per decimal cell on the record read path, and
building a Schema costs
+ // orders of magnitude more than the conversion itself. The
(precision, scale) space is tiny
+ // and fixed per column, so cache the schemas globally.
+ private static final Map<Long, Schema> DECIMAL_SCHEMAS = new
ConcurrentHashMap<>();
BigDecimal convert(int precision, int scale, byte[] bytes)
{
- Schema schema = new
Schema.Parser().parse(format("{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":%d,\"scale\":%d}",
precision, scale));
+ Schema schema = DECIMAL_SCHEMAS.computeIfAbsent(
+ (((long) precision) << 32) | (scale & 0xFFFFFFFFL),
Review Comment:
🤖 nit: the bit-packing here is non-obvious — a reader has to reason about
sign extension and whether `& 0xFFFFFFFFL` is necessary. Since precision is
always 1–38 and scale is always 0–38, a much simpler `(long) precision * 100 +
scale` conveys the same uniqueness with zero bit-trick overhead. Alternatively,
a short comment like `// packs two non-negative ints: high 32 bits = precision,
low 32 bits = scale` would remove the mental speedbump.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
