voonhous commented on code in PR #19483:
URL: https://github.com/apache/hudi/pull/19483#discussion_r3705571449
##########
hudi-trino/src/main/java/io/trino/plugin/hudi/util/HudiAvroSerializer.java:
##########
@@ -524,10 +548,17 @@ private static void writeMap(MapBlockBuilder output,
MapType mapType, Map<?, ?>
static class AvroDecimalConverter
{
private static final Conversions.DecimalConversion
AVRO_DECIMAL_CONVERSION = new Conversions.DecimalConversion();
+ // convert() runs once per decimal cell on the record read path, and
building a Schema costs
+ // orders of magnitude more than the conversion itself. The
(precision, scale) space is tiny
+ // and fixed per column, so cache the schemas globally.
+ private static final Map<Integer, Schema> DECIMAL_SCHEMAS = new
ConcurrentHashMap<>();
Review Comment:
You're right, and it's worse than that -- the cache wasn't just unnecessary,
the whole converter was. Deleted it in #19495 rather than re-keying it.
`DecimalConversion.fromBytes` reads only the scale (never precision, and it
doesn't reference its `schema` argument at all), and `encodeShortScaledValue`
then calls `setScale` to that same scale, which `BigDecimal` short-circuits to
`return this`. The scale cancels end to end, so the pair really is `new
BigInteger(fixed.bytes()).longValueExact()`. That's also faster than the cache:
no map lookup, no `ByteBuffer`, no intermediate `BigDecimal`.
Left a comment at the call site explaining why, so it doesn't get "restored"
later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
