[ https://issues.apache.org/jira/browse/HIVE-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ratandeep Ratti updated HIVE-18410: ----------------------------------- Attachment: HIVE-18410_2.patch > [Performance][Avro] Reading flat Avro tables is very expensive in Hive > ---------------------------------------------------------------------- > > Key: HIVE-18410 > URL: https://issues.apache.org/jira/browse/HIVE-18410 > Project: Hive > Issue Type: Improvement > Affects Versions: 3.0.0, 2.3.2 > Reporter: Ratandeep Ratti > Assignee: Ratandeep Ratti > Priority: Major > Fix For: 3.0.0, 2.3.2 > > Attachments: HIVE-18410.patch, HIVE-18410_1.patch, > HIVE-18410_2.patch, profiling_with_patch.nps, profiling_with_patch.png, > profiling_without_patch.nps, profiling_without_patch.png > > > There's a performance penalty when reading flat [no nested fields] Avro > tables. When reading the same flat dataset in Pig, it takes half the time. > On profiling, a lot of time is spent in > {{AvroDeserializer.deserializeSingleItemNullableUnion()}}. The bulk of the > time is spent in GenericData.get().resolveUnion(), which calls > GenericData.getSchemaName(Object datum), which does a lot of instanceof > checks. This could be simplified with performance benefits. A approach is > described in this patch which almost halves the runtime. -- This message was sent by Atlassian JIRA (v7.6.3#76005)