[
https://issues.apache.org/jira/browse/IMPALA-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-3746.
-----------------------------------
Resolution: Later
> Optimize ReadZInteger() for avro scanner perf improvements
> ----------------------------------------------------------
>
> Key: IMPALA-3746
> URL: https://issues.apache.org/jira/browse/IMPALA-3746
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.7.0
> Reporter: Tim Armstrong
> Priority: Minor
> Labels: perf, ramp-up
>
> ReadWriteUtil::ReadZInteger() is the bottleneck for scanning many avro files.
> We made some efforts to optimize it in a recent patch
> (http://gerrit.cloudera.org:8080/3072), but we could have made further
> improvements:
> E.g.
> * Unrolling the decode loop for each possible byte length, and making sure
> that memory accesses got converted to wider 64-bit reads.
> * Using SIMD instructions to quickly find the length then decode it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
