[
https://issues.apache.org/jira/browse/DRILL-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623876#comment-15623876
]
Parth Chandra commented on DRILL-4800:
--------------------------------------
Some perf numbers on a new setup (so these are not directly comparable with the
ones in the proposal doc)-
Other reader (target): 17,142 ms
DRILL 1.9.0 SNAPSHOT : 24,517 ms
#1 DRILL 1.9.0 with buffering: 18,287 ms
#2 DRILL 1.9.0 with Async Page Reader + buffering: 18,055 ms
#3 DRILL 1.9.0 with Async Page Reader + buffering + Parallel decoding: 16,281 ms
Under concurrent loads -
#1, #2 scale linearly and with 5 concurrent queries take ~42s
#3 also scales linearly, but degrades much faster, taking ~59s
Under #3, CPU starts becoming a bottleneck
For the PR, I'm proposing to go with #2 on by default, since it is still much
better than what we have.
Note that these numbers are from a scan heavy query on the TPCH lineitem table.
> Improve parquet reader performance
> ----------------------------------
>
> Key: DRILL-4800
> URL: https://issues.apache.org/jira/browse/DRILL-4800
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Parth Chandra
> Assignee: Parth Chandra
> Labels: doc-impacting
>
> Reported by a user in the field -
> We're generally getting read speeds of about 100-150 MB/s/node on PARQUET
> scan operator. This seems a little low given the number of drives on the node
> - 24. We're looking for options we can improve the performance of this
> operator as most of our queries are I/O bound.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
