[
https://issues.apache.org/jira/browse/IMPALA-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334601#comment-17334601
]
ASF subversion and git services commented on IMPALA-10682:
----------------------------------------------------------
Commit f672c315bc4d08d56cc7399b86767d30c9676287 in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f672c31 ]
IMPALA-10682: Add buffering to hs2-http client in impala-shell
This change reduces to following command from 8.5s to 1.5s on my
machine:
shell/impala_shell.py -B -q "select * from tpch_parquet.lineitem limit 100000;"
--protocol hs2-http > /dev/null
This nearly eliminates the speed difference between hs2 and hs2-http.
The root cause of the original slowness is the large number of
calls to socket.recv(). The query above used to call it 2809090 times,
now it is only 9007.
Testing:
- ran shell tests
Change-Id: If11f287be65b10bee2b0afffea118e3dc70fdbbd
Reviewed-on: http://gerrit.cloudera.org:8080/17346
Reviewed-by: Quanlong Huang <[email protected]>
Tested-by: Csaba Ringhofer <[email protected]>
> impala-shell is slow with hs2-http
> ----------------------------------
>
> Key: IMPALA-10682
> URL: https://issues.apache.org/jira/browse/IMPALA-10682
> Project: IMPALA
> Issue Type: Improvement
> Components: Clients
> Reporter: Csaba Ringhofer
> Priority: Major
>
> The following command takes 1.5 sec with hs2 and 8.5 sec with hs2-http
> ptotocol on my dev machine:
> {code}
> shell/impala_shell.py -B -q "select * from tpch_parquet.lineitem limit
> 100000;" --protocol hs2-http > /dev/null
> {code}
> Profiling has shown that the main culprit is calling socket.recv() 2809090
> times. The root cause is the lack of buffering in our current http client
> implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
