Csaba Ringhofer created IMPALA-12101:
----------------------------------------
Summary: Inconsistent speeds with result spooling
Key: IMPALA-12101
URL: https://issues.apache.org/jira/browse/IMPALA-12101
Project: IMPALA
Issue Type: Improvement
Components: Backend, Clients
Reporter: Csaba Ringhofer
Noticed a case when enabling result spooling makes query execution much slower:
impala-shell -B -q "set spool_query_results=1; select cast(l_shipdate as
timestamp) from tpch_parquet.lineitem;" > /dev/null
Fetched 6001215 row(s) in 23.81s
impala-shell -B -q "set spool_query_results=0; select cast(l_shipdate as
timestamp) from tpch_parquet.lineitem;" > /dev/null
Fetched 6001215 row(s) in 9.92s
Using beeswax leads to completely different results:
impala-shell --protocol=beeswax -B -q "set spool_query_results=0; select
cast(l_shipdate as timestamp) from tpch_parquet.lineitem;" > /dev/null
Fetched 6001215 row(s) in 11.87s
impala-shell --protocol=beeswax -B -q "set spool_query_results=1; select
cast(l_shipdate as timestamp) from tpch_parquet.lineitem;" > /dev/null
Fetched 6001215 row(s) in 10.32s
This anomaly seems to occur when both the client and the coordinator needs
significant time to process the returned rows.
Note that the slow result generation from timestamps (and dates) is a known
performance issue in the coordinator - most time is spent in converting
dates/timestamps to strings. On the other side I don't understand how enabling
result spooling can slow down a query.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
