[
https://issues.apache.org/jira/browse/IMPALA-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161445#comment-17161445
]
Sahil Takiar commented on IMPALA-9953:
--------------------------------------
Okay, I can reproduce this pretty easily now using some of the queries from
{{tests/hs2/test_fetch_timeout.py}}:
{code:java}
stakiar @ stakiar-desktop -bash ~/Impala 2020-07-20 11:19:54 master
[18] → ./bin/impala-shell.sh
[localhost:21000] default> set spool_query_results=true;
SPOOL_QUERY_RESULTS set to true
[localhost:21000] default> set batch_size=1;
BATCH_SIZE set to 1
[localhost:21000] default> set fetch_rows_timeout_ms=1;
FETCH_ROWS_TIMEOUT_MS set to 1
[localhost:21000] default> set num_nodes=1;
NUM_NODES set to 1
[localhost:21000] default> select bool_col, avg(id) from functional.alltypes
group by bool_col having avg(id) != sleep(5000);
Query: select bool_col, avg(id) from functional.alltypes group by bool_col
having avg(id) != sleep(5000)
Query submitted at: 2020-07-20 11:20:13 (Coordinator:
http://stakiar-desktop:25000)
Query progress can be monitored at:
http://stakiar-desktop:25000/query_plan?query_id=dd4d059983dc9049:de5bef7d00000000
Fetched 0 row(s) in 0.25s
[localhost:21000] default> quit;
Goodbye stakiar
stakiar @ stakiar-desktop -bash ~/Impala 2020-07-20 11:20:25 master
[19] → ./bin/impala-shell.sh
[localhost:21000] default> select bool_col, avg(id) from functional.alltypes
group by bool_col having avg(id) != sleep(5000);
Query: select bool_col, avg(id) from functional.alltypes group by bool_col
having avg(id) != sleep(5000)
Query submitted at: 2020-07-20 11:20:28 (Coordinator:
http://stakiar-desktop:25000)
Query progress can be monitored at:
http://stakiar-desktop:25000/query_plan?query_id=094023aa62d35937:c617705300000000
+----------+---------+
| bool_col | avg(id) |
+----------+---------+
| true | 3649 |
| false | 3650 |
+----------+---------+
Fetched 2 row(s) in 5.22s{code}
> Shell does not return all rows if a fetch times out in FINISHED state
> ---------------------------------------------------------------------
>
> Key: IMPALA-9953
> URL: https://issues.apache.org/jira/browse/IMPALA-9953
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
> Reporter: Tim Armstrong
> Assignee: Sahil Takiar
> Priority: Blocker
> Labels: correctness
>
> I noticed that if a fetch times out, impala-shell will stop returning rows
> and close the query. It looks like this happens if the query transitions to
> FINISHED state, then the fetch times out
> I ran into this on an experimental branch where a sort deadlocked. I haven't
> been able to repro on master yet but I thought I should report it.
> The bug is here:
> {noformat}
> diff --git a/shell/impala_shell.py b/shell/impala_shell.py
> index e0d802626..323aee6c9 100755
> --- a/shell/impala_shell.py
> +++ b/shell/impala_shell.py
> @@ -1182,8 +1182,7 @@ class ImpalaShell(cmd.Cmd, object):
>
> for rows in rows_fetched:
> # IMPALA-4418: Break out of the loop to prevent printing an
> unnecessary empty line.
> - if len(rows) == 0:
> - break
> + if len(rows) == 0: continue
> self.output_stream.write(rows)
> num_rows += len(rows)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
