Lars Volker created IMPALA-9032:
-----------------------------------
Summary: Impala returns 0 rows over hs2-http without waiting for
TODO timeout
Key: IMPALA-9032
URL: https://issues.apache.org/jira/browse/IMPALA-9032
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.4.0
Reporter: Lars Volker
This looks like a bug to me but I'm not entirely sure. I'm trying to run our
tests over hs2-http (IMPALA-8863) and after the change for IMPALA-7312 to
introduce a non-blocking mode for FetchResults() it looks like we sometimes
return an empty result way before {{fetch_rows_timeout_ms}} has elapsed. This
triggers a bug in Impyla
([#369|https://github.com/cloudera/impyla/issues/369]), but it also seems like
something we should investigate and fix in Impala.
{noformat}
I1007 22:10:10.697760 56550 impala-hs2-server.cc:821] FetchResults():
query_id=764d4313dbc64e20:2831560c00000000 fetch_size=1024I1007 22:10:10.697760
56550 impala-hs2-server.cc:821] FetchResults():
query_id=764d4313dbc64e20:2831560c00000000 fetch_size=1024I1007 22:10:10.697988
56527 scheduler.cc:468] 6d4cba4d2e8ccc42:66ce26a800000000] Exec at coord is
falseI1007 22:10:10.698014 54090 impala-hs2-server.cc:663]
GetOperationStatus(): query_id=0d43fd73ce4403fd:da25dde900000000I1007
22:10:10.698173 127 control-service.cc:142]
0646e91fd6a0a953:02949ff300000000] ExecQueryFInstances():
query_id=0646e91fd6a0a953:02949ff300000000 coord=b04a12d76e27:22000
#instances=1I1007 22:10:10.698356 56527 admission-controller.cc:1270]
6d4cba4d2e8ccc42:66ce26a800000000] Trying to admit
id=6d4cba4d2e8ccc42:66ce26a800000000 in pool_name=root.default
executor_group_name=default per_host_mem_estimate=52.02 MB
dedicated_coord_mem_estimate=110.02 MB max_requests=-1 (configured statically)
max_queued=200 (configured statically) max_mem=29.30 GB (configured
statically)I1007 22:10:10.698386 56527 admission-controller.cc:1282]
6d4cba4d2e8ccc42:66ce26a800000000] Stats: agg_num_running=9, agg_num_queued=0,
agg_mem_reserved=8.34 GB, local_host(local_mem_admitted=9.09 GB,
num_admitted_running=9, num_queued=0, backend_mem_reserved=6.70 GB)I1007
22:10:10.698415 56527 admission-controller.cc:871]
6d4cba4d2e8ccc42:66ce26a800000000] Admitting query
id=6d4cba4d2e8ccc42:66ce26a800000000I1007 22:10:10.698479 56527
impala-server.cc:1713] 6d4cba4d2e8ccc42:66ce26a800000000] Registering query
locationsI1007 22:10:10.698529 56527 coordinator.cc:97]
6d4cba4d2e8ccc42:66ce26a800000000] Exec()
query_id=6d4cba4d2e8ccc42:66ce26a800000000 stmt=select count(*) from alltypes
where month=1I1007 22:10:10.698992 56527 coordinator.cc:361]
6d4cba4d2e8ccc42:66ce26a800000000] starting execution on 3 backends for
query_id=6d4cba4d2e8ccc42:66ce26a800000000I1007 22:10:10.699383 56523
coordinator.cc:375] 0646e91fd6a0a953:02949ff300000000] started execution on 1
backends for query_id=0646e91fd6a0a953:02949ff300000000I1007 22:10:10.699409
56534 scheduler.cc:468] e1495f928c2cd4f6:eeda82aa00000000] Exec at coord is
falseI1007 22:10:10.700017 127 control-service.cc:142]
6d4cba4d2e8ccc42:66ce26a800000000] ExecQueryFInstances():
query_id=6d4cba4d2e8ccc42:66ce26a800000000 coord=b04a12d76e27:22000
#instances=1I1007 22:10:10.700147 56534 scheduler.cc:468]
e1495f928c2cd4f6:eeda82aa00000000] Exec at coord is falseI1007 22:10:10.700234
325 TAcceptQueueServer.cpp:340] New connection to server
hiveserver2-http-frontend from client <Host: 172.18.0.1 Port: 51580>I1007
22:10:10.700286 329 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client <Host: 172.18.0.1
Port: 51580>I1007 22:10:10.700314 329 TAcceptQueueServer.cpp:245]
TAcceptQueueServer: hiveserver2-http-frontend finished connection setup for
client <Host: 172.18.0.1 Port: 51580>I1007 22:10:10.700371 56550
impala-hs2-server.cc:844] FetchResults():
query_id=764d4313dbc64e20:2831560c00000000 #results=1 has_more=trueI1007
22:10:10.700508 56551 impala-server.cc:1969] Connection
8249c7defcb10124:1bc65ed9ea562aab from client 172.18.0.1:51576 to server
hiveserver2-http-frontend closed. The connection had 1 associated
session(s).I1007 22:10:10.700688 53748 impala-beeswax-server.cc:260] close():
query_id=e9473ff80c5d4afe:733cefe000000000I1007 22:10:10.700711 53748
impala-server.cc:1129] UnregisterQuery():
query_id=e9473ff80c5d4afe:733cefe000000000I1007 22:10:10.700721 53748
impala-server.cc:1234] Cancel():
query_id=e9473ff80c5d4afe:733cefe000000000I1007 22:10:10.700742 53748
coordinator.cc:716] CancelBackends()
query_id=e9473ff80c5d4afe:733cefe000000000, tried to cancel 0 backendsI1007
22:10:10.700690 56534 scheduler.cc:468] e1495f928c2cd4f6:eeda82aa00000000] Exec
at coord is falseI1007 22:10:10.701387 56534 admission-controller.cc:1270]
e1495f928c2cd4f6:eeda82aa00000000] Trying to admit
id=e1495f928c2cd4f6:eeda82aa00000000 in pool_name=root.default
executor_group_name=default per_host_mem_estimate=231.95 MB
dedicated_coord_mem_estimate=109.92 MB max_requests=-1 (configured statically)
max_queued=200 (configured statically) max_mem=29.30 GB (configured
statically)I1007 22:10:10.701417 56534 admission-controller.cc:1282]
e1495f928c2cd4f6:eeda82aa00000000] Stats: agg_num_running=10, agg_num_queued=0,
agg_mem_reserved=8.34 GB, local_host(local_mem_admitted=9.84 GB,
num_admitted_running=10, num_queued=0, backend_mem_reserved=6.70 GB)I1007
22:10:10.701447 56534 admission-controller.cc:871]
e1495f928c2cd4f6:eeda82aa00000000] Admitting query
id=e1495f928c2cd4f6:eeda82aa00000000I1007 22:10:10.701503 56534
impala-server.cc:1713] e1495f928c2cd4f6:eeda82aa00000000] Registering query
locationsI1007 22:10:10.701534 56534 coordinator.cc:97]
e1495f928c2cd4f6:eeda82aa00000000] Exec()
query_id=e1495f928c2cd4f6:eeda82aa00000000 stmt=with t1 as (select int_col x,
bigint_col y from functional.alltypes limit 2), t2 as (select int_col
x, bigint_col y from functional.alltypestiny limit 2), t3 as (select
int_col x, bigint_col y from functional.alltypessmall limit 2) select *
from t1, t2, t3 where t1.x = t2.x and t2.x = t3.xI1007 22:10:10.702641 56534
coordinator.cc:361] e1495f928c2cd4f6:eeda82aa00000000] starting execution on 3
backends for query_id=e1495f928c2cd4f6:eeda82aa00000000I1007 22:10:10.703843
56556 query-state.cc:643] 6d4cba4d2e8ccc42:66ce26a800000000] Executing
instance. instance_id=6d4cba4d2e8ccc42:66ce26a800000000 fragment_idx=0
per_fragment_instance_idx=0 coord_state_idx=2 #in-flight=18I1007
22:10:10.704418 56550 impala-server.cc:1969] Connection
41442926b263eea1:b922dd2b3a2513bd from client 172.18.0.1:51572 to server
hiveserver2-http-frontend closed. The connection had 1 associated
session(s).I1007 22:10:10.704545 325 TAcceptQueueServer.cpp:340] New
connection to server hiveserver2-http-frontend from client <Host: 172.18.0.1
Port: 51584>I1007 22:10:10.704847 56553 query-state.cc:643]
0646e91fd6a0a953:02949ff300000000] Executing instance.
instance_id=0646e91fd6a0a953:02949ff300000000 fragment_idx=0
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=19I1007
22:10:10.705631 327 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client <Host: 172.18.0.1
Port: 51584>I1007 22:10:10.705659 327 TAcceptQueueServer.cpp:245]
TAcceptQueueServer: hiveserver2-http-frontend finished connection setup for
client <Host: 172.18.0.1 Port: 51584>I1007 22:10:10.705934 56555
impala-server.cc:1969] Connection 0b41c31be5df55d6:c2d9fcea092c26b2 from client
172.18.0.1:51580 to server hiveserver2-http-frontend closed. The connection had
1 associated session(s).I1007 22:10:10.706935 56557 impala-hs2-server.cc:821]
FetchResults(): query_id=764d4313dbc64e20:2831560c00000000 fetch_size=1024I1007
22:10:10.707320 325 TAcceptQueueServer.cpp:340] New connection to server
hiveserver2-http-frontend from client <Host: 172.18.0.1 Port: 51588>I1007
22:10:10.707453 56535 query-state.cc:652] 764d4313dbc64e20:2831560c00000001]
Instance completed. instance_id=764d4313dbc64e20:2831560c00000001 #in-flight=18
status=OKI1007 22:10:10.713026 329 TAcceptQueueServer.cpp:227]
TAcceptQueueServer: hiveserver2-http-frontend started connection setup for
client <Host: 172.18.0.1 Port: 51588>I1007 22:10:10.713054 329
TAcceptQueueServer.cpp:245] TAcceptQueueServer: hiveserver2-http-frontend
finished connection setup for client <Host: 172.18.0.1 Port: 51588>I1007
22:10:10.714377 127 control-service.cc:142]
e1495f928c2cd4f6:eeda82aa00000000] ExecQueryFInstances():
query_id=e1495f928c2cd4f6:eeda82aa00000000 coord=b04a12d76e27:22000
#instances=6I1007 22:10:10.715759 56527 coordinator.cc:375]
6d4cba4d2e8ccc42:66ce26a800000000] started execution on 3 backends for
query_id=6d4cba4d2e8ccc42:66ce26a800000000I1007 22:10:10.717445 56560
query-state.cc:643] e1495f928c2cd4f6:eeda82aa00000000] Executing instance.
instance_id=e1495f928c2cd4f6:eeda82aa00000000 fragment_idx=0
per_fragment_instance_idx=0 coord_state_idx=2 #in-flight=19I1007
22:10:10.718142 127 coordinator.cc:745] Backend completed:
host=1161aa24dbe8:22000 remaining=3
query_id=99454b2820335de0:0a0a549e00000000I1007 22:10:10.718163 127
coordinator-backend-state.cc:338] query_id=99454b2820335de0:0a0a549e00000000:
first in-progress backend: 38e8ee0f58f6:22000I1007 22:10:10.720214 56557
impala-hs2-server.cc:844] FetchResults():
query_id=764d4313dbc64e20:2831560c00000000 #results=0 has_more=true
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
