[
https://issues.apache.org/jira/browse/IMPALA-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17435387#comment-17435387
]
ASF subversion and git services commented on IMPALA-10967:
----------------------------------------------------------
Commit 78ce235db6d5b720f3e3319ff571a2da054a2602 in impala's branch
refs/heads/master from Qifan Chen
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=78ce235 ]
IMPALA-10967 Load data should handle AWS NLB-type timeout
This patch addresses Impala client hang due to AWS network load balancer
timeout which is fixed at 350s. When some long data loading operations
are executing and the timeout happens, AWS silently drops the connection
and the Impala client enters the hang state.
The fix maintains the current TCLIService protocol between the client
and Impala server and utilizes a separate thread to run the data loading
and metadata refresh operation. Since this thread is waited for in a
wait thread which runs asynchronously, the execution of the entire
operation will not cause a wait on the Impala client. The Impala client
can check the status of the operation via repeated GetOperationStatus()
call.
External behavior change:
1. A new query option 'enable_async_load_data_execution', default to
true, is added. It can be set to false to turn off the patch.
Testing:
1. Added a new test in test_load.py to verify that the asynchronous
execution in BE keeps the session live for hs2, hs2-http and
beeswax three clients;
2. Ran core tests successfully.
Change-Id: I8c2437e9894510204303ec07710cad60102c8821
Reviewed-on: http://gerrit.cloudera.org:8080/17955
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Load data should handle AWS NLB-type timeout
> --------------------------------------------
>
> Key: IMPALA-10967
> URL: https://issues.apache.org/jira/browse/IMPALA-10967
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Qifan Chen
> Assignee: Qifan Chen
> Priority: Major
>
> Currently, since Impala handles the load data statement request in a single
> thread, the client can experience AWS NLB-type timeout (see IMPALA-10811) if
> the data loading takes more than 350s to complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
