[
https://issues.apache.org/jira/browse/IMPALA-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-7115.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.1.0
IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value
The value is chosen to allow only queries that have a reasonable chance
of succeeding, albeit with poor performance because of the high
number of threads.
Testing:
Added a test to make sure that the default value rejects a large query.
Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384
Reviewed-on: http://gerrit.cloudera.org:8080/10628
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Set a default THREAD_RESERVATION_LIMIT value
> --------------------------------------------
>
> Key: IMPALA-7115
> URL: https://issues.apache.org/jira/browse/IMPALA-7115
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: resource-management
> Fix For: Impala 3.1.0
>
>
> As a follow on to IMPALA-6035, we should set a default value that actually
> will help protect again insanely complex queries.
> Motivating discussion is here:
> https://gerrit.cloudera.org/#/c/10365/9/common/thrift/ImpalaInternalService.thrift
> {quote}
> Tim Armstrong
> 1:11 PM
> Dan suggested setting a default here. I started doing some experiments to see
> what our current practical limits are.
> On stock Ubuntu 16.04 I start getting thread_resource_error at around 8000
> reserved threads. I'm not sure that the config reflects what people would use
> on production systems so continuing to investigate.
> Dan Hecht
> 1:31 PM
> We could also consider choosing a default dynamically based on the OS's
> setting, if that's necessary.
> Tim Armstrong
> 3:45 PM
> I increased some of the configs (I think I was limited by
> /sys/fs/cgroup/pids/user.slice/user-1000.slice/pids.max == 12288) and now it
> got oom-killed at ~26000 threads.
> I think unfortunately there are a lot of different OS knobs that impact this
> and they seem to evolve over time, so it's probably not feasible with a
> reasonable amount of effort to get it working on all common Linux distros.
> I was thinking ~5000, since 1000-2000 plan nodes is the most I've seen for a
> query running successfully in production.
> Maybe I should do this in a follow-on change, since we probably also want to
> add a test query at or near this limit.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
