[
https://issues.apache.org/jira/browse/IMPALA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned IMPALA-9754:
------------------------------------
Assignee: Qifan Chen
> buffer_pool_limit error message is confusing
> --------------------------------------------
>
> Key: IMPALA-9754
> URL: https://issues.apache.org/jira/browse/IMPALA-9754
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Qifan Chen
> Priority: Minor
> Labels: newbie, ramp-up
>
> The error message for an invalid value of {{buffer_pool_limit}} is confusing.
> For example:
> {code:java}
> I0515 18:30:20.143811 1 exec-env.cc:480] System memory available: 30.63
> GB (from physical mem)
> I0515 18:30:20.144037 1 exec-env.cc:487] CGroup memory limit for this
> process reduces physical memory available to: 9.31 GB
> I0515 18:30:20.144050 1 exec-env.cc:503] Using process memory limit: 7.45
> GB (--mem_limit=80% of 9.31 GB)
> I0515 18:30:20.150154 1 status.cc:129] Invalid --buffer_pool_limit value,
> must be a percentage or positive bytes value or percentage: 85%
> @ 0xc33b79
> @ 0x111afb8
> @ 0x1251256
> @ 0xb730b3
> @ 0x7f60da4c9554
> @ 0xbf3e86
> F0515 18:30:20.150177 1 impalad-main.cc:73] Invalid --buffer_pool_limit
> value, must be a percentage or positive bytes value or percentage: 85%
> . Impalad exiting. {code}
> 1: "must be a percentage or positive bytes value or percentage" needs to be
> revised to something like "must be a positive bytes value or percentage"
> 2: The error message makes it sound like the value of 85% is invalid, when
> in reality, it is a valid value - Impala does some internal calculations of
> what the actual {{buffer_pool_limit}} should be internally (see
> ExecEnv::Init) - these calculations first compute the mem limit for the
> process (ChooseProcessMemLimit) based on the physical machines memory +
> cgroup settings, and then if mem_limit_includes_jvm == true it subtracts the
> value of Xmx from the amount of available memory - we don't have logic to
> capture if the results of that subtraction becomes a negative number, which
> causes the buffer_pool_limit parsing to fail in a weird way. We should add a
> check to see if {{admit_mem_limit_ \-=
> JvmMemoryMetric::HEAP_MAX_USAGE->GetValue()}} is negative, and then exit with
> an informative status message
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
