[
https://issues.apache.org/jira/browse/HBASE-29679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Connell updated HBASE-29679:
------------------------------------
Description:
When under heavy load, a RegionServer may need to serve a very large number of
{{RpcThrottlingExceptions}} per second. Ideally, these should be cheap to send,
because they are HBase's load shedding mechanism.
At my company, we sometimes see that sending many {{RpcThrottlingExceptions}}
isn't always easy. The most expensive part is generating the exception's stack
trace, and then serializing that over the wire. This is not necessary, so it
can be skipped to save a lot of work.
I'm attaching a CPU-time profile, wall-clock-time profile, and allocation
profile, showing the problem in action. In the allocation profile,
{{StringUtils.stringifyException}} is responsible for 26% of allocations. In
the CPU-time profile, {{StringUtils.stringifyException}} plus
{{RpcThrottlingException.<init>}} is directly responsible for 4% of CPU time,
and indirectly responsible for more time spent garbage collecting later.
was:
When under heavy load, a RegionServer may need to serve a very large number of
RpcThrottlingExceptions per second. Ideally, these should be cheap to send,
because they are HBase's load shedding mechanism.
At my company, we sometimes see that sending many RpcThrottlingExceptions
isn't always easy. The most expensive part is generating the exception's stack
trace, and then serializing that over the wire. This is not necessary, so it
can be skipped to save a lot of work.
I'm attaching a CPU-time profile, wall-clock-time profile, and allocation
profile, showing the problem in action. In the allocation profile,
StringUtils.stringifyException is responsible for 26% of allocations. In the
CPU-time profile, StringUtils.stringifyException plus
{{RpcThrottlingException.<init>}} is directly responsible for 4% of CPU time,
and indirectly responsible for more time spent garbage collecting later.
> Suppress stack trace in RpcThrottlingException
> ----------------------------------------------
>
> Key: HBASE-29679
> URL: https://issues.apache.org/jira/browse/HBASE-29679
> Project: HBase
> Issue Type: Improvement
> Components: Quotas
> Reporter: Charles Connell
> Priority: Minor
> Attachments: failRegionAction.alloc.html, failRegionAction.cpu.html,
> failRegionAction.wall.html
>
>
> When under heavy load, a RegionServer may need to serve a very large number
> of {{RpcThrottlingExceptions}} per second. Ideally, these should be cheap to
> send, because they are HBase's load shedding mechanism.
> At my company, we sometimes see that sending many
> {{RpcThrottlingExceptions}} isn't always easy. The most expensive part is
> generating the exception's stack trace, and then serializing that over the
> wire. This is not necessary, so it can be skipped to save a lot of work.
> I'm attaching a CPU-time profile, wall-clock-time profile, and allocation
> profile, showing the problem in action. In the allocation profile,
> {{StringUtils.stringifyException}} is responsible for 26% of allocations. In
> the CPU-time profile, {{StringUtils.stringifyException}} plus
> {{RpcThrottlingException.<init>}} is directly responsible for 4% of CPU time,
> and indirectly responsible for more time spent garbage collecting later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
