Hi Vlad, Thank you for the suggestion. I recreated the issue and attached the stack traces I took. Let me know if there’s any other info I can provide. We narrowed the issue down to occurring when upgrading from 1.3.0 to any 1.4.x version.
Thanks,
Srinidhi
On 9/4/18, 8:19 PM, "Vladimir Rodionov" <vladrodio...@gmail.com> wrote:
Hi, Srinidhi
Next time you will see this issue, take jstack of a RS several times in a
row. W/o stack traces it is hard
to tell what was going on with your cluster after upgrade.
-Vlad
On Tue, Sep 4, 2018 at 3:50 PM Srinidhi Muppalla <srinid...@trulia.com>
wrote:
> Hello all,
>
> We are currently running Hbase 1.3.0 on an EMR cluster running EMR 5.5.0.
> Recently, we attempted to upgrade our cluster to using Hbase 1.4.4 (along
> with upgrading our EMR cluster to 5.16). After upgrading, the CPU usage
for
> all of our region servers spiked up to 90%. The load_one for all of our
> servers spiked from roughly 1-2 to 10 threads. After upgrading, the number
> of operations to the cluster hasn’t increased. After giving the cluster a
> few hours, we had to revert the upgrade. From the logs, we are unable to
> tell what is occupying the CPU resources. Is this a known issue with
1.4.4?
> Any guidance or ideas for debugging the cause would be greatly
> appreciated. What are the best steps for debugging CPU usage?
>
> Thank you,
> Srinidhi
>
<<attachment: hbasestacktrace.txt.zip>>
