RE: Region server idle

2021-01-27 Thread Marc Hoppins
. -Original Message- From: Sean Busbey Sent: Tuesday, January 26, 2021 5:46 PM To: Hbase-User Subject: Re: Region server idle EXTERNAL You can change the logging for just the balancer system so that not so much goes into the logs. The logger to change

RE: Region server idle

2021-01-26 Thread Marc Hoppins
. -Original Message- From: Sean Busbey Sent: Tuesday, January 26, 2021 5:46 PM To: Hbase-User Subject: Re: Region server idle EXTERNAL You can change the logging for just the balancer system so that not so much goes into the logs. The logger to change is org.apache.hadoop.hbase.master.balancer

Re: Region server idle

2021-01-26 Thread Sean Busbey
; Sent: Tuesday, January 26, 2021 3:59 PM > To: Hbase-User > Subject: Re: Region server idle > > EXTERNAL > > Hi Marc! > > Did turning up the balancer's logging confirm that it is actually making > plans? Did it confirm what weights are being used for the various balancer

RE: Region server idle

2021-01-26 Thread Marc Hoppins
. -Original Message- From: Sean Busbey Sent: Tuesday, January 26, 2021 3:59 PM To: Hbase-User Subject: Re: Region server idle EXTERNAL Hi Marc! Did turning up the balancer's logging confirm that it is actually making plans? Did it confirm what weights are being used for the various balancer

Re: Region server idle

2021-01-26 Thread Sean Busbey
were well-integrated within a day or so. > > -Original Message- > From: Josh Elser > Sent: Sunday, January 24, 2021 10:22 PM > To: user@hbase.apache.org > Subject: Re: Region server idle > > EXTERNAL > > Yes, each RegionServer has its own writeahead log (wh

RE: Region server idle

2021-01-26 Thread Marc Hoppins
in October 2020 and they were well-integrated within a day or so. -Original Message- From: Josh Elser Sent: Sunday, January 24, 2021 10:22 PM To: user@hbase.apache.org Subject: Re: Region server idle EXTERNAL Yes, each RegionServer has its own writeahead log (which is named with that RS

Re: Region server idle

2021-01-24 Thread Josh Elser
sage- From: Josh Elser Sent: Tuesday, January 12, 2021 4:56 PM To: user@hbase.apache.org Subject: Re: Region server idle EXTERNAL Yes, in general, HDFS rebalancing will cause a decrease in the performance of HBase as it removes the ability for HBase to short-circuit some read logic. It sh

RE: Region server idle

2021-01-18 Thread Marc Hoppins
gt; -Original Message- > From: Josh Elser > Sent: Monday, January 11, 2021 5:34 PM > To: user@hbase.apache.org > Subject: Re: Region server idle > > EXTERNAL > > The Master stacktrace you have there does read as a bug, but it shouldn't be > affecting balancing. &

Re: Region server idle

2021-01-12 Thread Josh Elser
, 2021 5:34 PM To: user@hbase.apache.org Subject: Re: Region server idle EXTERNAL The Master stacktrace you have there does read as a bug, but it shouldn't be affecting balancing. That Chore is doing work to apply space quotas, but your quota here is only doing RPC (throttle) quotas. Might

RE: Region server idle

2021-01-12 Thread Marc Hoppins
compaction. Once again, experience in this area may be giv8ing me misinformation. -Original Message- From: Josh Elser Sent: Monday, January 11, 2021 5:34 PM To: user@hbase.apache.org Subject: Re: Region server idle EXTERNAL The Master stacktrace you have there does read as a bug

Re: Region server idle

2021-01-11 Thread Josh Elser
pins Sent: Monday, January 11, 2021 9:52 AM To: user@hbase.apache.org Subject: RE: Region server idle EXTERNAL I tried. Appears to have failed reading data from hbase:meta. These are repeated errors for the whole run of list_quotas. A balance task was run on Friday. It took 9+ hours. The a

RE: Region server idle

2021-01-11 Thread Marc Hoppins
used by GSSException: No valid credentia ls provided (Mechanism level: Failed to find any Kerberos tgt)]] -Original Message- From: Stack Sent: Saturday, January 9, 2021 1:52 AM To: Hbase-User Subject: Re: Region server idle E

RE: Region server idle

2021-01-11 Thread Marc Hoppins
on: No valid credentia ls provided (Mechanism level: Failed to find any Kerberos tgt)]] -Original Message- From: Stack Sent: Saturday, January 9, 2021 1:52 AM To: Hbase-User Subject: Re: Region server idle EXTERNAL Looking at code around except

Re: Region server idle

2021-01-08 Thread Stack
lang.Thread.run(Thread.java:748) > > -Original Message- > From: Marc Hoppins > Sent: Friday, January 8, 2021 10:57 AM > To: user@hbase.apache.org > Subject: RE: Region server idle > > EXTERNAL > > So, I tried decommission that RS and recommission it. No change.

RE: Region server idle

2021-01-08 Thread Marc Hoppins
- From: Marc Hoppins Sent: Friday, January 8, 2021 10:57 AM To: user@hbase.apache.org Subject: RE: Region server idle EXTERNAL So, I tried decommission that RS and recommission it. No change. Server still idle. Tried decommission another server and see if HBASE sets itself right. Now I have two

RE: Region server idle

2021-01-08 Thread Marc Hoppins
-Original Message- From: Sean Busbey Sent: Thursday, January 7, 2021 7:30 PM To: Hbase-User Subject: Re: Region server idle EXTERNAL Sounds like https://issues.apache.org/jira/browse/HBASE-24139 The description of that jira has a workaround. On Thu, Jan 7, 2021, 05:23 Marc Hoppins wrote

Re: Region server idle

2021-01-07 Thread Sean Busbey
Sounds like https://issues.apache.org/jira/browse/HBASE-24139 The description of that jira has a workaround. On Thu, Jan 7, 2021, 05:23 Marc Hoppins wrote: > Hi all, > > I have a setup with 67 region servers. 29 Dec one system had to be shut > down to have EMM module swapped out - which took

Re: Region server idle

2021-01-07 Thread Stack
out that MEMSTORE and STOREFILES are also 0, and > that USED HEAP is only <200MB, whereas all other hosts heap used is >16GB > and <23GB. > > -Original Message- > From: Marc Hoppins > Sent: Thursday, January 7, 2021 12:23 PM > To: user@hbase.apache.org > Subject: Region

RE: Region server idle

2021-01-07 Thread Marc Hoppins
t: Region server idle EXTERNAL Hi all, I have a setup with 67 region servers. 29 Dec one system had to be shut down to have EMM module swapped out - which took one work day. Host was back online 30 Dec. My HBASE is very basic so I appreciate your patience. My understanding of the defaults that

Region server idle

2021-01-07 Thread Marc Hoppins
Hi all, I have a setup with 67 region servers. 29 Dec one system had to be shut down to have EMM module swapped out - which took one work day. Host was back online 30 Dec. My HBASE is very basic so I appreciate your patience. My understanding of the defaults that are setup is that a major