Re: Does HBase master process need to do listSnapshots

[Abhishek Gupta]

Hi Esteban,

Thank you so much for the response, it was indeed the problem. Had a
followup question of this, why is this RPC call not respecting the HBase
RPC time-out
set for 90 seconds by us.

Thanks


On Thu, May 30, 2019 at 10:59 PM Esteban Gutierrez
 wrote:

> Hello Abhishek,
>
> The HBase master UI will try to list the snapshots if there are available.
> Since GetCompletedSnapshots goes to the filesystem to scan for all the
> possible snapshots that can take some time depending on the number of
> snapshots or the load of the NN.
>
> thanks,
> esteban.
>
>
> --
> Cloudera, Inc.
>
>
>
> On Thu, May 30, 2019 at 12:05 PM Abhishek Gupta 
> wrote:
>
> > Hi,
> >
> > We are trying to debug an issue where multiple listSnapshots calls seem
> to
> > be running extremely slow RPC calls on HBase Master.
> > These calls appear to be happening from Master node itself. I wanted to
> > understand if there is any Master chore service that needs to
> periodically
> > do an operation which involves a lisSnapshots?
> >
> >
> *{"call":"GetCompletedSnapshots(org.apache.hadoop.hbase.protobuf.generated.MasterProtos$GetCompletedSnapshotsRequest)","starttimems":1559233674685,"responsesize":175421,"method":"GetCompletedSnapshots","processingtimems":420615,"client":"
> > 10.0.1.193:58296
> > <http://10.0.1.193:58296>","queuetimems":0,"class":"HMaster"}*
> > Some of these calls are taking 1000 seconds of processing time.
> >
> > Thanks in advance,
> > Abhishek
> >
>

Does HBase master process need to do listSnapshots

[Abhishek Gupta]

Hi,

We are trying to debug an issue where multiple listSnapshots calls seem to
be running extremely slow RPC calls on HBase Master.
These calls appear to be happening from Master node itself. I wanted to
understand if there is any Master chore service that needs to periodically
do an operation which involves a lisSnapshots?
*{"call":"GetCompletedSnapshots(org.apache.hadoop.hbase.protobuf.generated.MasterProtos$GetCompletedSnapshotsRequest)","starttimems":1559233674685,"responsesize":175421,"method":"GetCompletedSnapshots","processingtimems":420615,"client":"10.0.1.193:58296
","queuetimems":0,"class":"HMaster"}*
Some of these calls are taking 1000 seconds of processing time.

Thanks in advance,
Abhishek

Re: NoSQL day summary

[Abhishek Gupta]

Thank yo so much in advance Josh for the youtube videos.

On Wed, May 29, 2019, 8:19 PM Josh Elser  wrote:

>
>
> On 5/28/19 1:04 AM, Anoop John wrote:
> > Thanks for sharing the details Josh. Good to know it went well.
> >
> >> Talks were recorded with their slide presentation. Editing/processing on
> > these will take some time -- I'd expect a month before I'm able to get
> > these posted on YouTube for everyone
> >
> > Waiting to see the presentation videos..  The ppts alone already
> available ?
>
> No, they are not.
>
> You'll find when organizing such events that it's harder than pulling
> teeth to get people to send you slides at all, much less prior to the
> event :)
>

Re: Requesting access to hbase slack channel

[Abhishek Gupta]

Hi,

I would like an invitation too abhila...@gmail.com

Thanks

On Fri, Jan 18, 2019 at 11:38 AM Manjeet Singh 
wrote:

> Done
>
> On Fri, 18 Jan 2019, 11:24 Buchi Reddy Busi Reddy  wrote:
>
> > Can you also invite mailtobu...@gmail.com please?
> >
> > On Thu, Jan 17, 2019 at 8:34 PM Manjeet Singh <
> manjeet.chand...@gmail.com>
> > wrote:
> >
> > > Seems someone else already did it
> > >
> > > Manjeet
> > >
> > > On Wed, 16 Jan 2019, 17:33 Nihal Jain  > >
> > > > Hi
> > > >
> > > > Could you please invite me: nihaljain...@gmail.com <
> > > nihaljain...@gmail.com
> > > > >?
> > > >
> > > > Regards,
> > > > Nihal
> > > >
> > > > On Wed, 16 Jan, 2019, 2:54 PM Peter Somogyi  > wrote:
> > > >
> > > > > Sent invitation to madhurpan...@gmail.com.
> > > > >
> > > > > On Wed, Jan 16, 2019 at 7:27 AM Madhur Pant <
> madhurpan...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Team,
> > > > > >
> > > > > > I was wondering if I could get access to the HBase user slack
> > channel
> > > > > >
> > > > > > https://apache-hbase.slack.com
> > > > > >
> > > > > > It says here 
> > > > that I
> > > > > > should email you guys  :)
> > > > > >
> > > > > > Thanks,
> > > > > > Madhur Pant
> > > > > >
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (HBASE-21234) Archive folder not getting cleaned due to SnapshotHFileCleaner error

[Abhishek Gupta (JIRA)]

Abhishek Gupta created HBASE-21234:
--

 Summary: Archive folder not getting cleaned due to 
SnapshotHFileCleaner error
 Key: HBASE-21234
 URL: https://issues.apache.org/jira/browse/HBASE-21234
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 1.1.2
Reporter: Abhishek Gupta


Getting following exception during ChoreService runs in HBase Master logs. As a 
result we are accumulating a lot of data in archive folder as archive is not 
getting reclaimed. 

 
{code:java}
2018-09-26 10:15:06,188 ERROR [master01,16000,1536315941769_ChoreService_3] 
snapshot.SnapshotHFileCleaner: Exception while checking if files were valid, 
keeping them just in case. java.io.IOException: ExecutionException at 
org.apache.hadoop.hbase.snapshot.SnapshotManifestV2.loadRegionManifests(SnapshotManifestV2.java:161)
 at 
org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:364)
 at 
org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:130)
 at 
org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:128)
 at 
org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.getHFileNames(SnapshotReferenceUtil.java:357)
 at 
org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.getHFileNames(SnapshotReferenceUtil.java:340)
 at 
org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner$1.filesUnderSnapshot(SnapshotHFileCleaner.java:88)
 at 
org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getSnapshotsInProgress(SnapshotFileCache.java:303)
 at 
org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:194)
 at 
org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:63)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:287)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:211)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:234)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:206)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:234)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:206)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:234)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:206)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:234)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:206)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:234)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:206)
 at 
org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:130)
 at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.java:748) Caused by: 
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos at 
org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.internalGetFieldAccessorTable(SnapshotProtos.java:1190)
 at 
com.google.protobuf.GeneratedMessage.getDescriptorForType(GeneratedMessage.java:98)
 at 
com.google.protobuf.AbstractMessage$Builder.findMissingFields(AbstractMessage.java:789)
 at 
com.google.protobuf.AbstractMessage$Builder.findMissingFields(AbstractMessage.java:780)
 at 
com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
 at 
com.google.protobuf.AbstractMessage.newUninitializedMessageException(AbstractMessage.java:237)
 at 
com.google.protobuf.AbstractParser.newUninitializedMessageException(AbstractParser.java:57)
 at 
com.google.protobuf.AbstractParser.checkMessageInitialized(AbstractParser.java:71)
 at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)