Re: Parallel scanning of snapshots using hbase client api

2016-03-03 Thread Deepti Bhogle
We upgraded the lib version to 5.4.8 and though the parallel scanning exception issue is fixed we see a bigger problem with running multiple scanners. Our intended use for a single scanner was to let the snapshot scanner scan a given range of keys along with the region info they are located on a

Re: Parallel scanning of snapshots using hbase client api

2016-01-08 Thread Deepti Bhogle
We may not be able to upgrade to 5.4.8, since our 5.4.5 upgrade is pretty recent. I will discuss the possibility if the patch doesn’t work for us. Deepti Bhogle | Senior Software Engineer | Connexity, Inc. Phone: 858.652.4318 | dbho...@connexity.com On 1/8/16, 10:26 AM, "Matteo Bertozzi" w

Re: Parallel scanning of snapshots using hbase client api

2016-01-08 Thread Matteo Bertozzi
I think HBASE-14302 solved the problem by cutting the link creation. if you are using cdh 5.4.5 as mentioned above, try to upgrade to 5.4.8+ Matteo On Fri, Jan 8, 2016 at 6:55 AM, Enis Söztutar wrote: > Thanks Ted for the link. > > @Deepti, can you please do test the patch, and report back her

Re: Parallel scanning of snapshots using hbase client api

2016-01-08 Thread Enis Söztutar
Thanks Ted for the link. @Deepti, can you please do test the patch, and report back here or in jira about the findings. We should commit the patch. Enis On Thu, Jan 7, 2016 at 4:50 PM, Deepti Bhogle wrote: > Yes I think its a similar issue. The JIRA mentions a patch created that > should have

Re: Parallel scanning of snapshots using hbase client api

2016-01-07 Thread Deepti Bhogle
Yes I think its a similar issue. The JIRA mentions a patch created that should have the fix for the TableSnapshotScanner, we will check if we can apply the patch to our current version. Thanks, Deepti Bhogle | Senior Software Engineer | Connexity, Inc. Phone: 858.652.4318 | dbho...@connexity.com

Re: Parallel scanning of snapshots using hbase client api

2016-01-07 Thread Ted Yu
This seems related: HBASE-14128 FYI On Thu, Jan 7, 2016 at 9:24 AM, Deepti Bhogle wrote: > We currently do exactly that. We create multiple instances of > TableSnapshotScanner each with a unique dir location against the same > snapshot. But doing so gives us the exception mentioned. Does that

Re: Parallel scanning of snapshots using hbase client api

2016-01-07 Thread Deepti Bhogle
We currently do exactly that. We create multiple instances of TableSnapshotScanner each with a unique dir location against the same snapshot. But doing so gives us the exception mentioned. Does that mean we can’t run multiple instances at the same time? Deepti Bhogle | Senior Software Engineer

Re: Parallel scanning of snapshots using hbase client api

2016-01-07 Thread Enis Söztutar
TableSnapshotScanner itself does not support more than one scanner. Are you creating more than 1 TableSnapshotScanner in your parallel scan? Everytime a snapshot scanner is initiated, it will try to "restore" the snapshot to a temporary location out of the regular root directory in hdfs. You can t