Thanks Marcos for reference. We are using filter on only one column. And we further narrow down filter and results using ->
scan.addColumn(family, qualifier); PageFilter pageFilter = new PageFilter(size); mainFilterList.addFilter(pageFilter); we have two columns family - content ( most of the data) and metadata (only metadata). And the column we are applying filter against is in content family. In that case, will HBASE-5416 help? We are also using ROW bloomfilter. Will ROWCOlUMN bloomfilter help in our scenario? Saurabh. From: Marcos Luis Ortiz Valmaseda [mailto:[email protected]] Sent: Thursday, March 28, 2013 10:36 PM To: [email protected]; Agarwal, Saurabh [CCC-OT_IT]; [email protected] Subject: Re: coprocessor is timing out in 0.94 Regards, Saurabh. I see that you are using SingleColumnValueFilter. Look for these links: http://gbif.blogspot.com/2012/05/optimizing-hbase-mapreduce-scans-for.html http://mapredit.blogspot.com/2012/05/using-filters-in-hbase-to-match-two.html Take a look later to this link, about the working to improve scans: https://issues.apache.org/jira/browse/HBASE-5416 2013/3/28 Agarwal, Saurabh <[email protected]<mailto:[email protected]>> Ted, Thanks for response. Here is the filter we are using - SingleColumnValueFilter(Bytes.toBytes(columnFamily), Bytes.toBytes(columnQualifier), CompareFilter.CompareOp.EQUAL, new RegexStringComparator("(?i)"+"keyword")); The thread dump at different points show that coprocessor is getting called. Also logs showed it keep processing. But the speed is much slower compare to 0.92. Regards, Saurabh. -----Original Message----- From: Ted Yu [mailto:[email protected]<mailto:[email protected]>] Sent: Thursday, March 28, 2013 6:57 PM To: [email protected]<mailto:[email protected]> Subject: Re: coprocessor is timing out in 0.94 bq. I checked thread dump If there was no exception in region server logs, thread dump of region server when your coprocessor was running would reveal where it got stuck. >From your description below, looks like you can utilize HBASE-5416 Improve >performance of scans with some kind of filters. bq. to apply the filter on one of the column Basically this column is the essential column. Cheers On Thu, Mar 28, 2013 at 3:22 PM, Ted Yu <[email protected]<mailto:[email protected]>> wrote: > bq. when I removed the filter, it ran fine in 0.94 > > Can you disclose more information about your filter ? > > BTW 0.94.6 was just released which is fully compatible with 0.94.2 > > Cheers > > On Thu, Mar 28, 2013 at 3:18 PM, Agarwal, Saurabh < > [email protected]<mailto:[email protected]>> wrote: > >> Hi, >> >> We are in process of migrating from 0.92.1 to 0.94.2. A coprocessor >> was running fine in 0.92. After migrating to 0.94, the client is >> timing out (java.net.SocketTimeoutException). We are using >> coprocessor to apply the filter on one of the column and return the >> columns that match with that filter criteria. I checked thread dump, >> region server, web UI, logs. There is no error or exception. One >> thing I noticed that when I removed the filter, it ran fine in 0.94 as well. >> >> Please advise if there is any specific setting we need to make in 0.94. >> >> Thanks, >> Saurabh. >> > > -- Marcos Ortiz Valmaseda, Data-Driven Product Manager at PDVSA Blog: http://dataddict.wordpress.com/ LinkedIn: http://www.linkedin.com/in/marcosluis2186 Twitter: @marcosluis2186<http://twitter.com/marcosluis2186>
