Re: ANN: HBase 0.94.2 is available for download

2012-10-19 Thread syed kather
+1 pushing to maven repo Thanks and Regards, S SYED ABDUL KATHER On Thu, Oct 18, 2012 at 10:32 PM, lars hofhansl lhofha...@yahoo.com wrote: I'm on it. :) - Original Message - From: Amit Sela am...@infolinks.com To: user@hbase.apache.org Cc: Sent: Thursday,

回复: thrift usage

2012-10-19 Thread Xujingang
dear,hua i don't know 2012-10-19 Kind regards, Xu Jingang(徐金刚) Mobile: +86-18651865779 Email:jingan...@gmail.com 发件人:hua xiang 发送时间:2012-10-19 14:13 主题:thrift usage 收件人:user@hbase.apache.orguser@hbase.apache.org 抄送: HI, where is the interface of thrift, which is installed

HRegionInfo returns empty values.

2012-10-19 Thread Henry JunYoung KIM
Hi, hbase-users. To get a start-key and end-key from each region, I implemented simple code like this. HTable table = new HTable(admin.getConf(), admin.getTableName()); NavigableMapHRegionInfo, ServerName locations = table.getRegionLocations(); for

RE: HRegionInfo returns empty values.

2012-10-19 Thread Ramkrishna.S.Vasudevan
Can you tell me how the splitkeys are formed when the table was created? Or there are no splits at all for your table? If there are no splits then you will get empty start and endkey. Regards Ram -Original Message- From: Henry JunYoung KIM [mailto:henry.jy...@gmail.com] Sent:

RE: HRegionInfo returns empty values.

2012-10-19 Thread Anoop Sam John
Actually how many regions in your table? Only one region? In that case it will be having startkey and endkey as empty.. So your case what it prints looks to be correct. -Anoop- From: Henry JunYoung KIM [henry.jy...@gmail.com] Sent: Friday, October 19,

Re: HRegionInfo returns empty values.

2012-10-19 Thread Henry JunYoung KIM
yes, right ;) currently, I have just a single region. in this case, to get a start-key and and end-key, I need to use just a scanner. right? 2012. 10. 19., 오후 5:51, Ramkrishna.S.Vasudevan ramkrishna.vasude...@huawei.com 작성: Can you tell me how the splitkeys are formed when the table was

RE: HRegionInfo returns empty values.

2012-10-19 Thread Ramkrishna.S.Vasudevan
What is your need? You want to scan the rows i.e data in the table? Or you want to start and endkeys. Actually for single region just empty bytes represent the start and endkey. So am not getting what you want from that. If you want data then create a scan object for the table and do a scan :)

Re: thrift usage

2012-10-19 Thread Harsh J
Hey Hua, The RPM (I'm assuming you speak of Apache Bigtop and Bigtop-derived packages) does not seem to include this file anywhere. I filed https://issues.apache.org/jira/browse/BIGTOP-756 to address that. On Fri, Oct 19, 2012 at 11:43 AM, hua xiang adam_...@yahoo.com wrote: HI, where is

Follow-up to regionservers not being online - more logs included

2012-10-19 Thread Dan Brodsky
I'm still having several issues with my cluster. This used to all work, and there have been no recent configuration changes. To recap, Master and regionservers all appear to start successfully, but several regionservers do not show as online on Hbase master status page. Moreover, there appear to

Re: Hbase sequential row merging in MapReduce job

2012-10-19 Thread Doug Meil
As long as you know your keyspace, you should be able to create your own splits. See TableInputFormatBase for the default implementation (which is 1 input split per region) On 10/19/12 9:32 AM, Eric Czech eczec...@gmail.com wrote: Hi everyone, Is there any way to create an InputSplit for

Question on Scanner REST API Usage

2012-10-19 Thread Jerry Lam
Hi HBase community: I have a few questions on the usage of Scanner via REST API: - From the XML schema ( http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/rest/package-summary.html#xmlschema), we can set the maximum number of values to return for each call to next() by specifying the batch

Re: Hbase sequential row merging in MapReduce job

2012-10-19 Thread Michael Segel
Outch... That could get very nasty. You may end up with a lot of uneven splits. Suppose your 'metric1' spans 3 regions, 'metric2' 1 but its still in the same split as 'metric1' and then 'metric3' is in two regions, 'metric4' is in two regions where its split between the end of 'metric3' and

Re: Checking major compaction

2012-10-19 Thread kiran
Hi all, I mistakenly execution major compaction on the entire table instead of region by region. It is causing so much delay in the write performance, also on the disk and networkI/O. How can i stop major compaction now ? Thanks Kiran On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan

Re: Checking major compaction

2012-10-19 Thread Kevin O'dell
Kiran, I don't believe you can. If you have a sane region count/region size it should be over soon. On Fri, Oct 19, 2012 at 11:37 AM, kiran kiran.sarvabho...@gmail.com wrote: Hi all, I mistakenly execution major compaction on the entire table instead of region by region. It is causing

Re: Unable to add co-processor to table through HBase api

2012-10-19 Thread anil gupta
It worked. :) On Thu, Oct 18, 2012 at 10:54 PM, Ramkrishna.S.Vasudevan ramkrishna.vasude...@huawei.com wrote: Ok Anil.. Not a problem.. My intention was to just see if the api was working during createtable so that it will help you. Regards Ram -Original Message- From: anil

Re: Checking major compaction

2012-10-19 Thread kiran
Kevin, Thanks for your reply, can restarting region servers help to kill the already existing compaction processes in region servers.. Thanks Kiran On Fri, Oct 19, 2012 at 9:16 PM, Kevin O'dell kevin.od...@cloudera.comwrote: Kiran, I don't believe you can. If you have a sane region

Re: Checking major compaction

2012-10-19 Thread Jean-Daniel Cryans
You could just disable then enable the table. J-D On Fri, Oct 19, 2012 at 9:10 AM, kiran kiran.sarvabho...@gmail.com wrote: Kevin, Thanks for your reply, can restarting region servers help to kill the already existing compaction processes in region servers.. Thanks Kiran On Fri, Oct

Re: hbase.client.scanner.timeout.period not being respected

2012-10-19 Thread Jean-Daniel Cryans
That config is only available in trunk (per https://issues.apache.org/jira/browse/HBASE-6170), are you using a trunk snapshot? J-D On Thu, Oct 18, 2012 at 8:25 AM, Bai Shen baishen.li...@gmail.com wrote: I've set hbase.client.scanner.timeout.period on my client to 30, but I'm still getting

Re: Follow-up to regionservers not being online - more logs included

2012-10-19 Thread ramkrishna vasudevan
Can you try like this. Just stop your cluster. Start one Master and RegionServer. May be for temporarily just have one ZK to which the master and RS is able to connect. After this is done, just see on the master UI whether the ROOT and META table got assigned? means you can see in the MASTER UI

Re: Hbase sequential row merging in MapReduce job

2012-10-19 Thread Eric Czech
Well it looks like I might be able to make it work in TableInputFormatBase if I parse the start and end keys and add the logic there (thanks Doug). I definitely want to avoid the reduce step and since I am storing timeseries data, I can probably just live without putting any part of the date in

Re: [DISCUSS] Default to Hadoop 1.1 in HBase 0.94?

2012-10-19 Thread Ted Yu
Here is some more background information from https://issues.apache.org/jira/browse/HBASE-6962 : hadoop 1.1 release annoucement: http://search-hadoop.com/m/0eTo41c8GSb The thinking behind new recommendation is that future patches for reducing MTTR may depend on the improvement unique to hadoop

RE: Follow-up to regionservers not being online - more logs included

2012-10-19 Thread rama krishna
Hi Dan I can check this in some time and see what is the problem. I can try helping you as far as possible. RegardsRam From: ramkrishna.vasude...@huawei.com To: ram_krish...@hotmail.com Subject: FW: Follow-up to regionservers not being online - more logs included Date: Fri, 19 Oct 2012

HRegionInfo returns empty values.

2012-10-19 Thread JUN YOUNG KIM
Hi, hbase-users. To get a start-key and end-key from each region, I implemented simple code like this. HTable table = new HTable(admin.getConf(), admin.getTableName()); NavigableMapHRegionInfo, ServerName locations = table.getRegionLocations(); for

Re: HRegionInfo returns empty values.

2012-10-19 Thread Ted Yu
Which HBase version are you using ? How many regions are there for the underlying table ? Thanks On Fri, Oct 19, 2012 at 1:36 AM, JUN YOUNG KIM juneng...@me.com wrote: Hi, hbase-users. To get a start-key and end-key from each region, I implemented simple code like this. HTable

Re: High IPC Latency

2012-10-19 Thread Yousuf Ahmad
Hi Lars, We are following your suggestion and testing against a single region server. We just ran a test against a remote region server and soon we will test against a local one as well. We will get back to you soon with the results. It will take us a couple of days to port to and test our code

RE: High IPC Latency

2012-10-19 Thread Pamecha, Abhishek
Also, I hope no coprocessors are in play. Thanks, Abhishek -Original Message- From: Yousuf Ahmad [mailto:myahm...@gmail.com] Sent: Friday, October 19, 2012 10:12 AM To: user@hbase.apache.org Cc: Ivan Brondino; Ricardo Vilaça Subject: Re: High IPC Latency Hi Lars, We are following

Re: Thrift Python client with regex

2012-10-19 Thread Stack
On Thu, Oct 18, 2012 at 7:13 PM, Norbert Burger norbert.bur...@gmail.com wrote: We had the same question earlier. Unfortunately the documentation is wrong on this account; scannerOpen resolves to either a call to scan.addFamily or scan.addColumn, and neither directly supports regex matching.

RE: Using filters in REST/stargate returns 204 (No content)

2012-10-19 Thread Kumar, Suresh
Here is the hbase shell command which works, I am not able to get these results using curl/stargate. scan 'apachelogs', { COLUMNS = 'mylog:pcol', FILTER = SingleColumnValueFilter('mylog','pcol', =, 'regexstring: ERROR x.') } Here is the curl command which does not work: curl -v -H

Re: Using filters in REST/stargate returns 204 (No content)

2012-10-19 Thread Jerry Lam
Hi Suresh: Have you tried to create a scanner without the filter? Does it return errors as well? Best Regards, Jerry On Fri, Oct 19, 2012 at 1:16 PM, Kumar, Suresh suresh.kum...@emc.comwrote: Here is the hbase shell command which works, I am not able to get these results using

Re: Question on Scanner REST API Usage

2012-10-19 Thread Andrew Purtell
Hi Jerry, Is there a way to set the number of rows for caching that will be passed to scanners (setCaching)? No. Scanners on the REST server will use the default value for setCaching provided in the hbase-site.xml deployed for the REST server. Overriding it hasn't been considered a good idea

Re: Using filters in REST/stargate returns 204 (No content)

2012-10-19 Thread Andrew Purtell
That's a good suggestion. Additionally, if you run the REST server at DEBUG level logging there may be additional clues in the log. On Fri, Oct 19, 2012 at 10:54 AM, Jerry Lam chiling...@gmail.com wrote: Hi Suresh: Have you tried to create a scanner without the filter? Does it return errors

RE: High IPC Latency

2012-10-19 Thread Yousuf Ahmad
No coprocessors :-) On Oct 19, 2012 1:21 PM, Pamecha, Abhishek apame...@x.com wrote: Also, I hope no coprocessors are in play. Thanks, Abhishek -Original Message- From: Yousuf Ahmad [mailto:myahm...@gmail.com] Sent: Friday, October 19, 2012 10:12 AM To: user@hbase.apache.org

RE: Using filters in REST/stargate returns 204 (No content)

2012-10-19 Thread Kumar, Suresh
curl -v -H Content-Type: text/xml -d 'Scanner/' http://localhost:8080/apachelogs/scanner/ returns a Location URL which returns all the data, if I send the filter, I do not get errors, I just don't get any data (204 No content) from the location URL. Let me try turning on DEBUG at the REST

Re: Where is code in hbase that physically delete a record?

2012-10-19 Thread PG
Hi, Anoop and Ram, As I have coded the idea, the detailed instructions are very helpful. One minor thing to add is that coming out from scanner are the KeyValues which are already sorted by column qualifier and time stamps. though i did not find it mentioned in java doc, but i found it very

RE: thrift with filters

2012-10-19 Thread Kumar, Suresh
Jon: Welcome to the club. I am having a very bad experience using thrift (python) and REST (stargate) with filters. Filters work well with a Java client, but no luck with thrift or REST. Good luck, Suresh -Original Message- From: Jonathan Bishop [mailto:jbishop@gmail.com]

Re: thrift with filters

2012-10-19 Thread Andrew Purtell
Yes, it is fair to say that because filters have been evolving at a different rate than either the Thrift or REST gateway, the first class support for filters is the Java client. I would expect this to be addressed in subsequent releases as we near 1.0 and of course patches are always welcome. On

Re: thrift with filters

2012-10-19 Thread Jonathan Bishop
Thanks for the update Andrew. I'll keep an eye out for updates. Jon On Fri, Oct 19, 2012 at 4:39 PM, Andrew Purtell apurt...@apache.org wrote: Yes, it is fair to say that because filters have been evolving at a different rate than either the Thrift or REST gateway, the first class support

RE: scaling a low latency service with HBase

2012-10-19 Thread Pamecha, Abhishek
Here are a few of my thoughts: If possible, you might want to localize your data to a few regions if you can and then may be have exclusive access to those regions. This way, external load will not impact you. I have heard that write penalty of SSDs is quite high. But I think, they will still

Re: scaling a low latency service with HBase

2012-10-19 Thread Amandeep Khurana
Answers inline On Fri, Oct 19, 2012 at 4:31 PM, Dave Latham lat...@davelink.net wrote: I need to scale an internal service / datastore that is currently hosted on an HBase cluster and wanted to ask for advice from anyone out there who may have some to share. The service does simple key value

Re: crafting your key - scan vs. get

2012-10-19 Thread Neil Yalowitz
Thanks Ian! Very helpful breakdown. For this use case, I think the multi-version row structure is ruled out. We will investigate the onekey-manycolumn approach. Also, the more I study the mechanics behind a SCAN vs GET, the more I believe the informal test I did is inaccurate. What does

Re: thrift with filters

2012-10-19 Thread Stack
On Fri, Oct 19, 2012 at 3:56 PM, Jonathan Bishop jbishop@gmail.com wrote: Hi, Taking a look at the thrift interface to hbase and I am having a hard time finding any way to set the filterString for scans. Anyone know how to do this? What have you tried? St.Ack

Re: scaling a low latency service with HBase

2012-10-19 Thread Andrew Purtell
What Amandeep said, and also: You said your working set is randomly distributed but, if frequent invalidation isn't a concern and read accesses are still clustered temporally, an in-memory cache out in front of the cluster would smooth over periods when the disks are busy servicing MR workload or

Re: [DISCUSS] Default to Hadoop 1.1 in HBase 0.94?

2012-10-19 Thread lars hofhansl
Hmm... Probably not the best idea then. I'll just bump the default to 1.0.4. - Original Message - From: Ted Yu yuzhih...@gmail.com To: d...@hbase.apache.org; user@hbase.apache.org Cc: lars hofhansl lhofha...@yahoo.com Sent: Friday, October 19, 2012 9:29 AM Subject: Re: [DISCUSS] Default

Re: Where is code in hbase that physically delete a record?

2012-10-19 Thread Anoop John
Yes the KVs coming out from your delegate Scanner will be in sorted form.. Also with all other logic applied like removing TTL expired data, handling max versions etc.. Thanks for updating.. -Anoop- On Sat, Oct 20, 2012 at 1:11 AM, PG pengyunm...@gmail.com wrote: Hi, Anoop and Ram, As I have

Re: Where is code in hbase that physically delete a record?

2012-10-19 Thread ramkrishna vasudevan
Hi Always any KV that comes from scanning are sorted lexographically and the recent timestamps will come out first. So even if your data writes col qualifier c2 first and then c1 because of lexographical ordering c1 will be coming first. Also recent versions of a row will be coming out first and