RE: Hbase pseudo distributed setup not starting

2013-06-24 Thread rajeshbabu chintaguntla
Hi Rajkumar, bq. my hdfs-site.xml file is The configurations you have mentioned should be in hbase-site.xml. Can you check zk quorum,Namenode and DataNode are running fine? One more thing is fs.defaultFs or fs.default.name in core-site.xml of hadoop should be hdfs://localhost:9000/ property

hbase.client.ipc.pool.size problem with hbase 0.95.1

2013-06-24 Thread Xiong LIU
Hi, all, when I configure ipc pool in hbase-site.xml as follows: property namehbase.client.ipc.pool.type/name valueRoundRobinPool/value /property property namehbase.client.ipc.pool.size/name value1/value /property the start and stop processes of hbase run without any problem.

Re: How to specify the hbase.zookeeper.quorum on command line invoking hbase shell

2013-06-24 Thread Shahab Yunus
Have you tried creating your own small script in which you set the relevant environment variables per session (using 'export' for example)? On Mon, Jun 24, 2013 at 1:33 AM, Stephen Boesch java...@gmail.com wrote: We want to connect to a non-default / remote hbase server by setting

RE: How to specify the hbase.zookeeper.quorum on command line invoking hbase shell

2013-06-24 Thread rajeshbabu chintaguntla
Can you try copying hbase-site.xml to other folder and change hbase.zookeeper.quorum to remote server and then use below command. ./hbase --config other folder shell Then shell will start with the configurations in hbase-site.xml of new folder. I didnt try this, but it should help you.

Re: How to specify the hbase.zookeeper.quorum on command line invoking hbase shell

2013-06-24 Thread Shahab Yunus
'hbase' does not seem to have --config/-config parameter. Regards, Shahab On Mon, Jun 24, 2013 at 8:39 AM, rajeshbabu chintaguntla rajeshbabu.chintagun...@huawei.com wrote: Can you try copying hbase-site.xml to other folder and change hbase.zookeeper.quorum to remote server and then use

RE: How to specify the hbase.zookeeper.quorum on command line invoking hbase shell

2013-06-24 Thread rajeshbabu chintaguntla
--config conf dir arguments of a command will be parsed by hbase-config.sh which is imported in hbase script. It will work. {code} . $bin/hbase-config.sh {code} Thanks, Rajeshbabu From: Shahab Yunus [shahab.yu...@gmail.com] Sent: Monday, June 24, 2013

Filtering help

2013-06-24 Thread Aji Janis
Hello, In my Hbase table I have four families. Not all rows have data for all the families. And not all rows have the same qualifiers either. So imagine this scenario: rowid |f1 |f2 |f3| --- row1 |q1, q2 |q4 |q6 | row2 |q1, q3 |q4,q5 |

Is hbase 0.94.8 compatible with hadoop (hdfs) 0.23.8 ?

2013-06-24 Thread Arnab Mukherjee
Thanks. -- Arnab Mukherjee 510-304-0881

Re: Replication not suited for intensive write applications?

2013-06-24 Thread Jean-Daniel Cryans
Given that the region server writes to a single WAL at a time, doing it with multiple threads might be hard. You also have to manage the correct position up in ZK. It might be easier with multiple WALs. In any case, Inserting at such date might not be doable over long periods of time. How long

RE: Scan performance

2013-06-24 Thread Tony Dean
Hi James, I do plan on looking more closely at Phoenix for SQL access to HBase. Thanks. -Original Message- From: James Taylor [mailto:jtay...@salesforce.com] Sent: Saturday, June 22, 2013 1:18 PM To: user@hbase.apache.org Subject: Re: Scan performance Hi Tony, Have you had a look at

RE: Scan performance

2013-06-24 Thread Tony Dean
Lars, I'm waiting for some time to exchange out hbase jars in cluster (that support FuzzyRow filter) in order to try out. In the meantime, I'm wondering why RowFilter regex is not more helpful. I'm guessing that FuzzyRow filter helps in disk io while Row filter just filters after the disk io

Re: Scan performance

2013-06-24 Thread lars hofhansl
RowFilter can help. It depends on the setup. RowFilter skip all column of the row when the row key does not match. That will help with IO *if* your rows are larger than the HFile block size (64k by default). Otherwise it still needs to touch each block. An HTable does some priming when it is

Re: Writing unit tests against HBase

2013-06-24 Thread Adam Phelps
On 6/18/13 4:22 PM, Stack wrote: On Tue, Jun 18, 2013 at 4:17 PM, Varun Sharma va...@pinterest.com wrote: Hi, If I wanted to write to write a unit test against HTable/HBase, is there an already available utility to that for unit testing my application logic. I don't want to write code

Re: Writing unit tests against HBase

2013-06-24 Thread Dhaval Shah
Why don't you spin up a mini cluster for your tests (there is a MiniHBaseCluster which brings up an in-memory cluster for testing and you can tear it down at the end of your test)? The benefit you get is that you no longer need to mock HBase responses and you will be talking to an actual

Re: Writing unit tests against HBase

2013-06-24 Thread Adam Phelps
What I'm currently looking for is a method of adding quick unit tests (ie preferably run time of a few seconds) to test some algorithms that read hbase data and perform some operations on it. Mocking seems a much better way to handle this, though I'm open to other suggestions. I'll try out

Re: Writing unit tests against HBase

2013-06-24 Thread Christophe Taton
Hey Adam, Here is how to use fake-hbase from Java: final Configuration conf = new Configuration(); final FakeHBase fakeHBase = new FakeHBase(); final HBaseAdmin admin = fakeHBase.getAdminFactory().create(conf); final HTableDescriptor desc = new HTableDescriptor(table); desc.addFamily(new

Re: Writing unit tests against HBase

2013-06-24 Thread Dhaval Shah
Yup I hear ya. MiniHBaseCluster adds an extra minute to the tests which kind of sucks. It gives me peace of mind though   Regards, Dhaval - Original Message - From: Adam Phelps a...@opendns.com To: user@hbase.apache.org Cc: Sent: Monday, 24 June 2013 6:00 PM Subject: Re: Writing unit

removing ttl

2013-06-24 Thread Kireet
I need to remove the TTL setting from an existing HBase table and remove the TTL from all existing rows. I think this is the proper command for removing the TTL setting: alter 't', {NAME = 'cf', TTL = '2147483647'} After doing this, do I need to rewrite all the existing data to remove the

Re: removing ttl

2013-06-24 Thread Jean-Marc Spaggiari
Have you tried something like alter 't1', METHOD = 'table_att_unset', NAME = 'TTL' ? And I don't think you will need to rewrite anything. JM 2013/6/24 Kireet kir...@feedly.com: I need to remove the TTL setting from an existing HBase table and remove the TTL from all existing rows. I think

Re: removing ttl

2013-06-24 Thread Jean-Daniel Cryans
TTL is enforced when compactions are running so there's no need to rewrite the data. The alter is sufficient. J-D On Mon, Jun 24, 2013 at 4:15 PM, Kireet kir...@feedly.com wrote: I need to remove the TTL setting from an existing HBase table and remove the TTL from all existing rows. I think

Re: Is hbase 0.94.8 compatible with hadoop (hdfs) 0.23.8 ?

2013-06-24 Thread Azuryy Yu
its compaiible with hadoop 2.x, I dont think HBase tested under hdfs 0.23? --Send from my Sony mobile. On Jun 25, 2013 1:49 AM, Arnab Mukherjee for.ar...@gmail.com wrote: Thanks. -- Arnab Mukherjee 510-304-0881

RE: Scan performance

2013-06-24 Thread Tony Dean
My row key is relatively small so that's why RowFilter is not making any difference. Thanks. I am scanning a single table/cf for a particular qualifier. The scan has a start row and stop row with a filter. In my test case, only 1 result is returned. I put this call in a 10 iteration loop

Re: Writing unit tests against HBase

2013-06-24 Thread Adam Phelps
Thansk, that set me on the right path. For anyone else reading along, I also needed to add: dependency groupIdorg.scala-lang/groupId artifactIdscala-library/artifactId version2.9.0/version scopetest/scope /dependency