Re: Co-Processor in scanning the HBase's Table

2013-02-22 Thread Farrokh Shahriari
@Anoop : Here is my situation: I have 2 columnFamilies(cf1,cf2),5-50 qualifiers(they are variable),first I should check the value of qualifiers in cf2,then if they equal to some values I'm going to get the values of cf1 by that rowkey. @Amita: thank you.I will check that too. Farrokh On Thu, Feb

Re: Co-Processor in scanning the HBase's Table

2013-02-22 Thread Ted
Hbase-5416 should help your use case. Cf2 would be the essential column family. This feature is available in 0.94.5 Cheers On Feb 22, 2013, at 5:55 AM, Farrokh Shahriari mohandes.zebeleh...@gmail.com wrote: @Anoop : Here is my situation: I have 2 columnFamilies(cf1,cf2),5-50

Re: Co-Processor in scanning the HBase's Table

2013-02-22 Thread Anoop John
yes what Ted mentioned, u can try out.. -Anoop- On Fri, Feb 22, 2013 at 7:46 PM, Ted yuzhih...@gmail.com wrote: Hbase-5416 should help your use case. Cf2 would be the essential column family. This feature is available in 0.94.5 Cheers On Feb 22, 2013, at 5:55 AM, Farrokh Shahriari

attributes - basic question

2013-02-22 Thread Toby Lazar
What is the purpose of the getAttribute and setAttribute methods for classes implementing OperationWithAttributes? I don't see much documentation about them, haven't seen much discussion on this list about them, and am wondering what people use them for. Or, are they mostly just used internally?

does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread kaveh minooie
Hi everyone, I had a quick question. do I need to be running datanode on the server that is running hbase master or datanode should be running only on servers that are running as regionserver? thanks, -- Kaveh Minooie

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Mohammad Tariq
Hello Kaven, It's not all necessary to run HM on server running DN. Actually it's not advisable to do so, as HM has high memory requirements and running other daemons on the same machine may lead you to problems. The basic cluster setup, which people normally use looks somewhat like this :

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Jean-Marc Spaggiari
Just to add to Mohammad's advices, you should avoid to run ZK on the same servers as you are running HBase. Reason is, if you are running in long GCs, ZK might miss the heartbeats and thinks servers are down. So safer to run same separately if you can. JM 2013/2/22 Mohammad Tariq

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Mohammad Tariq
Yeah..Good point JM. Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com On Sat, Feb 23, 2013 at 1:19 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Just to add to Mohammad's advices, you should avoid to run ZK on the same servers as you are running HBase. Reason

issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread kaveh minooie
Hi everyone I am having this problem for couple of days now and would appreciate any idea or suggestion that any one might have. I am using nutch 2.x with hbase. due to nutch requirement I need to use older version of hbase (I am using 0.90.6 over hadoop 1.1.1 with 10 nodes with zookeepr

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread alxsss
Hi, Do you see any errors in HMaster and Regionserver logs? Alex. -Original Message- From: kaveh minooie ka...@plutoz.com To: user user@hbase.apache.org; user u...@nutch.apache.org; user u...@zookeeper.apache.org Sent: Fri, Feb 22, 2013 12:31 pm Subject: issue with

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread alxsss
Turn off iptables and see if it resolves errors. You need to open all ports that hbase and zookeeper need to connect each node. Alex. -Original Message- From: kaveh minooie ka...@plutoz.com To: user user@hbase.apache.org Sent: Fri, Feb 22, 2013 12:56 pm Subject: Re: issue with

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Harsh J
While this general guideline of not requiring a DN on HM is correct, I just wanted to point out that the HM is hardly high on memory usage/requirements (as possibly stated below), as all it does is maintain the cluster and not serve nor hold any client-useful data or frequent requests. On Sat,

Re: attributes - basic question

2013-02-22 Thread Harsh J
The attributes are serialized along with the base operation request. There's perhaps no immediate client-side usage of this, it is used by the Mutation class to set a cluster ID in HBase's Replication context:

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Harish Krishnan
So in case of a stand alone set up, how come we are not experiencing those issues? Is it because of the limited data set that we are operating on? Thanks Regards, Harish.T.K On Fri, Feb 22, 2013 at 1:19 PM, Harsh J ha...@cloudera.com wrote: While this general guideline of not requiring a DN

Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Ted Yu
Hi, Sergey has 51 issues under his name: * https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20assignee%20%3D%20sershe%20AND%20status%20in%20(Resolved%2C%20Closed) * * * * He was the driving force in finishing HBASE-5416 Improve performance of scans with some kind of filters.

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread kaveh
all the ports are open. there is no iptable. -Original Message- From: alx...@aim.com Sent: Friday, February 22, 2013 1:02pm To: user@hbase.apache.org Subject: Re: issue with nutch-gora+hbase+zookeeper Turn off iptables and see if it resolves errors. You need to open all ports that hbase

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Andrew Purtell
Congratulations Sergey! On Fri, Feb 22, 2013 at 1:39 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, Sergey has 51 issues under his name: * https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20assignee%20%3D%20sershe%20AND%20status%20in%20(Resolved%2C%20Closed) * * * * He

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Jean-Marc Spaggiari
Hi Harish, HBase still run ZooKeeper even in standalone mode: http://hbase.apache.org/book/standalone_dist.html So you still can face same kind of issues. But since it's not using Hadoop, there is more memory available for you HBase, which might reduce GCs issues. JM 2013/2/22 Harish Krishnan

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Gary Helmling
Congrats, Sergey! Great work! On Fri, Feb 22, 2013 at 2:10 PM, Enis Söztutar enis@gmail.com wrote: Congrats. Well deserved. On Fri, Feb 22, 2013 at 1:57 PM, Andrew Purtell apurt...@apache.org wrote: Congratulations Sergey! On Fri, Feb 22, 2013 at 1:39 PM, Ted Yu

Re: attributes - basic question

2013-02-22 Thread Toby Lazar
Your last point was exactly what I was looking for. I am thinking about using attributes along with coprocessors to impose some application-level authorization constraints. For example, in a get, I can pass username and credential attributes and have the coprocessor filter results based on some

Question about object parameters to mappers

2013-02-22 Thread Gaetan Deputier
Hi Hbase users, (Using HBase 0.92.1 from Cloudera CDH4.1.1) I am currently writing a TableMapper which needs an object X defined outside my TableMapper class. I was wondering how can I pass this object X to my TableMapper ? I want to be sure that once the jobs are deployed on our cluster, X

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Elliott Clark
Welcome and Congrats. On Fri, Feb 22, 2013 at 2:15 PM, Gary Helmling ghelml...@gmail.com wrote: Congrats, Sergey! Great work! On Fri, Feb 22, 2013 at 2:10 PM, Enis Söztutar enis@gmail.com wrote: Congrats. Well deserved. On Fri, Feb 22, 2013 at 1:57 PM, Andrew Purtell

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread lars hofhansl
Congratulations, Sergey and welcome :) From: Ted Yu yuzhih...@gmail.com To: d...@hbase.apache.org; user@hbase.apache.org Sent: Friday, February 22, 2013 1:39 PM Subject: Please welcome our newest committer: Sergey Shelukhin Hi, Sergey has 51 issues under his

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Jesse Yates
Great job Sergey! Welcome and keep up the good work. --- Jesse Yates @jesse_yates jyates.github.com On Fri, Feb 22, 2013 at 4:20 PM, lars hofhansl la...@apache.org wrote: Congratulations, Sergey and welcome :) From: Ted Yu

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Sergey Shelukhin
Thanks guys! On Fri, Feb 22, 2013 at 4:24 PM, Jesse Yates jesse.k.ya...@gmail.comwrote: Great job Sergey! Welcome and keep up the good work. --- Jesse Yates @jesse_yates jyates.github.com On Fri, Feb 22, 2013 at 4:20 PM, lars hofhansl la...@apache.org wrote:

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Jonathan Hsieh
Congrats Sergey, looking forware to more contributions. On Fri, Feb 22, 2013 at 1:39 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, Sergey has 51 issues under his name: *

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Marcos Ortiz
Congratulations, Sergey. On 02/22/2013 04:39 PM, Ted Yu wrote: Hi, Sergey has 51 issues under his name: * https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20assignee%20%3D%20sershe%20AND%20status%20in%20(Resolved%2C%20Closed) * * * * He was the driving force in finishing

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread ramkrishna vasudevan
Congrats Sergey. Regards Ram On Sat, Feb 23, 2013 at 7:17 AM, Marcos Ortiz mlor...@uci.cu wrote: Congratulations, Sergey. On 02/22/2013 04:39 PM, Ted Yu wrote: Hi, Sergey has 51 issues under his name: * https://issues.apache.org/**jira/issues/?jql=project%20%**

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread kaveh minooie
In case anyone was wondering, the issue was resolved by copying the zoo.cfg in the hadoop conf directory (on classpath really) on the entire cluster. thanks On 02/22/2013 12:31 PM, kaveh minooie wrote: Hi everyone I am having this problem for couple of days now and would appreciate any idea

RE: attributes - basic question

2013-02-22 Thread Anoop Sam John
We have used setAttribute() along with Scan which we are using in the CP. Ya it will work fine. Pls try with ur use case and if finding any issue pls report -Anoop- From: Toby Lazar [tla...@gmail.com] Sent: Saturday, February 23, 2013 4:07 AM To:

Re: attributes - basic question

2013-02-22 Thread James Taylor
Same with us on Phoenix - we use the setAttribute on the client side and the getAttribute on the server side to pickup state on the Scan being executed. Works great. One thing to keep in mind, though: for a region observer coprocessor, the state you set on the client side will be sent to each

Re: Problem In Understanding Compaction Process

2013-02-22 Thread Anty
Thanks Sergey In my use case. I want to directly analyze the underlying HFiles, So i can't tolerance duplicate data. Can you give me some pointers about how to make this procedure atomic? On Thu, Feb 21, 2013 at 6:07 AM, Sergey Shelukhin ser...@hortonworks.comwrote: There should be no

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread anil gupta
On Feb 22, 2013 11:50 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Just to add to Mohammad's advices, you should avoid to run ZK on the same servers as you are running HBase. Reason is, if you are running in long GCs, ZK might miss the heartbeats and thinks servers are down. So