Re: How to join 2 tables using hadoop?

2013-07-19 Thread Pavan Sudheendra
If i have the value of a row in JSON format, would pig we able to parse it and join the fields as per my needs? On Fri, Jul 19, 2013 at 10:00 PM, Shahab Yunus wrote: > You can also look into Pig, if you already haven't. It supports various > kinds of joins and is simpler than writing your own M/R

Re: Batch.call and dead Region Server.

2013-07-19 Thread Ted Yu
coprocessorExec() calls connection.processExecs(). In HConnectionManager, we have this code (0.94) : Future future = pool.submit( new Callable() { ... futures.put(r, future); } for (Map.Entry> e : futures.entrySet()) { try { e.getValue().g

Batch.call and dead Region Server.

2013-07-19 Thread Kim Chew
Suppose I have a table "foo" which spans four RS, RS1 RS2 RS3 RS4 I have deployed my coprocessor to table "foo" and I invoke my coprocessor using Batch.call and pass this object to HTable's "coprocessorExec". Some how RS2 has died, it seems like I am at the mercy of the RPC after

Re: Encountered problems when prefetch META table

2013-07-19 Thread Zhenzhen Yan
Thank you, Ted! On Fri, Jul 19, 2013 at 4:17 PM, Ted Yu wrote: > How did you create the table ? > > If using shell, you can get help by typing: > help 'create' > > You can also create table programmatically. > See http://hbase.apache.org/book.html#rowkey.regionsplits > > Cheers > > On Fri, Jul

Re: Encountered problems when prefetch META table

2013-07-19 Thread Zhenzhen Yan
Hi, Ted, in the hbase shell I typed 'list', the 'HBaseSamples' table is not there. Following is the java file. package hbase; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.client.HTable;

Re: Encountered problems when prefetch META table

2013-07-19 Thread Ted Yu
How did you create the table ? If using shell, you can get help by typing: help 'create' You can also create table programmatically. See http://hbase.apache.org/book.html#rowkey.regionsplits Cheers On Fri, Jul 19, 2013 at 3:05 PM, Zhenzhen Yan wrote: > Hi, Ted, in the hbase shell I typed 'list

Re: Encountered problems when prefetch META table

2013-07-19 Thread Ted Yu
Exception in thread "main" org.apache.hadoop.hbase.TableNotFoundException: HBaseSamples Were you able to access HBaseSamples in hbase shell ? Did you see any exception in master log w.r.t. the creation of HBaseSamples ? Cheers On Fri, Jul 19, 2013 at 2:29 PM, Zhenzhen Yan wrote: > Hello, I'm a

Encountered problems when prefetch META table

2013-07-19 Thread Zhenzhen Yan
Hello, I'm a newbie to Hadoop HBase. I downloaded and installed Hadoop Traning Ubundu Image (Cloudera Distribution for Hadoop). When I tried to create a table and create put instances using java API. I got the following error. Could anybody help me out? Thanks! 13/07/19 16:46:45 WARN client.HConne

Re: Expanding a ZK quorum with replication

2013-07-19 Thread Jeremy Carroll
You should be able to do a rolling restart from 3 => 5 without losing quorum. As long as the replication peer can connect to any ZooKeeper node which is in quorum, it should be OK. I agree that you would want to add additional peers to the replication agreement. I have never done this personally.

Accessing HBase using java native api from weblogic

2013-07-19 Thread rjoshi
Hello, I have a java native api to access HBase and it works fine as a standalone program. But when I deploy same jar file as part of JAX-RS service, it's not able to get zookeeper. I have given below both logs when accessed (doesn't work) from weblogic and accessed (works fine) as a standalone

Re: Accessing HBase using java native api from weblogic

2013-07-19 Thread Stack
You see anything in zk logs at the time of your connectionloss? 192.168.56.101:2181 is where a zk ensemble member resides (and is up and running)? St.Ack On Fri, Jul 19, 2013 at 11:33 AM, rjoshi wrote: > Hello, > > I have a java native api to access HBase and it works fine as a > standalone

M/R to HBase - affinity to Region Server

2013-07-19 Thread Asaf Mesika
Hi, Is there a way in MRv2 to direct a reducer to run in the RS it will write its Puts, thus achieve write locality? Thanks! Asaf

Expanding a ZK quorum with replication

2013-07-19 Thread Ian Friedman
Hey guys, looking for some advice on this issue: We'd like to move our Hbase cluster from 3 to 5 Zookeeper nodes. Normally this would be easily handled by a rolling restart. However we also have replication between our 2 production clusters enabled, which uses the ZK quorum of either side as t

Re: How to join 2 tables using hadoop?

2013-07-19 Thread Shahab Yunus
You can also look into Pig, if you already haven't. It supports various kinds of joins and is simpler than writing your own M/R job (assuming that you don't have complex or custom requirements.) Regards, Shahab On Fri, Jul 19, 2013 at 12:24 AM, Pavan Sudheendra wrote: > Hi, > > I know that HBas

RE: Hbase: Is possible to filter by integer value if the value is saved as string?

2013-07-19 Thread Frank Luo
OK, let's say it is stored as an integer, how to compare then? As Jean-Marc pointed out, negative numbers is greater than positive ones using ByteArrayComparable. -Original Message- From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org] Sent: Friday, July 19, 2013 5:08 AM To: user@h

Re: Hbase: Is possible to filter by integer value if the value is saved as string?

2013-07-19 Thread Kevin
The only way I was able to do negative numbers was to create a new WritableComparable that took signed-ness into account, but that involved deploying the class to the cluster. On Fri, Jul 19, 2013 at 10:44 AM, Frank Luo wrote: > OK, let's say it is stored as an integer, how to compare then? > >

Re: Hbase: Is possible to filter by integer value if the value is saved as string?

2013-07-19 Thread Jean-Marc Spaggiari
I agree with Anil... Store it as an integer, not as as String. That will help for the comparison (be carreful with negative values comparisons.). Worst case, run a MR job to convert them. 2013/7/19 anil gupta > Only way to achieve this is to write your own ByteArrayComparable. > BinaryComparato

Re: How to join 2 tables using hadoop?

2013-07-19 Thread yonghu
You can write one MR job to finish this. First read two tables at Map function, the output key will be the reference key for one table and primary key for the other table. At the Reduce function, you can "join" the tuples which contain the same key. Please note this is a very naive approach, for m

Re: How to join 2 tables using hadoop?

2013-07-19 Thread Nitin Pawar
Try hive with hbase storage handler On Fri, Jul 19, 2013 at 9:54 AM, Pavan Sudheendra wrote: > Hi, > > I know that HBase by default doesn't support table joins like RDBMS.. > But anyway, I have a table who value contains a json with a particular > ID in it.. > This id references another table wh