Re: querying hbase

2013-06-01 Thread Michel Segel
Is there a benefit to restarting a regionserver in an OSGi container versus restarting a Java process? Was that rhetorical? Absolutely. Think of a production environment where you are using HBase to serve data in real time. Sent from a remote device. Please excuse any typos... Mike Segel

Re: querying hbase

2013-06-01 Thread Andrew Purtell
Isn't the time to restart and the steps necessary more or less the same? Or will the objects that hold the in memory state survive across the reload? Will they still share a classloader (maintain equality tests)? What if the implementation / bundle version changes? We are taking about an upgrade

Re: What is the best hbase table schema for following json data?

2013-06-01 Thread AnilKumar B
Thanks Ted Michael. On Fri, May 31, 2013 at 12:39 AM, Michael Segel michael_se...@hotmail.comwrote: But you should be able to write a custom column filter that handles JSON records within a cell. On May 30, 2013, at 11:48 AM, Ted Yu yuzhih...@gmail.com wrote: bq. Still these

HTable streaming

2013-06-01 Thread Simon Majou
Hi, I don't see any methods returning streams in the Java client HTable : http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html That means we need to use the thrift client to get streams ? Simon

HTable and streaming

2013-06-01 Thread Simon Majou
Hi, I don't see any methods returning streams in the Java client HTable : http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html That means we need to use the thrift client to get streams ? Simon

Re: HTable and streaming

2013-06-01 Thread Ted Yu
I assume you want to scan the table: http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[]) Can you point out which thrift client methods provide streaming capability ? Thanks On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou si...@majou.org wrote: Hi, I

Re: HTable and streaming

2013-06-01 Thread Simon Majou
No I don't want to scan a table, I want a stream of one result. In case for example of big records. With thrift it is up to the client to deal with the response, so in theory you can build a client which returns streams. But when I look at the current implementation of thrift for node for

Re: querying hbase

2013-06-01 Thread Michael Segel
Well, What happens when you restart the RS? Suppose I'm running a scan on a completely different table and you restart the RS? What happens to me? I havent thought through the whole problem, but you need to put each table's CP in to its own sandbox. (There's more to it and would require

Re: querying hbase

2013-06-01 Thread James Taylor
These approaches all sound somewhat brittle and unlikely to be relied on for a production system (more here: https://issues.apache.org/jira/browse/HBASE-8607). Sounds like a rolling restart is the best option in the near/medium term. Our pain points are more around how to get to the point

Re: HBASE install shell script on cluster

2013-06-01 Thread Stack
On Fri, May 31, 2013 at 1:13 PM, Asaf Mesika asaf.mes...@gmail.com wrote: We have developed some custom scripts on top of fabric ( http://docs.fabfile.org/en/1.6/). I've asked the developer on our team to see if can share some of it to the community. It's mainly used for

Re: HTable and streaming

2013-06-01 Thread Stack
Yeah, no streaming API in our current client (nor does our thrift client give you a streaming API). St.Ack On Sat, Jun 1, 2013 at 8:21 AM, Simon Majou si...@majou.org wrote: No I don't want to scan a table, I want a stream of one result. In case for example of big records. With thrift it is

Multiple different failures

2013-06-01 Thread Jean-Marc Spaggiari
Hi, Today I faced a power outage. 4 computers stayed up. The 3 ZK servers, the Master, the NN and 2 DN/RS. They was on UPS. While everything was going back up... Guess what... I faced a 2nd one! After bringing HBase up, about 97% of my data was missing. (19M rows in my main table) I ran HBCK

Re: Multiple different failures

2013-06-01 Thread Varun Sharma
Are you saying 97 % data was lost or was it offlined until the region servers came back up ? Varun On Sat, Jun 1, 2013 at 6:31 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Hi, Today I faced a power outage. 4 computers stayed up. The 3 ZK servers, the Master, the NN and 2 DN/RS.

Re: querying hbase

2013-06-01 Thread Michael Segel
Sure, but that wont change the fact that Coprocessors should go under a massive rewrite. You're hitting a problem that Sybase faced while Informix (datablades) didn't when it came to running end user code within the engine. But I'm dating myself... On Jun 1, 2013, at 3:20 PM, James Taylor

Weird Replication exception

2013-06-01 Thread Asaf Mesika
Hi, I have a weird error in a cluster I'm checking Replication with. I have two clusters set up, each on its own DC (different continents). Each has 1 master, and 3 RS. I've done all required setup, started replication and pushed in some data into the master. I had an issue where the slave