Hbase schema design question for time based data

Hi, I am trying design schema for some data to be moved from HDFS into HBase for real-time access. Questions - 1. Is the use of new API for bulk upload recommended over old API? If yes, is the new API stable and is there sample executable code around ? 2. The data is over time. I need to be ab

Re: help for designing a hbase

Hey Johannes, We're using hbase for something similar at Mendeley. We store all our raw http logs in hdfs then use pig scripts to process these into hits per day for each of our articles. We store this as follows into a hbase table :- articleId_date => counts:total:30, counts:unqiue:10, ... So t

Re: experiences with hbase-2492

I did not try the tcp_tw_reuse, because I am a bit fuzzy on what that exactly does. That is, what is the difference between recycling and reusing? The man page for tcp just mentions that tcp_tw_reuse allows to reuse TIME_WAIT sockets if it is safe from the protocol viewpoint and that it should n

Re: experiences with hbase-2492

Friso, You may be knowing this already, but please bear in mind there is a potential risk of packets from previous connections that were in flight reach the new connections (that's the reason for the TIME_WAIT state in TCP).. And that may lead to unexpected behaviour.. Vidhya On 6/15/10 9:1

Re: experiences with hbase-2492

Might be worth trying tcp_tw_reuse before turning on tw_recycle - as I understand it, the former is a lot safer than the latter. Can't wait for HDFS-941 some day :) -Todd On Tue, Jun 15, 2010 at 9:10 AM, Jean-Daniel Cryans wrote: > Friso, > > This is very interesting, and nobody answered probab

Re: experiences with hbase-2492

Doh! I meant tcp_tw_reuse. Sorry, paste-o. - Andy --- On Tue, 6/15/10, Andrew Purtell wrote: > From: Andrew Purtell > Subject: Re: experiences with hbase-2492 > To: user@hbase.apache.org > Cc: fvanvollenho...@xebia.com > Date: Tuesday, June 15, 2010, 9:59 AM > tcp_tw_recycle did not do what

Re: experiences with hbase-2492

tcp_tw_recycle did not do what you needed? - Andy > On Mon, Jun 14, 2010 at 11:40 PM, Friso van Vollenhoven wrote: > Hi all, > > Since I got no replies to my previous message (see > below), I went ahead and set the tcp_tw_recycle to true. > This worked like a charm. The number of sockets in TI

Re: experiences with hbase-2492

Friso, This is very interesting, and nobody answered probably because no one tried tcp_tw_recycle. I personally didn't even know about that config until a few minutes ago ;) So from the varnish mailing list, it seems that machines behind firewalls or NAT won't play well with that config, but I do

Re: Error opening job jar

This isn't a HBase question, this is for mapreduce-u...@hadoop.apache.org J-D On Tue, Jun 15, 2010 at 8:21 AM, yshintre1982 wrote: > > i am running wordcount example on linux vmware on hadoop. > i get the following exception > > Exception in thread "main" java.io.IOException: Error opening job j

Error opening job jar

i am running wordcount example on linux vmware on hadoop. i get the following exception Exception in thread "main" java.io.IOException: Error opening job jar: /usr/yogesh/wordcount.jar at org.apache.hadoop.util.RunJar.main(RunJar.java:90) Caused by: java.util.zip.ZipException: error in op

Re: I want to know whether HBase single row written is row lock default

On Tue, Jun 15, 2010 at 4:52 AM, zhengbing li wrote: > hi, >    I want to know whether the following code is row locked default > Yes. >   Put put = new Put(rowID) >   hTable.put(put); > You are missing put.add... in the above, adding actual values to insert. St.Ack

I want to know whether HBase single row written is row lock default

hi, I want to know whether the following code is row locked default Put put = new Put(rowID) hTable.put(put);