Re: Bulk loading a CSV file into HBase

2012-03-08 Thread anil gupta
Hi Stack, I decompiled the ImportTsv class and added some sysout statements in main() to figure out the problem. Please find the modified class here: http://pastebin.com/sKQcMXe4 With help of Keshav, i got to know that csv import works fine when i provide -Dimporttsv.separator=, as first

Re: Problem in using HBase Bulk Load

2012-03-08 Thread Jonathan Hsieh
Hi Anil, Looks like the zookeeper information isn't in your classpath when using the hadoop command line. You may want to either add /etc/zookeeper/conf to you class path (possibly by prefacing the import command with HADOOP_CLASSPATH=/etc/zookeeper/conf), or add xml style zookeeper config

Solr+Hbase

2012-03-08 Thread Mohammad Tariq
Hello list, We are planning to index our data stored in HBase using Solr.As we are totally new to Solr, we would like to have some comments from someone who is already doing it..While looking over the internet we came across Liliy.Is there any other other option, or Lily is the only preferable

Re: Problem in using HBase Bulk Load

2012-03-08 Thread anil gupta
Hi Jonathan, Thanks for your reply. Fortunaltely, i fixed the problem by removing 127.0.0.1 from etc/hosts file and added the conf directory in Hadoop_Classpath. Thanks, Anil Gupta On Thu, Mar 8, 2012 at 12:18 AM, Jonathan Hsieh j...@cloudera.com wrote: Hi Anil, Looks like the zookeeper

Re: Problem in using HBase Bulk Load

2012-03-08 Thread anil gupta
Please find the corrected message below. Sorry for typos. On Thu, Mar 8, 2012 at 12:34 AM, anil gupta anilg...@buffalo.edu wrote: Hi Jonathan, Thanks for your reply. Fortunately, i had fixed the problem around 2 days ago by removing 127.0.0.1 entries from /etc/hosts file and adding the conf

Re: Solr+Hbase

2012-03-08 Thread Damien HARDY
Le 08/03/2012 09:18, Mohammad Tariq a écrit : Hello list, We are planning to index our data stored in HBase using Solr.As we are totally new to Solr, we would like to have some comments from someone who is already doing it..While looking over the internet we came across Liliy.Is there any

Re: Hbase HregionServer on slave

2012-03-08 Thread Gopal
On 3/7/2012 11:56 PM, Stack wrote: In essence, there is no filesystem for hbase to go against, not until Thank you Michael, I saw your you tube video. You Rock. On a different note, isnt it Best practice to not have a DataNode running along with a namenode on a production set up? If that is

Re: Solr+Hbase

2012-03-08 Thread Mohammad Tariq
Hello Damien, Thanks a lot for the useful info.Elasticsearch looks appealing.I'll look at both the options. Regards,     Mohammad Tariq On Thu, Mar 8, 2012 at 2:52 PM, Damien HARDY dha...@figarocms.fr wrote: Le 08/03/2012 09:18, Mohammad Tariq a écrit : Hello list,    We are planning

Re: Hadoop User Group Cologne

2012-03-08 Thread alo alt
HI, we've setup few days ago a German UG: http://mapredit.blogspot.com/2012/03/hadoop-ug-germany.html Deutsch / german: Wir haben eine UHG gegruendet, erstmal Gruppen in XING / LinkedIn und eine Website, die aber wirklich recht neu ist :) Wenn Du mitmachen willst, melden! Danke und bis bald,

Fwd: org.apache.hadoop.conf.Configuration - error parsing conf file

2012-03-08 Thread Something Something
-- Forwarded message -- From: Something Something mailinglist...@gmail.com Date: Thu, Mar 8, 2012 at 8:43 AM Subject: Re: org.apache.hadoop.conf.Configuration - error parsing conf file To: u...@pig.apache.org, manishbh...@rocketmail.com *Stack*: Explicit message would be one

Re: HTable.getEndKeys() returning empty results

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 8:25 AM, de Souza Medeiros Andre andre.medei...@aalto.fi wrote: Hi all, Has someone had problems with HTable.getEndKeys() method? In my application it returns an array of byte[], but the first (and only) byte[] is empty, so essentially it's telling me that the

Re: org.apache.hadoop.conf.Configuration - error parsing conf file

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 8:45 AM, Something Something mailinglist...@gmail.com wrote: Explicit message would be one that would tell me which configuration file it is ;) Agreed. As for JVM: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03-384-10M3425) Java

Re: Solr+Hbase

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 12:18 AM, Mohammad Tariq donta...@gmail.com wrote: Hello list,   We are planning to index our data stored in HBase using Solr.As we are totally new to Solr, we would like to have some comments from someone who is already doing it..While looking over the internet we

Re: Hbase HregionServer on slave

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 1:30 AM, Gopal absoft...@gmail.com wrote: On a different note, isnt it Best practice to not have a DataNode running along with a namenode on a production set up? This is how its usually done, yes. If that is the norm rather than an exception then I think i have a

Re: Bulk loading a CSV file into HBase

2012-03-08 Thread Stack
On Wed, Mar 7, 2012 at 11:59 PM, anil gupta anilg...@buffalo.edu wrote: I tried to analyze the problem and as per my analysis there is a problem with String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs(); on line#102. Let me know you views. So, its just where you put

Re: Coprocessor execution with bulk data

2012-03-08 Thread Stack
On Wed, Mar 7, 2012 at 10:59 PM, raghavendhra rahul raghavendhrara...@gmail.com wrote: 2012-03-08 12:03:09,475 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call execCoprocessor([B@50cb21, getProjection(), rpc version=1, client version=0, methodsFingerPrint=0), rpc version=1,

Re: HTable.getEndKeys() returning empty results

2012-03-08 Thread Doug Meil
I believe that's covered here... http://hbase.apache.org/book.html#arch.catalog Notes on HRegionInfo: the empty key is used to denote table start and table end. A region with an empty start key is the first region in a table. If region has both an empty start and an empty end key, its

Re: Coprocessor execution with bulk data

2012-03-08 Thread Andrew Purtell
You can increase the RPC timeout period to keep this from happening. But maybe it makes sense to give our RPC a keepalive option for calls that may run for a long time (like execCoprocessor)? Or fold this into a broader rework into a proper async RPC model.   Best regards,     - Andy

Re: Install hbase on Ubuntu 11.10

2012-03-08 Thread Mahdi Negahi
Dear James thanks for ur advice this line sudo apt-get install hbase show this error E: Unable to locate package hbase James

Re: HTable.getEndKeys() returning empty results

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 9:27 AM, Doug Meil doug.m...@explorysmedical.com wrote: I believe that's covered here... http://hbase.apache.org/book.html#arch.catalog Notes on HRegionInfo:  the empty key is used to denote table start and table end.  A region with an empty start key       is the

Re: Bulk loading a CSV file into HBase

2012-03-08 Thread anil gupta
Hi Stack, Yes, the separator argument is sensitive to position in the command. Currently, it needs to be specified just after program name. The same is not mentioned in the docs. I have got two suggestion for fixing this so that other don't run into same problem: 1. Update the HBase bulk load

Re: Bulk loading a CSV file into HBase

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 11:14 AM, anil gupta anilg...@buffalo.edu wrote: 1. Update the HBase bulk load documentation and specify that separator argument should be next to program name. This would help. 2. Fix the problem in the code itself by handling the separator argument explicitly.

RE: HTable.getEndKeys() returning empty results

2012-03-08 Thread de Souza Medeiros Andre
Thanks for the answer. This is quite confusing though, what is the point of having a method to retrieve the last key if we know that anyway it will be an empty byte array? Or does this happen only with one-region tables? Any way, is there some way of getting the last _data_ row of a table?

Re: HTable.getEndKeys() returning empty results

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 11:50 AM, de Souza Medeiros Andre andre.medei...@aalto.fi wrote: Or does this happen only with one-region tables? Yes Any way, is there some way of getting the last _data_ row of a table? Other than scanning the whole thing... No. St.Ack

Re: Bulk loading a CSV file into HBase

2012-03-08 Thread Shrijeet Paliwal
GenericOptionsParser stops parsing the arguments as soon as first non option is specified (refer : http://commons.apache.org/cli/api-1.2/org/apache/commons/cli/Parser.html#parse(org.apache.commons.cli.Options, java.lang.String[], boolean)) So in this cases as soon parses sees the table name arg ,

Re: Coprocessor execution with bulk data

2012-03-08 Thread lars hofhansl
+1 on keepalive. It's a shame (especially for long running server code) to do all the work, just to find out at the end that the client has given up. Or maybe there should be a way to cancel an operation if the clients decides it does not want to wait any longer (PostgreSQL does that for

Re: Coprocessor execution with bulk data

2012-03-08 Thread Andrew Purtell
I opened https://issues.apache.org/jira/browse/HBASE-5543   Best regards,     - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) - Original Message - From: lars hofhansl lhofha...@yahoo.com To: user@hbase.apache.org

Re: HTable.getEndKeys() returning empty results

2012-03-08 Thread Michael Drzal
On Thu, Mar 8, 2012 at 2:52 PM, Stack st...@duboce.net wrote: On Thu, Mar 8, 2012 at 11:50 AM, de Souza Medeiros Andre andre.medei...@aalto.fi wrote: Or does this happen only with one-region tables? Yes Even with multiregion tables, you will still get a null start key for the first

Re: HTable.getEndKeys() returning empty results

2012-03-08 Thread Michael Drzal
On Thu, Mar 8, 2012 at 2:50 PM, de Souza Medeiros Andre andre.medei...@aalto.fi wrote: Thanks for the answer. This is quite confusing though, what is the point of having a method to retrieve the last key if we know that anyway it will be an empty byte array? Or does this happen only with

Re: Bulk loading a CSV file into HBase

2012-03-08 Thread anil gupta
Yeah after digging further into the code: Line#374 in GenericOptionsParser.java commandLine = parser.parse(opts, args, true); is the culprit. Nice find, Shrijeet. That answers my question. :) Stack: Could you please tell me the meaning of IIRC? Updating the document is good but as per the

Re: org.apache.hadoop.conf.Configuration - error parsing conf file

2012-03-08 Thread Something Something
There's a 'quietmode' variable in org/apache/hadoop/conf/Configuration.java class: /** * Set the quietness-mode. * * In the quiet-mode, error and informational messages might not be logged. * * @param quietmode codetrue/code to set quiet-mode on, codefalse/code *

Re: Getting Merge failed exception

2012-03-08 Thread Jean-Daniel Cryans
The merge tool doesn't work with bulk loaded files. J-D On Thu, Mar 8, 2012 at 1:39 PM, Rohit Nigam tanuro...@gmail.com wrote: Hi I am trying to merge 2 regions in a table and i get this exception 2012-03-08 12:43:51,679 FATAL [main] util.Merge(114): Merge failed java.io.IOException: Files

SocketTimeoutException upon 'create' command

2012-03-08 Thread Yermalkar, Sanjay
Hi, I get the following exception upon Œcreate¹ table command. However most of the times the table does get created. Also, this exception doesn¹t occur upon other commands. Any help is appreciated. hbase(main):008:0 create 't1','cf1','cf2' ERROR: java.net.SocketTimeoutException: Call to

Re: gc pause killing regionserver

2012-03-08 Thread Michael Segel
Hey do that, things go boom. :-) Before you do that I would suggest running top and seeing if there is any swapping occurring. Sent from my iPhone On Mar 8, 2012, at 4:29 PM, Jean-Daniel Cryans jdcry...@apache.org wrote: When real cpu is bigger than user cpu it very often points to

re: gc pause killing regionserver

2012-03-08 Thread Gaojinchao
We encountered the same thing. we set swappiness priority to 0. But swap is still working. So we disable swap. -邮件原件- 发件人: jdcry...@gmail.com [mailto:jdcry...@gmail.com] 代表 Jean-Daniel Cryans 发送时间: 2012年3月9日 6:29 收件人: user@hbase.apache.org 主题: Re: gc pause killing regionserver When

RE: gc pause killing regionserver

2012-03-08 Thread Laxman
Hi Ferdy, I'm running regionservers with 2GB heap and following tuning options: -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewRatio=16 -XX:CMSInitiatingOccupancyFraction=70 - XX:+UseCMSInitiatingOccupancyOnly -XX:MaxGCPauseMillis=100 GC with huge heaps may take longer GC pauses. But

Problem in start Hbase Rest Server

2012-03-08 Thread Mahdi Negahi
Dear All Friends I install Hbase in standalone mode in my laptop (Ubuntu 11.10) . I want to use Restful API to programming. So I read this link : http://wiki.apache.org/hadoop/Hbase/Stargate so when I run the following command : ./bin/hbase rest start the command execution stop at last line

Re: Problem in start Hbase Rest Server

2012-03-08 Thread Stack
On Thu, Mar 8, 2012 at 9:27 PM, Mahdi Negahi negahi.ma...@hotmail.com wrote: 12/03/09 00:21:26 INFO util.VersionInfo: HBase 0.92.0 12/03/09 00:21:26 INFO util.VersionInfo: Subversion https://svn.apache.org/repos/asf/hbase/branches/0.92 -r 1231986 12/03/09 00:21:26 INFO util.VersionInfo:

Re: Solr+Hbase

2012-03-08 Thread Mohammad Tariq
Hello Stack, Thanks a lot for your valuable comments.I have started looking into both the options.I'll surely let you know as we proceed further. Regards,     Mohammad Tariq On Thu, Mar 8, 2012 at 10:26 PM, Stack st...@duboce.net wrote: On Thu, Mar 8, 2012 at 12:18 AM, Mohammad Tariq