[ANN] HUG3 -- The Third HBase User Group Meeting, 20th May in San Francisco

See http://upcoming.yahoo.com/event/672690 Thanks, St.Ack

[ANN] hbase-0.1.2 release now available for download

hbase-0.1.2 resolves 27 issues including critical fixes for 'missing' edits and unreliable onlining/offlining of tables. We recommend all upgrade to this latest version. To download, please go to http://hadoop.apache.org/hbase/releases.html. Thanks to all who contributed to this release. You

[ANN] HUG3 -- The Third HBase User Group Meeting, 20th May in San Francisco

See http://upcoming.yahoo.com/event/672690 Thanks, St.Ack

how to debug a fully-distributed wordcount performance?

how to debug a fully-distributed wordcount performance? [11:01] I have installed Hadoop on standalone, pseudo distributed, and fully distributed, [11:01] then run the hadoop-0.16.4-examples.jar wordcount, [11:02] I found the wordcount can run quickly on local and pseudo mo

everything becomes very slow when the number of writes is larger than the size of the cluster using *TestDFSIO* benchmark?

Hi all, I run the *TestDFSIO* benchmark on a simple cluster of 2 nodes. The file size is the same in all cases 2GB. The number of files tried is 1,2,4,8(only write). The bufferSize is 65536 bytes. The file replication is 1. the results as below: files 1 2 4 8 write -- Throughout(mb/s) 52.89 52.

Re: Read timed out, Abandoning block blk_-5476242061384228962

I'm using the machine running the namenode to run maps as well - could that be a source of my problem? The load is fairly high, essentially no idle time. 8 cores per machine, so I've got 8 maps running. I'm guessing I'd be better off running 80 smaller machines instead of 20 larger ones for t

Re: why it stopped at Reduce phase?

Also, hadoop.tmp.dir defaults to storing files in /tmp/hadoop-{username} - it's very common for machines to not have a lot of space available in /tmp. You'll probably want to change that location. -- James Moore | [EMAIL PROTECTED] blog.restphone.com

Trouble hooking up my app to HDFS

I'm trying to create a java application that writes to HDFS. I have it set up such that hadoop-0.16.3 is on my machine, and the env variables HADOOP_HOME and HADOOP_CONF_DIR point to the correct respective directories. My app lives elsewhere, but generates it's classpath by looking in those

Re: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find task_200805131750_0005_m_000001_0/file.out.index in any of the configured local directories

hadoop.tmp.dir tmp_storage Could you try and change the above to an absolute path and check? That path should be relevant on each of the tasktrackers. Of course, you can configure each tasktracker independently by editing it's hadoop-site.xml. Arun

org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find task_200805131750_0005_m_000001_0/file.out.index in any of the configured local directories

I've followed carefully the instructions on http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Single-Node_Cluster%29, but the task fails with the following stack trace : 08/05/13 19:16:26 INFO mapred.JobClient: map 100% reduce 0% 08/05/13 19:18:55 INFO mapred.JobClient: Task Id :

Re: Streaming and subprocess error code

Hi Rick, Thank you for the quick response! I see this feature is in trunk and not available in last stable release. Anyway will try if it works for me from the trunk, and will try does it catch segmentation faults too. Rick Cox wrote: Try "-jobconf stream.non.zero.exit.status.is.failure=tru

Re: why it stopped at Reduce phase?

Wang, On May 13, 2008, at 8:12 AM, wangxiaowei wrote: hi all: I uses two computers A and B as a hadoop cluster,A is JobTracker and NameNode,both A and B are slaves. The input data size is about 80MB,including 100,000records. The job is to read one record a time and find some useful conte

Re: Streaming and subprocess error code

On May 13, 2008, at 8:09 AM, Rick Cox wrote: Try "-jobconf stream.non.zero.exit.status.is.failure=true". Anyone willing to document this on http://hadoop.apache.org/core/docs/ current/streaming.html? Ideally HADOOP-2057 should have documented this useful feature, better late than never...

Re: HDFS corrupt...how to proceed?

Hi, I'd love to see the DRBD+Hadoop write up! Not only would this be useful for Hadoop, I can see this being useful for Solr (master replication). Thanks, Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: C G <[EMAIL PROTECTED]> > To: core-

Re: why it stopped at Reduce phase?

It appears that your hard disk is full on one of your 2 slaves, that is all. If you are on UNIX/linux, type this at the prompt: df You should see 100% for the partition where you put HDFS. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: w

Re: Streaming and subprocess error code

Try "-jobconf stream.non.zero.exit.status.is.failure=true". That will tell streaming that a non-zero exit is a task failure. To turn that into an immediate whole job failure, I think configuring 0 task retries (mapred.map.max.attempts=1 and mapred.reduce.max.attempts=1) will be sufficient. rick

Streaming and subprocess error code

Hi all, I'm looking a way to force Streaming to shutdown the whole job in case when some of its subprocesses exits with non-zero error code. We have next situation. Sometimes either mapper or reducer could crush, as a rule it returns some exit code. In this case entire streaming job finishes