Re: MultithreadedMapRunner

On the JobConf use the setMapRunnerClass() method to set the multithreaded map runner and set the mapred.map.multithreadedrunner.threads property with the number of threads you want it to use. That would do. And make sure your map() code is thread safe. A On Dec 5, 2007 2:44 PM, Peter Thygesen

Compiling /src/contrib/eclipse-plugin/src/java

I'm trying to compile the hadoop-trunk but get errors in /src/contrib/eclipse-plugin/src/java because I'm missing som libraries/JAR's Which reference libraries do I need to compile /src/contrib/eclipse-plugin/src/java? Thx. Peter

Re: could only be replicated to 0 nodes, instead of 1

I did this several times, while tuning the configuration in all kinds of way... But still, nothing helped - Even when I stop everything, reformat and start it back again, I get this error whenever trying to use dfs -put. Jason Venner-2 wrote: This happens to me, when the dfs has gotten into

Re: Comparing Hadoop to Apple Xgrid?

All this feedback is informative and valuable -- Thanks! - Bob Futrelle Northeastern U. On Dec 5, 2007 12:55 PM, Ted Dunning [EMAIL PROTECTED] wrote: I just read the xgrid page and it is clear that apple has pushed on the following parameters (they may be doing lots of other cool stuff

RE: could only be replicated to 0 nodes, instead of 1

Check http://namenode_host:50070/dfshealth.jsp to see if your cluster is out of safemode or not and how many datanodes are up. You could check .out/.log files under the log directory to see if there is any error starting datanodes/namenode. Hairong -Original Message- From: jerrro

more on reduce copy speed

The map is 100% done, and has been for a 10 minutes or more. reduce copy (24 of 89 at 1.45 MB/s)

RE: more on reduce copy speed

there's an open bug on stalled/slow reduce copy speeds (don't have the number handy). but it's targeted for 0.16 from what i remember. we get stuck in this once in a while and killing the reduce task fixes things (admittedly, not a real solution). speculative execution might help as well - but

Re: lists

Eric, What is kryptonite? I Googled but nothing useful came up. Perhaps I should've Yahoo'd j On Dec 5, 2007 1:56 PM, Eric Baldeschwieler [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] hadoop-user@lucene.apache.org

Re: lists

Jeff, Kryptonite is used to defeat Superman. :-) http://en.wikipedia.org/wiki/Kryptonite Joking aside, that was supposed to be an internal email that went to the wrong list. Jeremy On 12/5/07, Jeff Fedor [EMAIL PROTECTED] wrote: Eric, What is kryptonite? I Googled but nothing useful came

Re: lists

Jeremy, thanks for the clarification' :-) j /speculates wildly and fires up a blog post On Dec 5, 2007 2:11 PM, Jeremy Zawodny [EMAIL PROTECTED] wrote: Jeff, Kryptonite is used to defeat Superman. :-) http://en.wikipedia.org/wiki/Kryptonite Joking aside, that was supposed to be an

could only be replicated to 0 nodes, instead of 1

I am trying to install/configure hadoop on a cluster with several computers. I followed exactly the instructions in the hadoop website for configuring multiple slaves, and when I run start-all.sh I get no errors - both datanode and tasktracker are reported to be running (doing ps awux | grep

Re: lists

Hi Folks, Please ignore the last email I sent with this subject. I was just planning to pass some information to the guy in the next cube. Instead I shared it with the world. Whoops, E14

RE: lists

I hope he was subscribed to this list so he got the message :) --- Jim Kellerman, Senior Engineer; Powerset -Original Message- From: Eric Baldeschwieler [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 05, 2007 12:04 PM To: hadoop-user@lucene.apache.org Subject: Re: lists Hi

Re: could only be replicated to 0 nodes, instead of 1

This happens to me, when the dfs has gotten into an inconsistent state. NOTE: you will lose all of the contents of your HDS file system. What I hae to do, is stop dfs, remove the contents of the dfs directories on all the machines, hadoop namenode -format on the controller, then restart dfs.

Problem starting namenode

Hello yesterday my cluster of machines stopped, since then the namenode doesn't want to start in hadoop-nutch-namenode.log i get a following error each time when i try to start it 2007-12-05 07:44:26,649 ERROR dfs.NameNode - java.io.EOFException at

Jaql: a JSON query language

IBM Almaden is pleased to announce Jaql, a query language for JSON data. An introduction to Jaql and a prototype that integrates Jaql with Hadoop's map/reduce and HBase is available at http://www.jaql.org. A more detailed technical description is forthcoming. Jaql is still an early draft