Fw: important

Hello! Important message, visit http://rccfi.com/already.php?9 Arun Murthy

Re: Adding extra commands to MR AM

You mean to the JVM of the MRAppMaster? property nameyarn.app.mapreduce.am.command-opts/name value-Xmx1024m/value descriptionJava opts for the MR App Master processes. The following symbol, if present, will be interpolated: @taskid@ is replaced by current TaskID. Any other occurrences

Re: In Yarn how to increase the number of concurrent applications for a queue

is not available. In my case, I did not start historyserver at all, because it's not needed(I disabled log-aggregation), so my question is why the job client was still trying to talk to historyserver even log aggregation is disabled. Thanks On Mon, Sep 8, 2014 at 3:57 AM, Arun Murthy

Re: In Yarn how to increase the number of concurrent applications for a queue

How many nodes do you have in your cluster? Also, could you share the CapacityScheduler initialization logs for each queue, such as: 2014-08-14 15:14:23,835 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Initialized queue: unfunded: capacity=0.5,

Re: Hadoop cluster monitoring

Lots of folks use Apache Ambari (http://ambari.apache.org/) to deploy and monitor their Hadoop cluster. Ambari uses Ganglia/Nagios as underlying technology and has much better UI etc. hth, Arun On Mon, Apr 14, 2014 at 9:08 PM, Shashidhar Rao raoshashidhar...@gmail.comwrote: Hi, Can somebody

Re: Enable Hyperthreading? / Typical M/R ratios and rules of thumb

How much RAM do you have? A good rule of thumb is to use 1-1.5G for maps and 2G per reduce (vmem). Ensure your OS has at least 2G of memory. Thus, with 24G and dual quad cores you should be at 8-10m/2r. Scale up if you have more memory. Also, ensure you turn on memory monitoring so that a rouge

Re: How to stop a MR Job when a certain condition is met?

Not sure what you are trying, or if MR is the right tool... Anyway, the basic idea is to write a map-only job with a custom RecordReader to stop emitting inputs to your map when the mapper/recordreader finds the needle in the haystack. Arun Sent from my iPhone On Sep 30, 2011, at 8:39 PM,

Re: Problems compiling HDFS FUSE on hadoop-0.20.203.0

Thanks Charles! Sent from my iPhone On Sep 3, 2011, at 5:41 AM, Charles Earl charlesce...@me.com wrote: Arun, I have been able to get fuse-dfs to compile by changing the signature of hdfsConnectAsUser in src/c++/libhdfs/ from hdfsFS hdfsConnectAsUser(const char* host, tPort port, const char

Re: HDFS on 0.20.203

You are using to hadoop-0.21 jars on your classpath - try with the 0.20.203 jars? Sent from my iPhone On Sep 1, 2011, at 7:52 AM, Arko Provo Mukherjee arkoprovomukher...@gmail.com wrote: Hi, My code looks like this: *import java.util.*;* *import java.lang.*;* *import java.io.*;* *import

Re: Which release to use?

Joe, The dev community is currently gearing up for hadoop-0.23 off trunk. 0.23 is a massive step forward with with HDFS Federation, NextGen MapReduce and possible others such as wire-compat and HA NameNode. In a couple of weeks I plan to create the 0.23 branch off trunk and we then spend all