Hadoop 2.2 Built-in Counters

Hi all, I'm trying to analyze my mapreduce job performance via built-in counters such as physical memory usage, heap memory usage... When the job is running, I can watch these counters via Resource manager website(namenode:8088). However, when the job is done, counter information is not available

Re: Hadoop 2.2 Built-in Counters

You can see them in the MR Job HistoryServer UI if you have one configured. You can find config properties and launch command in http://hadoop.apache.org/docs/stable2/hadoop-project-dist/hadoop-common/ClusterSetup.html On Thu, Aug 14, 2014 at 11:44 AM, ou senshaw sens...@gmail.com wrote: Hi all,

Re: Hadoop 2.2 Built-in Counters

Hi, This is the content of my shell script for running the job history server: cd $HADOOP_PREFIX hadoop fs -mkdir -p /mr-history/tmp hadoop fs -chmod -R 1777 /mr-history/tmp hadoop fs -mkdir -p /mr-history/done hadoop fs -chmod -R 1777 /mr-history/done sbin/mr-jobhistory-daemon.sh start

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

Hi, I have been using Hadoop since Christmas loosely and from May for an Software engineering MSc at Heriot Watt University in Edinburgh, Scotland. I have written a genetic sequence alignment algorithm. I have installed Hadoop in various places including a 32 node cluster and am using eclipse

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

As a beginner, it depends on what you want to learn? Do you want to program MapReduce, just do some SQL queries to hadoop, or install, deploy and monitor a Hadoop cluster? This article might help making a good decision: spoilt for choice - how to choose the right Hadoop distribution

Relationship between number of reducers and number of regions in the table

I couldn't decide that whether it is an HBase question or Hadoop/Yarn. In the utility class for MR jobs integerated with HBase, *org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil, * in the method: *public static void initTableReducerJob(String table,* *Class? extends TableReducer

Re: Can anyone help me resolve this Error: unable to create new native thread

Hi Chris! When is this error caused? Which logs do you see this in? Are you sure you are setting the ulimit for the correct user? What application are you trying to run which is causing you to run up against this limit? HTH Ravi On Saturday, August 9, 2014 6:07 AM, Chris MacKenzie

Re: Hadoop 2.2 Built-in Counters

Thanks guys! It's working. On Thu, Aug 14, 2014 at 2:39 AM, Chris MacKenzie stu...@chrismackenziephotography.co.uk wrote: Hi, This is the content of my shell script for running the job history server: cd $HADOOP_PREFIX hadoop fs -mkdir -p /mr-history/tmp hadoop fs -chmod -R 1777

Data Staging exp

Hi Friends, Do anyone of you have Data Staging exp using *Rest API* for hadoop project. Please share you exp. Thanks Kripa

Re: hadoop/yarn and task parallelization on non-hdfs filesystems

I've looked a bit into this problem some more, and from what another person has written, HDFS is tuned to scale appropriately [1] given the number of input splits, etc. In the case of utilizing the local filesystem (which is really a network share on a parallel filesystem), the settings might be

Re: Can anyone help me resolve this Error: unable to create new native thread

Hi Ravi, I resolved this. Many thanks. Regards, Chris MacKenzie telephone: 0131 332 6967 email: stu...@chrismackenziephotography.co.uk corporate: www.chrismackenziephotography.co.uk http://www.chrismackenziephotography.co.uk/ http://plus.google.com/+ChrismackenziephotographyCoUk/posts