spilled records

Hi, According to Hadoop: the definitive guide, when mapreduce.job.shuffle.input.buffer.percent is large enough, the map outputs are copied directly into the reduce JVM memory. I set this parameter to 0.5 which is large enough to hold map outputs, but #spilled records is still the same as

Web interface not show applications

Dear all, I am using hadoop 2.4.0 in pseudo distributed mode, I tried to run the example wordcount program. It finished successfully, but I am not able to see the application/job from the localhost:8088 web interface. I started the job history daemon, and nothing from localhost:19888 either.

Re: HDFS Symbolic Links support - What's the status ?

Any info on this will be appreciated. Thanks, On Wed, May 7, 2014 at 3:01 PM, Manoj Samel manojsamelt...@gmail.comwrote: Hi, There are some JIRAs for supporting symbolic links in HDFS (e.g. HDFS-245, HADOOP-6421) but that feature does not seem to be available, at least from HDFS commands?

Re: Realtime sensor's tcpip data to hadoop

or you can use combination of kafka http://kafka.apache.org/ + phoenixhttp://phoenix.incubator.apache.org/ On Wed, May 7, 2014 at 8:55 PM, Azuryy Yu azury...@gmail.com wrote: Hi Alex, you can try Apache Flume. On Wed, May 7, 2014 at 10:48 AM, Alex Lee eliy...@hotmail.com wrote: Sensors'

Re: Realtime sensor's tcpip data to hadoop

Use the Flume. On Wed, May 7, 2014 at 8:18 AM, Alex Lee eliy...@hotmail.com wrote: Sensors' may send tcpip data to server. Each sensor may send tcpip data like a stream to the server, the quatity of the sensors and the data rate of the data is high. Firstly, how the data from tcpip can be

distcp2 with bandwidth throttle failed with NoSuchMethodError

Hi, I am using Hadoop 2.0.0-cdh4.2.0, and I copied hadoop-distcp-2.0.3-alpha.jar under /usr/lib/hadoop folder. I can see it support bandwidth parameter for throttle. However, I got below error. I found people have similar problem, but seems no answer.

Getting the name of the host on which Application Master is launched

Hi, Is there a way to get the name of the host where the AM is launched? I have seen that there is a method getHost() in the ApplicationReport that we get in YarnClient, but it is giving null. Is there a way to make it work? or is there any other way to get the host name? 2014-05-09 04:36:05

Re: issue about cluster balance

i record the disk status befor balance and after balance,from one of source node and one of destination node before source node /dev/sdd 1.8T 1009G 733G 58% /data/1 /dev/sde 1.8T 1005G 737G 58% /data/2 /dev/sda 1.8T 980G 762G 57% /data/3 /dev/sdb

question about job class

Hi I have a question about the Job class. Here is its declaration. public class Job extends JobContextImpl implements JobContext JobContextImpl already implements JobContext. Why is it declared here that Job implments JobContext? Is it necessary? Thanks. Libo

Re: Realtime sensor's tcpip data to hadoop

I will suggest that don't pipe the sensor data to the HDFS directly instead you can have some program(either java,python etc) on the server itself to process the incoming sensor data and writing it to the text/binary file(don't know the data format which you are currently receiving).now you can