Hi, folks!
I've deployed hadoop (0.20.203.0rc1) on 8-node cluster. After uploading
file onto hdfs I've got this file only on one of the nodes instead of being
uniformly distributed across all nodes. What can be the issue?
$HADOOP_HOME/bin/hadoop dfs -copyFromLocal ../data/rmat-20.0
Hi,
The 0.20.203.0rc1 is a very old version at this point. Why not use a
more current version if you're deploying a new cluster?
Onto your issue, your configuration XML files (core-site.xml,
hdfs-site.xml or mapred-site.xml) most likely have a dfs.replication
value set to 1 causing only that may
Hi Alex,
You should give the copyFromLocal command from the namenode or any machine
that is not a datanode to get the file distributed.
On Fri, Feb 7, 2014 at 10:53 AM, Harsh J ha...@cloudera.com wrote:
Hi,
The 0.20.203.0rc1 is a very old version at this point. Why not use a
more current
Hi,
I am having some failure test cases where my Application Master is
supposed to fail. But when it fails it is again started with appID_02 .
Is there a way for me to avoid the second instance of the Application
Master getting started? Is it re-started automatically by the RM after the
first
Thanks Park for sharing the above configs
But I am wondering if the above config changes would make any huge
difference in my case.
As per my logs, I am very worried about this line -
INFO org.apache.hadoop.mapred.MapTask: Record too large for in-memory
buffer: 644245358 bytes
If I am
Hi all,
I have a question about reserved containers in the YARN capacity
scheduler. After reading the source code and related document, it is
not very clear. What is the purpose or practical usage of the reserved
container? thx.
Hi Park,
Your explanation makes perfect sense in my case. Thanks for explaining what
is happening behind the scenes. I am wondering you used normal java
compression/decompression or is there a UDF already available to do this
stuff or some kind of property that we need to enable to say to PIG
Hi Prav,
You're thinking correctly, and it's true that Pig bags are spillable.
However, spilling is no magic, meaning you can still run into OOM with huge
bags like you have here. Pig runs Spillable Memory Manager (SMM) in a
separate thread. When spilling is triggered, SMM locks bags that it's
Thanks, I built 2.3 yesterday (checked out from from link suggested in
earlier post of this thread) without problems apart from VM running out of
memory which was fixed with
export MAVEN_OPTS=-Xmx2048m
At least, I got a message saying successful build.
Thanks for your help.
On 8 February
