hadoop-site.xml has been deprecated for a few years now. I'm not sure
if the benefits of splitting it up paid off, but it does mean one must
determine which property goes into which -site.xml file. What I do when
I install Hadoop is:
mkdir -p $HADOOP_HOME/conf/defaults
cd $HADOOP_HOME/conf/defaults
jar xvf ../../hadoop-*core.jar core-default.xml hdfs-default.xml
mapred-default.xml
so that the documented properties are easily at hand.
Paul
On 20111204 11:49 , Stephen Boesch wrote:
OK I'm using 11.10 (full install not AMI but should not matter i
think). I think I've used just hadoop-site.xml and it still works,
will double check.
2011/12/4 Periya.Data <[email protected]
<mailto:[email protected]>>
Ok. Thanks Stephen. But, any idea why I am unable to run hadoop
commands? I believe my next post...(regarding lack of JAVA and
Hadoop on Ubuntu 11.10 AMI) may be the reason...
PD
On Sun, Dec 4, 2011 at 11:42 AM, Stephen Boesch <[email protected]
<mailto:[email protected]>> wrote:
it sill works but is deprecated in favor of core-site.xml,
hdfs-site.xml, and mapred-site.xml
2011/12/4 Periya.Data <[email protected]
<mailto:[email protected]>>
Hi,
I just began using Whirr to deploy clusters on EC2. I
am following the instructions mentioned in Whirr
documentation/quick start guide. I am able to start
clusters and destroy from my laptop. However, I see an
issue when I try to connect to a machine on EC2.
I am using Whirr 0.5, hadoop-0.20.2-cdh3u2 on my Ubuntu
11.10 laptop.
The instances I created on EC2 are Ubuntu 11.10 Oneiric
with hadoop-0.20.2-cdh3u2.
Here is what I am doing:
(on teminal -1)
$ *. ~/.whirr/HadoopCluster/hadoop-proxy.sh *
Running proxy to Hadoop cluster at
ec2-75-101-193-84.compute-1.amazonaws.com
<http://ec2-75-101-193-84.compute-1.amazonaws.com>. Use
Ctrl-c to quit.
Warning: Permanently added
'ec2-75-101-193-84.compute-1.amazonaws.com
<http://ec2-75-101-193-84.compute-1.amazonaws.com>,75.101.193.84'
(ECDSA) to the list of known hosts.
channel 2: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
(on a different terminal , terminal - 2)
$ *export HADOOP_CONF_DIR=~/.whirr/HadoopCluster/*
$
$
$ *hadoop fs -ls /*
11/12/03 21:49:30 WARN conf.Configuration: DEPRECATED:
hadoop-site.xml found in the classpath. Usage of
hadoop-site.xml is deprecated. Instead use core-site.xml,
mapred-site.xml and hdfs-site.xml to override properties
of core-default.xml, mapred-default.xml and
hdfs-default.xml respectively
11/12/03 21:49:32 INFO ipc.Client: Retrying connect to
server:
ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020
<http://ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020>.
Already tried 0 time(s).
11/12/03 21:49:33 INFO ipc.Client: Retrying connect to
server:
ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020
<http://ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020>.
Already tried 1 time(s).
11/12/03 21:49:35 INFO ipc.Client: Retrying connect to
server:
ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020
<http://ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020>.
Already tried 2 time(s).
========================
I am executing all the above commands from my local laptop
and nothing is done on the remote instance.
Your suggestions would be helpful.
Thanks,
PD.
