Forrest,
I must admit. I am very new myself. Two months ago I didn't know what "$sudo"
was. :) So, do not take my comments too seriously. I could be wrong about many
issues.
There are 5 daemon scripts in /etc/init.d/. I am using MRv1, so mine are:
hadoop-hdfs-datanode
hadoop-hdfs-namenode
hadoop-hdfs-secondarynamenode
<specific to MRv1>
hadoop-0.20-mapreduce-jobtracker
hadoop-0.20-mapreduce-tasktracker
For scalability I chose to:
- only have NN/JT on my master node so I need to start|stop|restart
only hadoop-hdfs-namenode/hadoop-0.20-mapreduce-jobtracker
- different node for SNN so I need to only start|stop|restart
hadoop-hdfs-secondarynamenode
- all of my DNs -
hadoop-hdfs-datanode/hadoop-0.20-mapreduce-tasktracker
My script is VERY simple for now:
#!/bin/bash
# myHadoop.sh
dir=/etc/init.d
if [ $# -lt 1 ]; then
echo "USAGE: $`basename $0` [start|stop|restart]"
exit 1
fi
sudo ${dir}/hadoop-hdfs-namenode $1
sudo ${dir}/hadoop-0.20-mapreduce-jobtracker $1
exit 0
for SNN I altered script to run instead:
sudo ${dir}/hadoop-hdfs-secondarynamenode $1
and for DNs:
sudo ${dir}/hadoop-hdfs-datanode $1
sudo ${dir}/hadoop-0.20-mapreduce-tasktracker $1
p.s. Criticism on my post is welcomed.
AK47
From: Forrest Aldrich [mailto:[email protected]]
Sent: Tuesday, November 06, 2012 3:09 PM
To: [email protected]
Subject: Re: starting Daemons remotely, through ssh - SOLVED
Andy, I noticed you are splitting up your /etc/init.d startup -- we're new-ish
to Hadoop and found that the supplied init.d script doesn't necessarily stop
services properly, so I end up doing it manually. If you could share your
info that would be great... I can see where they can be time-delayed in the
startup with chkconfig (RHEL).
Thanks.
On 11/6/12 3:07 PM, Kartashov, Andy wrote:
Harsh/Ravi,
I wrote my own scripts to [start|stop|restart] [hdfs|mapred] daemons. The
content of the script is
$sudo service /etc/init.d/hadoop-[hdfs-*|mapred-*]
I have no problem starting Daemons locally on each node. I have sudo-access
granted. I run script and enter password.
I have passwordless ssh enabled. But when I am trying to start slave Daemons
from NN I get
"<remothsot>: sudo: sorry, you must have a tty to run sudo".
I just solved this issue by:
a. Modifying slaves.sh script to add -t after ssh "-t" <host-ip> <command>
a1. Someone suggested to:
$sudo sudeors and commenting out Defaults requiretty
But a. worked just fine.
Cheers,
AK47
From: Ravi Mutyala [mailto:[email protected]]
Sent: Tuesday, November 06, 2012 2:31 PM
To: [email protected]<mailto:[email protected]>
Subject: Re: starting Daemons remotely, through ssh
Andy,
Which specific scripts are these? As far as I know, all daemons on the hadoop
side can be started with hadoop/hdfs or any non-sudo username. You will require
passwordless ssh for that user if you are using slaves/regionservers configured
to start the daemons remotely from the NameNode/HBase Master.
On Tue, Nov 6, 2012 at 1:25 PM, Kartashov, Andy
<[email protected]<mailto:[email protected]>> wrote:
Hadoopers,
How does one start Daemons remotely when scripts normally require root user to
start them? Do you modify scripts?
Thanks,
NOTICE: This e-mail message and any attachments are confidential, subject to
copyright and may be privileged. Any unauthorized use, copying or disclosure is
prohibited. If you are not the intended recipient, please delete and contact
the sender immediately. Please consider the environment before printing this
e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont
confidentiels, protégés par le droit d'auteur et peuvent être couverts par le
secret professionnel. Toute utilisation, copie ou divulgation non autorisée est
interdite. Si vous n'êtes pas le destinataire prévu de ce courriel,
supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à
l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to
copyright and may be privileged. Any unauthorized use, copying or disclosure is
prohibited. If you are not the intended recipient, please delete and contact
the sender immediately. Please consider the environment before printing this
e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont
confidentiels, protégés par le droit d'auteur et peuvent être couverts par le
secret professionnel. Toute utilisation, copie ou divulgation non autorisée est
interdite. Si vous n'êtes pas le destinataire prévu de ce courriel,
supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à
l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to
copyright and may be privileged. Any unauthorized use, copying or disclosure is
prohibited. If you are not the intended recipient, please delete and contact
the sender immediately. Please consider the environment before printing this
e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont
confidentiels, protégés par le droit d'auteur et peuvent être couverts par le
secret professionnel. Toute utilisation, copie ou divulgation non autorisée est
interdite. Si vous n'êtes pas le destinataire prévu de ce courriel,
supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à
l'environnement avant d'imprimer le présent courriel
