How to remove ending tab separator in streaming output

2011-12-26 Thread devdoer bird
HI: In streming MR program, I use /bin/cat as a mapper and set reducer=NONE, but the outputs all end with a tab separator. How do I configure the streaming MR program to remove the ending separator? Thanks.

Secondary Namenode on hadoop 0.20.205 ?

2011-12-26 Thread praveenesh kumar
Hey people, How can we setup another machine in the cluster as Secondary Namenode in hadoop 0.20.205 ? Can a DN also act as SNN, any pros and cons of having this configuration ? Thanks, Praveenesh

RE: Hadoop configuration

2011-12-26 Thread Uma Maheswara Rao G
Hey Humayun, Looks your hostname still not resoling properly. even though you configured hostnames as master, slave...etc, it is getting humayun as hostname. just edit /etc/HOSTNAME file with correct hostname what you are expecting here. To confirm whether it is resolving properly or not, you

RE: Secondary Namenode on hadoop 0.20.205 ?

2011-12-26 Thread Uma Maheswara Rao G
Hey Praveenesh, You can start secondary namenode also by just giving the option ./hadoop secondarynamenode DN can not act as seconday namenode. The basic work for seconday namenode is to do checkpointing and getting the edits insync with Namenode till last checkpointing period. DN is to

Re: Secondary Namenode on hadoop 0.20.205 ?

2011-12-26 Thread Harsh J
(Answering beyond Uma's reply) Can a DN also act as SNN, any pros and cons of having this configuration ? You can run SNN on a regular slave box if you can't have a dedicate a box, it shouldn't be an issue for small clusters -- Do ensure its disk configuration is proper, and its allocated

Re: Hadoop configuration

2011-12-26 Thread Humayun kabir
Hi Uma, Thanks a lot. At last it is running without errors. Thank you very much for your suggestion. On 26 December 2011 20:04, Uma Maheswara Rao G mahesw...@huawei.com wrote: Hey Humayun, Looks your hostname still not resoling properly. even though you configured hostnames as master,

Re: How to remove ending tab separator in streaming output

2011-12-26 Thread Mahadev Konar
Bccing common-user and ccing mapred-user. Please use the correct mailing lists for your questions. You can use -Dstream.map.output.field.separator= for specifying the seperator.   The link below should have more information.

anyone upgrade from 0.21 to 0.22?

2011-12-26 Thread steven zhuang
hello all, I have a running hadoop 0.21.0 cluster, since 0.22 is the stable version and we don't use Kerberos. just want to be sure, has anybody upgrade from hadoop 0.21 to 0.22 siccessfully? any hint would be greatly appreciated! -- best wishes. Steven

Re: Secondary Namenode on hadoop 0.20.205 ?

2011-12-26 Thread praveenesh kumar
Thanks..But, my 1st question is still unanswered. I have a 8 DN/TT machines and 1 NN machine. I want to set one of my DN/TT machine as SNN. How I have to configure my conf/*.xml files to achieve this ? Thanks, Praveenesh On Mon, Dec 26, 2011 at 8:44 PM, Harsh J ha...@cloudera.com wrote:

Re: Secondary Namenode on hadoop 0.20.205 ?

2011-12-26 Thread Harsh J
The link Uma passed already covered that question: http://www.cloudera.com/blog/2009/02/multi-host-secondarynamenode-configuration/ [dfs.http.address in hdfs-site.xml pointing to NN_HOST:50070 should do.] Also, if you are using the tarball start/stop scripts, putting in the hostname for SNN in

Re: Secondary Namenode on hadoop 0.20.205 ?

2011-12-26 Thread Harsh J
Yes, checkpoints are helpful when your original NN image goes corrupt (very very rare, if you use dual or more dfs.name.dir points to be safe). On 27-Dec-2011, at 12:33 PM, praveenesh kumar wrote: Cool. I just did that.. So now I am seeing my fsimage file on SNN's hadoop.tmp.dir... So