Hi Team/Mingliang We have upgraded to Hadoop 3.2.1 and we are having following issues , can somebody please help me on this.
We have successfully started name node and journal-node. However, while starting the Datanode from namenode is not working , the command simply exits. [hdfs@sym-hdfsnn2 hadoop]$ hdfs version Hadoop 3.2.1 Source code repository https://gitbox.apache.org/repos/asf/hadoop.git -r b3cbbb467e22ea829b3808f4b7b01d07e0bf3842 Compiled by rohithsharmaks on 2019-09-10T15:56Z Compiled with protoc 2.5.0 >From source with checksum 776eaf9eee9c0ffc370bcbc1888737 This command was run using /home/hdfs/hadoop/share/hadoop/common/hadoop-common-3.2.1.jar [hdfs@sym-hdfsnn2 hadoop]$ hdfs --daemon stop datanode Regards, Kamaraj From: Muthupandiyan, Kamaraj<mailto:kamaraj.muthupandi...@hpe.com> Sent: Sunday, August 23, 2020 9:22 PM To: Mingliang Liu<mailto:lium...@gmail.com> Cc: user@hadoop.apache.org<mailto:user@hadoop.apache.org> Subject: RE: setting up Load balancer for name nodes Hi Mingliang Liu, Could you please help me with my additional questions 1. Do we have any other options for setting up the Load balancer for name node in Hadoop Version 2.9.2 ? 1. I got the following document for upgrading the HDFS cluster - https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html or Do we have any other document for upgrading Version 2.9.2 to Version 3.2.x Regards, Kamaraj InfoSight - SRE From: Muthupandiyan, Kamaraj<mailto:kamaraj.muthupandi...@hpe.com> Sent: Sunday, August 23, 2020 6:57 PM Subject: RE: setting up Load balancer for name nodes Hi Mingliang Liu, Thanks a lot for your response, I have read the document for Hadoop 3.3.0 and found that Load balancers will be supported on AWS/Azure, will it supported on A10 Loadbalancers or nginx ? Regards, Kamaraj InfoSight - SRE From: Mingliang Liu<mailto:lium...@gmail.com> Sent: Saturday, August 22, 2020 2:55 AM To: Muthupandiyan, Kamaraj<mailto:kamaraj.muthupandi...@hpe.com> Cc: user@hadoop.apache.org<mailto:user@hadoop.apache.org> Subject: Re: setting up Load balancer for name nodes Hi Kamaraj, I don't think the feature you are referring to exists in Hadoop 2.9.2. The feature was brought up by JIRA https://issues.apache.org/jira/browse/HADOOP-15707<https://issues.apache.org/jira/browse/HADOOP-15707> which is released in Hadoop 3.2+ (see "Fixed Version") Also, you may want to go through the Hadoop manual/doc for your current Hadoop version. If you change r3.3.0 to r2.9.2 in the URL you posted, you should be reading: https://hadoop.apache.org/docs/r2.9.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html<https://hadoop.apache.org/docs/r2.9.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html> Thanks, On Fri, Aug 21, 2020 at 11:02 AM Muthupandiyan, Kamaraj <kamaraj.muthupandi...@hpe.com<mailto:kamaraj.muthupandi...@hpe.com>> wrote: Hello team, I need to configure LB for our name node setup and we are using Hadoop 2.9.2 . I was going through the docs from https://hadoop.apache.org/docs/r3.3.0/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html<https://hadoop.apache.org/docs/r3.3.0/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html> and found the below option, but whenever I am using the HTTP endpoints as mentioned below I am getting 404 response If you are running a set of NameNodes behind a Load Balancer (e.g. Azure<https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-custom-probe-overview> or AWS<https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-healthchecks.html> ) and would like the Load Balancer to point to the active NN, you can use the /isActive HTTP endpoint as a health probe. http://NN_HOSTNAME/isActive<http://nn_hostname/isActive> will return a 200 status code response if the NN is in Active HA State, 405 otherwise. Regards, Kamaraj -- L
