Great. I'm glad you have it working. Yes, "hostname -f" and "hostname" commands should resolve to FQDN and short name, respectively, as you mentioned. There's a number of issues that arise when this is not set up as such.
Yusaku On Tue, Sep 23, 2014 at 2:08 PM, . . <[email protected]> wrote: > Sid and Yusaku: > > Thank you both for your inputs as they allowed me to identify the cause and > resolve the issue which appears to have been primarily a result of host > configuration that was not returning the FQDN when checked with hostname -f > as recommended in the install docs. The metrics started showing once the > host configuration on namenode and datanodes was corrected by updating the > /etc/hosts file. > > Additional detail: > - This cluster is setup with the namenode, which serves as the Ambari and > Ganglia host as well, being publicly accessible and datanodes only being > accessible from the namenode > - The hosts on the Hadoop cluster-local network were configured with private > IPs and ".localdomain" as the domainname > - /etc/sysconfig/network contained the FQDN, and the pre-issue-resolution > /etc/hosts config files contained "local-IP <hostname> > <hostname>.localdomain" entries which resulted in 'hostname' and 'hostname > -f' only returning the base hostname > - updating /etc/hosts to contain entries of the form "local-IP > <hostname>.localdomain <hostname>" (FQDN first) resulted in hostname -f > returning FQDN > - all metrics, including Heatmaps, now populate and display as designed > > Thanks again! > >> Date: Tue, 23 Sep 2014 13:18:11 -0700 >> Subject: Re: Not all metrics are populating >> From: [email protected] >> To: [email protected] > >> >> Sid, >> >> > 16:13:28,435 ERROR [pool-1-thread-6163] JMXPropertyProvider:540 - Caught >> > exception getting JMX metrics : Server returned HTTP response code: 500 for >> > URL: http://namenode.localdomain:8745/api/cluster/summary >> >> This is Ambari Server trying to get metrics from Storm REST API >> endpoint (I presume it's just co-hosted with NameNode). This error >> should be independent of HFDS/YARN metrics not coming through in >> Ambari. >> >> Yusaku >> >> On Tue, Sep 23, 2014 at 8:38 AM, Siddharth Wagle <[email protected]> >> wrote: >> > The JMX URL posted in the log looks wrong. >> > What is the value of "dfs.namenode.http-address" in your hdf-site.xml ? >> > And, can you access : http://namenode.localdomain:8745/jmx, from your >> > browser? >> > >> > Make sure the Ganglia host resolves from your ambari host by executing >> > the >> > URL's that you see in the ambari server log from the ambari host using >> > "curl >> > -u admin:admin <url-smaple>" >> > >> > -Sid >> > >> > >> > >> > >> > >> > On Tue, Sep 23, 2014 at 7:51 AM, . . <[email protected]> wrote: >> >> >> >> [Ambari 1.6.1, HDP 2.1, CentOS 6.5, multi-node cluster] >> >> >> >> On a fresh install, not all metrics are getting populated in Ambari. >> >> >> >> Some of the missing metrics include: >> >> >> >> Dashboard: Namenode CPU WIO (displays "n/a" instead) >> >> HDFS: All metrics except HDFS I/O (displays "No Data. There was no data >> >> available Possible reasons include inaccessible Ganglia service") >> >> YARN: All metrics (displays "No Data. There was no data available >> >> Possible >> >> reasons include inaccessible Ganglia service") >> >> >> >> >> >> The ideas suggested in the following two threads have not helped to >> >> resolve the issue: >> >> >> >> Hortonworks forum post >> >> Ambari mailing list post >> >> >> >> >> >> ambari-server.log shows the following warnings and errors >> >> (/var/log/messages has no Ganglia specific errors): >> >> >> >> 16:12:52,905 WARN [Thread-1] HeartbeatMonitor:126 - Heartbeat lost from >> >> host localhost.localdomain >> >> 16:13:28,435 ERROR [pool-1-thread-6163] JMXPropertyProvider:540 - >> >> Caught >> >> exception getting JMX metrics : Server returned HTTP response code: 500 >> >> for >> >> URL: http://namenode.localdomain:8745/api/cluster/summary >> >> >> >> >> >> Any suggestions on how to identify the cause and resolve this issue ? >> >> >> >> Thanks. >> >> >> >> Note: Cross-posted to Hortonworks forums. >> > >> > >> > >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or entity >> > to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the >> > reader of >> > this message is not the intended recipient, you are hereby notified that >> > any >> > printing, copying, dissemination, distribution, disclosure or forwarding >> > of >> > this communication is strictly prohibited. If you have received this >> > communication in error, please contact the sender immediately and delete >> > it >> > from your system. Thank You. >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> immediately >> and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
