Hello, I’m returning to an issue we’ve left hanging since July – we have now to fix Ambari on this cluster or take the whole cluster down and reinstall from scratch.
Our situation is that although our HDP 2.2 cluster is running well, Ambari cannot be used to install anything because the wizard is broken. I did a restart of Ambari server and agents per Artem, but without knowing exactly what changes to make to the postgres tables I’m reluctant to try that part. We also tried to add a new component (Spark) using the Ambari API instead of the wizard, but that also failed, as did trying to remove the Spark (again via the API) that had failed to install. We have 1.5T of monitoring data on this 4-node cluster that want to preserve. The cluster is dedicated to storing metrics in HBase via OpenTSDB and that is all it is used for. I just want to confirm with the group that since Ambari can only be used to manage a cluster that it installed itself, our best option in this scenario would be to: Shut down monitoring Copy all the data to another cluster Completely remove Ambari and HDP per https://cwiki.apache.org/confluence/display/AMBARI/Host+Cleanup+for+Ambari+and+Stack Do a fresh install of HDP 2.2 using the latest Ambari, and Copy the data back to the new cluster. Please let us know if this is a valid approach Thanks Ken From: <[email protected]<mailto:[email protected]>> on behalf of Artem Ervits <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Tuesday, July 28, 2015 at 12:48 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Any way to reset Ambari Install Wizard? try to restart ambari server and agents, then stop and start services, sometimes services need to announce themselves to Ambari that they're installed. Always refer to the ambari-server log. Worst case scenario, delete Ambari_metrics service with API and clean up the postgres DB manually, tables to concentrate on are hostservicedesiredstate, servicedesiredstate etc. This should be last resort. On Tue, Jul 28, 2015 at 3:11 PM, Benoit Perroud <[email protected]<mailto:[email protected]>> wrote: Some manual update in DB is most likely needed. *WARNING* use this at your own risk The table that needs to be updated is cluster_version. As far as I tested 2.1, it required less manual intervention than 2.0.1. Upgrade has a retry button for most of the steps, and this is really cool. Hope this help. Benoit 2015-07-28 20:01 GMT+02:00 Ken Barclay <[email protected]<mailto:[email protected]>>: Hello, I upgraded a small test cluster from HDP 2.1 to HDP 2.2 and Ambari 2.0.1. In following the steps to replace Nagios + Ganglia with the Ambari Metrics System using the Ambari Wizard, an install failure occurred on one node due to an outdated glibc library. I updated glibc and verified the Metrics packages could be installed, but couldn’t go back and finish the installation through the wizard. The problem is: it flags some of the default settings, saying they need to be changed, but it skips past the screen very quickly that enables those settings to be changed, without allowing anything to be entered. So the button that allows you to proceed with the installation never becomes enabled. I subsequently manually finished the Metrics installation using the Ambari API and have it running in Distributed mode. But Ambari’s wizard cannot be used for anything now: the same problem described above occurs for every service I try to install. Can Ambari be reset somehow in this situation, or do I need to reinstall it? Or do you recommend installing 2.1? Thanks Ken
