Lian – I assumed you were asking how to wait for all agents to register. How is Ansible bring triggered? If it’s running from a single playbook from one of your hosts than there is nothing that the client hosts need to wait for as the control host knows the cluster creation status.
If running automation independently on each host then you’ll need something to wait on from each host. That could simply by what you do on the ambari-server: * Poll the cluster creation status. Or, from Terraform use “remote-exec” provisioner which runs inside a “null_resource”: https://www.terraform.io/docs/provisioners/null_resource.html p.s. I agree that ambari would benefit from “pre/post-commands”. If there is not already a request in JIRA, make one. -- Sean Roberts @seano From: Lian Jiang <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Sunday, February 18, 2018 at 11:06 PM To: "[email protected]" <[email protected]> Subject: Re: How to run scripts on ambari client nodes after hadoop setup is done Sorry. Please help me understand. Polling /api/v1/hosts is assuming the clients will wait the server to be up. As mentioned above, the clients cannot do this since it can cause deadlock. The client scripts need to complete without waiting for anything so that all clients can be ready. Then the server waiting for the clients to be ready can continue. Plus, clients waiting for hosts available seems not to be helpful. They need to know for sure the hadoop cluster setup is done so that they can for example copy files to hdfs. Thanks. On Sun, Feb 18, 2018 at 1:12 PM, Sean Roberts <[email protected]<mailto:[email protected]>> wrote: Lian – Poll /api/v1/hosts until the appropriate number of hosts are available or be more strict by checking for the correct host names. Here is an ugly version of doing which checks the number of hosts required have registered (written in 2015 but still works): https://github.com/seanorama/ambari-bootstrap/blob/master/deploy/deploy-recommended-cluster.bash Or an ansible version: https://github.com/hortonworks/ansible-hortonworks/blob/master/playbooks/roles/ambari-config/tasks/main.yml#L97-L111 -- Sean Roberts @seano From: Lian Jiang <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Sunday, February 18, 2018 at 8:35 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: How to run scripts on ambari client nodes after hadoop setup is done Thanks Sean. In the 100% automated process of machine preparation and hadoop cluster setup, it is easy for ambari server to start the server and monitor. However the ambari clients cannot wait ambari server to start otherwise the server and client lock each other (ambari server waits for all clients to be ready while all clients wait for server to be up). How do you solve this issue? Thanks. On Sun, Feb 18, 2018 at 8:49 AM, Sean Roberts <[email protected]<mailto:[email protected]>> wrote: Yes. Best method is to monitor the cluster creation status and then trigger your post-actions. -- Sean Roberts ... blame the phone for typos > On 18 Feb 2018, at 03:30, Lian Jiang > <[email protected]<mailto:[email protected]>> wrote: > > Hi, > > I am using terraform and ambari blueprint to setup hadoop cluster in cloud. I > want to run scripts on the ambari client hosts to install something (e.g. > install ExtJS for ooize, add topology_script.py into hdfs conf folder...) > after the hadoop is setup automatically. > > What's the recommended way to do this? Should I make the bootstrap scripts > running on the ambari client nodes check the cluster creation status? > > http://{ambariServerIP}:8080/api/v1/clusters/${CLUSTER_NAME}/requests/1 > > Thanks for any clue. > >
