Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-20 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/
---

(Updated June 20, 2016, 1:28 p.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

- fix more tests


Bugs: AMBARI-17294
https://issues.apache.org/jira/browse/AMBARI-17294


Repository: ambari


Description
---

- use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
working with that too)
- rery logic for znode creation and remove ignoreFailure from zkcli commands


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 5ace399 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 368db03 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 eac60db 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 b86a09b 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
f627066 
  ambari-server/src/test/python/stacks/2.3/configs/default.json d8fb398 
  ambari-server/src/test/python/stacks/2.3/configs/secure.json 338f048 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py ebf563d 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 6d226a6 
  ambari-server/src/test/python/stacks/2.5/configs/default.json de05d76 

Diff: https://reviews.apache.org/r/48854/diff/


Testing
---

ambari server python tests passed.


Thanks,

Oliver Szabo

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-20 Thread Miklos Gergely 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/#review138578
---


Ship it!




Ship It!

- Miklos Gergely


On June 20, 2016, 10:42 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48854/
> ---
> 
> (Updated June 20, 2016, 10:42 a.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17294
> https://issues.apache.org/jira/browse/AMBARI-17294
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
> working with that too)
> - rery logic for znode creation and remove ignoreFailure from zkcli commands
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  1abf2c1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  b86a09b 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> de355ea 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> ebf563d 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 8729195 
> 
> Diff: https://reviews.apache.org/r/48854/diff/
> 
> 
> Testing
> ---
> 
> ambari server python tests passed.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-20 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/
---

(Updated June 20, 2016, 10:42 a.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

- updated tests


Bugs: AMBARI-17294
https://issues.apache.org/jira/browse/AMBARI-17294


Repository: ambari


Description
---

- use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
working with that too)
- rery logic for znode creation and remove ignoreFailure from zkcli commands


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 1abf2c1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 368db03 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 eac60db 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 b86a09b 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
de355ea 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py ebf563d 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 8729195 

Diff: https://reviews.apache.org/r/48854/diff/


Testing
---

ambari server python tests passed.


Thanks,

Oliver Szabo

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-20 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/#review138575
---


Ship it!




Ship It!

- Sebastian Toader


On June 20, 2016, 11:52 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48854/
> ---
> 
> (Updated June 20, 2016, 11:52 a.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17294
> https://issues.apache.org/jira/browse/AMBARI-17294
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
> working with that too)
> - rery logic for znode creation and remove ignoreFailure from zkcli commands
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  1abf2c1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  b86a09b 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> de355ea 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> ebf563d 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 8729195 
> 
> Diff: https://reviews.apache.org/r/48854/diff/
> 
> 
> Testing
> ---
> 
> ambari server python tests passed.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-20 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/
---

(Updated June 20, 2016, 9:52 a.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

- smaller interval with 30 retries


Bugs: AMBARI-17294
https://issues.apache.org/jira/browse/AMBARI-17294


Repository: ambari


Description
---

- use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
working with that too)
- rery logic for znode creation and remove ignoreFailure from zkcli commands


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 1abf2c1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 368db03 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 eac60db 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 b86a09b 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
de355ea 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py ebf563d 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 8729195 

Diff: https://reviews.apache.org/r/48854/diff/


Testing
---

ambari server python tests passed.


Thanks,

Oliver Szabo

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-20 Thread Oliver Szabo 


> On June 20, 2016, 8:03 a.m., Sebastian Toader wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py,
> >  lines 124-125
> > 
> >
> > Why the retry params are passed instead of using any of the rety 
> > annotations directly on the upload_configuration_to_zk function?

in logsearch-solr-client, there is an inner retry logic, so i used them (the 
default is about max 50 seconds, it can be too low if we install with 
blueprints and solr is on a different host from atlas/ranger/logsearch)


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/#review138542
---


On June 17, 2016, 6:38 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48854/
> ---
> 
> (Updated June 17, 2016, 6:38 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17294
> https://issues.apache.org/jira/browse/AMBARI-17294
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
> working with that too)
> - rery logic for znode creation and remove ignoreFailure from zkcli commands
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  1abf2c1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  b86a09b 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> de355ea 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> ebf563d 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 8729195 
> 
> Diff: https://reviews.apache.org/r/48854/diff/
> 
> 
> Testing
> ---
> 
> ambari server python tests passed.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-20 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/#review138542
---




ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 (lines 124 - 125)


Why the retry params are passed instead of using any of the rety 
annotations directly on the upload_configuration_to_zk function?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 (line 115)


Why the retry params are passed instead of using any of the rety 
annotations directly on the upload_configuration_to_zk function?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 (lines 97 - 103)


In case there is a persisten problem than this function will spend 30 x 15 
= 450 seconds (~ 8 mins).

Do we really need to wait 8 mins for Zk to come up? Usually Zk is started 
among the first components on the nodes as it doesn't have much dependecies to 
wait for.

By looking at Zk logs how much time it takes for the quorum to be formed 
and accept requests?



ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 (line 551)


Why the retry params are passed instead of using any of the rety 
annotations directly on the upload_configuration_to_zk function?


- Sebastian Toader


On June 17, 2016, 8:38 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48854/
> ---
> 
> (Updated June 17, 2016, 8:38 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17294
> https://issues.apache.org/jira/browse/AMBARI-17294
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
> working with that too)
> - rery logic for znode creation and remove ignoreFailure from zkcli commands
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  1abf2c1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  b86a09b 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> de355ea 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> ebf563d 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 8729195 
> 
> Diff: https://reviews.apache.org/r/48854/diff/
> 
> 
> Testing
> ---
> 
> ambari server python tests passed.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-17 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/
---

(Updated June 17, 2016, 6:38 p.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

- update tests


Bugs: AMBARI-17294
https://issues.apache.org/jira/browse/AMBARI-17294


Repository: ambari


Description
---

- use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
working with that too)
- rery logic for znode creation and remove ignoreFailure from zkcli commands


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 1abf2c1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 368db03 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 eac60db 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 b86a09b 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
de355ea 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py ebf563d 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 8729195 

Diff: https://reviews.apache.org/r/48854/diff/


Testing
---

ambari server python tests passed.


Thanks,

Oliver Szabo

Re: Review Request 48854: Logsearch: retry logic for znode creation during Solr startup

2016-06-17 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48854/
---

(Updated June 17, 2016, 6:26 p.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

- add inner retry logic for logsearch/ranger/atlas configuration upload (becase 
of bluprint installs ...it is possible that logsearch server and solr are on 
different host, then when solr have stil not created znode, 
logsearch/atlas/ranger tries to upload solrconfig.xml files to the znode)


Bugs: AMBARI-17294
https://issues.apache.org/jira/browse/AMBARI-17294


Repository: ambari


Description
---

- use zookeeper quorum instead of zookeeper hosts (although zkcli looks like 
working with that too)
- rery logic for znode creation and remove ignoreFailure from zkcli commands


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 1abf2c1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 368db03 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 eac60db 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 b86a09b 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 

Diff: https://reviews.apache.org/r/48854/diff/


Testing
---

ambari server python tests passed.


Thanks,

Oliver Szabo