date:20160612

Re: Review Request 48490: Last button in the log search pagination panel does not take the user to the last page and few more fixes

2016-06-12 Thread Dharmesh Makwana


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48490/
---

(Updated June 13, 2016, 6:36 a.m.)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
Mohanty, and Sebastian Toader.


Changes
---

Fixed review comments.


Bugs: AMBARI-17140
https://issues.apache.org/jira/browse/AMBARI-17140


Repository: ambari


Description
---

Patch contains
1) Last button in the log search pagination panel does not take the user to the 
last page.
2) Solr warming feature added.
3) Suffix in columns mapping issue fixed.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
 917956f 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 147e148 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
 53e386e 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
 ca63671 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/MgrBase.java
 1d069d3 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigMgr.java
 3b23f2a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/QueryGeneration.java
 4647c7b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditREST.java
 92bfb01 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/DashboardREST.java
 1e107ed 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/ConfigUtil.java
 44d184d 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/view/VUserConfig.java
 55ec1c0 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties 
25b4f1a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/META-INF/applicationContext.xml
 6a73d60 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collections/BaseCollection.js
 24b6974 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/Intro.js 
869a1d4 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js
 5f93c5a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dialog/ApplySearchFilterView.js
 4babebc 

Diff: https://reviews.apache.org/r/48490/diff/


Testing
---

Setup Logsearch on 2 node cluster and tested the above feature.


Thanks,

Dharmesh Makwana

Re: Review Request 48395: AMBARI-17027: Metrics Collector API: Introduce basic series aggregation functions

2016-06-12 Thread Jungtaek Lim



> On 6 10, 2016, 11:56 오후, Aravindan Vijayan wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java,
> >  line 257
> > 
> >
> > Just throwing out a question here. 
> > 
> > Does it make any difference to do SUM(M1,M2) before or after applying 
> > post processing (rate/diff)? 
> > 
> > Ideally, these operations should be commutative (Order should not 
> > matter). 
> > 
> > Rate of Sum and Sum or Rate should be the same.

If it is timestamp aligned, it is commutative for SUM.

Let's say value of (M1, t1) to x1 and value of (M2, t1) to y1.

t1  x1  y1
t2  x2  y2
t3  x3  y3
t4  x4  y4

If we calculate sum first, and rate, 
t1 = 0.0
t2 = ((x2 + y2) - (x1 + y1)) / (t2 - t1) = (x2 + y2 - x1 - y1) / (t2 - t1)
t3 = ((x3 + y3) - (x2 + y2)) / (t3 - t2) = (x3 + y3 - x2 - y2) / (t3 - t2)
t4 = ((x4 + y4) - (x3 + y3)) / (t4 - t3) = (x4 + y4 - x3 - y3) / (t4 - t3)

If we calculate rate first, and sum, 
t1 = 0.0
t2 = ((x2 - x1) + (y2 - y1)) / (t2 - t1) = (x2 + y2 - x1 - y1) / (t2 - t1)
t3 = ((x3 - x2) + (y3 - y2)) / (t3 - t2) = (x3 + y3 - x2 - y2) / (t3 - t2)
t4 = ((x4 - x3) + (y4 - y3)) / (t4 - t3) = (x4 + y4 - x3 - y3) / (t4 - t3)

So they're same.

But if it is not timestamp aligned, it will be messed up. For example, 

t1  x1  y1
t2  y2
t3  x3 
t4  x4  y4

If we calculate sum first, and rate, 
t1 = 0.0
t2 = (y2 - (x1 + y1)) / (t2 - t1) = (y2 - x1 - y1) / (t2 - t1)
t3 = (x3 - y2) / (t3 - t2) = (x3 - y2) / (t3 - t2)
t4 = (x4 + y4 - x3) / (t4 - t3)

If we calculate rate first, and sum, 
t1 = 0.0
t2 = N/A + (y2 - y1) / (t2 - t1) = (y2 - y1) / (t2 - t1)
t3 = (x3 - x1) / (t3 - t1) + N/A = (x3 - x1) / (t3 - t1)
t4 = ((x4 - x3) / (t4 - t3)) + ((y4 - t2) / (t4 - t2))

So they're not same.

I guess 'interpolation' seems important for this side.


- Jungtaek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48395/#review137086
---


On 6 10, 2016, 12:50 오전, Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48395/
> ---
> 
> (Updated 6 10, 2016, 12:50 오전)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Prajwal Rao, 
> Sriharsha Chintalapani, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17027
> https://issues.apache.org/jira/browse/AMBARI-17027
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMS doesn't provide tag so metric is identified by appId, metric name, 
> hostname, instanceId. In this situation metric name is normally consist of 
> origin metric name and tag values, like graphite, but unlike Graphite, AMS 
> also doesn't provide series aggregation functions so aggregation should be 
> done from caller side.
> 
> It would be great if Ambari Metrics Collector provides series aggregation 
> functions, like sumSeries / 
> averageSeries / minSeries / maxSeries on Graphite.
> 
> Query outputs: 
> https://gist.github.com/HeartSaVioR/f4f28b5b8b7bf2e5477e59d7fd56090f
> 
> Attached Grafana screenshots to AMBARI-17027. Please refer 
> https://issues.apache.org/jira/browse/AMBARI-17027 for details.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js 7390aa8 
>   
> ambari-metrics/ambari-metrics-grafana/ambari-metrics/partials/query.editor.html
>  b034c03 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js 2eb3613 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  1b2d02f 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
>  e37bc4d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
>  7d49070 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/SeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-m

Re: Review Request 48590: AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected queue instead of existing for only 'llap' named queue.

2016-06-12 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48590/#review137228
---


Ship it!




Ship It!

- Sumit Mohanty


On June 12, 2016, 10:57 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48590/
> ---
> 
> (Updated June 12, 2016, 10:57 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17179
> https://issues.apache.org/jira/browse/AMBARI-17179
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected 
> queue instead of existing for only 'llap' named queue.
> 
>  - With this change, LLAP config calculation code will do calculations 
> for any queue selected (Given that selected queue capacity is enough to do 
> caluclations).
>  - Further, if any selected queue capacity is < minimum required queue 
> capacity for LLAP/HIVE2 to work, a validation error will pop up at the SAVE 
> operation.
>  Screenshot acttached.
> -> 2 validations added : 1. When selectd queue is in STOPPED state. 
> 2. When selected queue size is less than minimum required for LLAP app to 
> run. 
> -> This will serve as a feedback to the user to increase the capacity 
> of the selected queue or select a different queue for the operation.
> -> **Point to note :** A slider for changing the queue capacity will 
> only be available if 'llap' queue is selected. For any other queue selected 
>from dropdown, any capacity change related operation, user will 
> have to go via already known means of altering the capacity (eg: CapSched 
> View).
>
>
>
> - Also fixes the following issues :
> - 
>
>  - Normalizing up the calculated value for 'Slider AM Container Size' 
> with YARN Minimum Container Size in fn. updateLlapConfigs(). 
>  
>
>  - 'hive.llap.io.threadpool.size' config value is now set same as value 
> calculated for 'hive.llap.daemon.num.executors' at all times.
>  
>  
>  - Updates the 2.5/services/HIVE/themes/theme.json, to allow 
> 'llap_queue_capacity' slider visibility to be set via recommendation from 
> stack_advisor.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> c2fd8a7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  6f806d2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ac92b86 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/themes/theme.json
>  1922b79 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> df6c65c 
>   
> ambari-server/src/test/python/stacks/2.5/common/services-normal-his-2-hosts.json
>  dcf6d26 
>   
> ambari-server/src/test/python/stacks/2.5/common/services-normal-his-valid.json
>  69c7e55 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 5a54548 
> 
> Diff: https://reviews.apache.org/r/48590/diff/
> 
> 
> Testing
> ---
> 
> - Python UT's added and modifed.
> - Python UT passes.
> 
> 
> File Attachments
> 
> 
> Screen Shot 2016-06-12 at 3.46.48 PM.png
>   
> https://reviews.apache.org/media/uploaded/files/2016/06/12/9e2997f5-2d81-410a-b84a-1e313800e4c2__Screen_Shot_2016-06-12_at_3.46.48_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Review Request 48609: AMBARI-17183: client.properties for Falcon should be configurable via Ambari

2016-06-12 Thread Venkat Ranganathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48609/
---

Review request for Ambari.


Bugs: AMBARI-17183
https://issues.apache.org/jira/browse/AMBARI-17183


Repository: ambari


Description
---

Currently falcon-client.properties is generated without functionality for user 
changes.   We should fix it for users of mirroring etc


Diffs
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 5e25325 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 8c2ad8e 
  
ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/services/FALCON/package/scripts/falcon.py
 9a72af1 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-client.properties.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/48609/diff/


Testing
---


Thanks,

Venkat Ranganathan

Review Request 48607: AMBARI-17181: Add some of value-attributes to property files in AMBARI_METRICS

2016-06-12 Thread Masahiro Tanaka


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48607/
---

Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-17181
https://issues.apache.org/jira/browse/AMBARI-17181


Repository: ambari


Description
---

Some of the property files in AMBARI_METRICS lack value-attributes.
It would be nice to have value-attributes on most of the properties. If so, we 
can notice a careless mistakes in Ambari Server WebUI.


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-env.xml
 eaafc6b 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-ini.xml
 da4599e 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
 b40923a 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml
 f4e5fb2 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 6484285 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-ssl-server.xml
 6f9c6dc 

Diff: https://reviews.apache.org/r/48607/diff/


Testing
---

mvn clean test


Thanks,

Masahiro Tanaka

Re: Review Request 48590: AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected queue instead of existing for only 'llap' named queue.

2016-06-12 Thread Swapan Shridhar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48590/
---

(Updated June 12, 2016, 10:57 p.m.)

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit Mohanty.

Bugs: AMBARI-17179
https://issues.apache.org/jira/browse/AMBARI-17179

Repository: ambari

Description
---

- AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected
queue instead of existing for only 'llap' named queue.

- With this change, LLAP config calculation code will do calculations for
any queue selected (Given that selected queue capacity is enough to do
caluclations).
- Further, if any selected queue capacity is < minimum required queue
capacity for LLAP/HIVE2 to work, a validation error will pop up at the SAVE
operation.
Screenshot acttached.
-> 2 validations added : 1. When selectd queue is in STOPPED state. 2.
When selected queue size is less than minimum required for LLAP app to run.
-> This will serve as a feedback to the user to increase the capacity
of the selected queue or select a different queue for the operation.
-> **Point to note :** A slider for changing the queue capacity will
only be available if 'llap' queue is selected. For any other queue selected
from dropdown, any capacity change related operation, user will have
to go via already known means of altering the capacity (eg: CapSched View).

- Also fixes the following issues :
-

- Normalizing up the calculated value for 'Slider AM Container Size' with
YARN Minimum Container Size in fn. updateLlapConfigs().

- 'hive.llap.io.threadpool.size' config value is now set same as value
calculated for 'hive.llap.daemon.num.executors' at all times.

- Updates the 2.5/services/HIVE/themes/theme.json, to allow
'llap_queue_capacity' slider visibility to be set via recommendation from
stack_advisor.

Diffs (updated)
-

ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
c2fd8a7

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
6f806d2

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
ac92b86

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/themes/theme.json
1922b79
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
df6c65c

ambari-server/src/test/python/stacks/2.5/common/services-normal-his-2-hosts.json
dcf6d26

ambari-server/src/test/python/stacks/2.5/common/services-normal-his-valid.json
69c7e55
ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 5a54548

Diff: https://reviews.apache.org/r/48590/diff/

Testing
---

- Python UT's added and modifed.
- Python UT passes.

File Attachments

Screen Shot 2016-06-12 at 3.46.48 PM.png

https://reviews.apache.org/media/uploaded/files/2016/06/12/9e2997f5-2d81-410a-b84a-1e313800e4c2__Screen_Shot_2016-06-12_at_3.46.48_PM.png

Thanks,

Swapan Shridhar

Review Request 48590: AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected queue instead of existing for only 'llap' named queue.

2016-06-12 Thread Swapan Shridhar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48590/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit Mohanty.

Bugs: AMBARI-17179
https://issues.apache.org/jira/browse/AMBARI-17179

Repository: ambari

Description
---

- AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected
queue instead of existing for only 'llap' named queue.

- Also fixes the following issues :
-

- Normalizing up the calculated value for 'Slider AM Container Size' with
YARN Minimum Container Size in fn. updateLlapConfigs().

- 'hive.llap.io.threadpool.size' config value is now set same as value
calculated for 'hive.llap.daemon.num.executors' at all times.

- Updates the 2.5/services/HIVE/themes/theme.json, to allow
'llap_queue_capacity' slider visibility to be set via recommendation from
stack_advisor.

Diffs
-

ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
c2fd8a7

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
6f806d2

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
ac92b86

ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/themes/theme.json
1922b79
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
df6c65c

ambari-server/src/test/python/stacks/2.5/common/services-normal-his-2-hosts.json
dcf6d26

ambari-server/src/test/python/stacks/2.5/common/services-normal-his-valid.json
69c7e55
ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 5a54548

Diff: https://reviews.apache.org/r/48590/diff/

Testing
---

- Python UT's added and modifed.
- Python UT passes.

File Attachments

Screen Shot 2016-06-12 at 3.46.48 PM.png

https://reviews.apache.org/media/uploaded/files/2016/06/12/9e2997f5-2d81-410a-b84a-1e313800e4c2__Screen_Shot_2016-06-12_at_3.46.48_PM.png

Thanks,

Swapan Shridhar

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-12 Thread Jonathan Hurley



> On June 12, 2016, 7:06 a.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py, lines 
> > 410-411
> > 
> >
> > This still seems like a warning which we want. It means that there was 
> > a name service defined, but no alias properties. If a name service is 
> > defined, then shouldn't there always be valid aliases?
> 
> Miklos Gergely wrote:
> The uri block of some alerts contains a high_availability sub block, 
> which may define a nameservice. If HA is not enabled than the defined 
> property is not present. I don't see any logic to tell earlier whether or not 
> HA is enabled, so the fact that the property the HA nameservice references is 
> missing is a good indicator that it's not. If there is anoyther way to decide 
> if HA is enabled let me know about it, that logic could be used at 
> base_alert:342 to decide whether or not to enter into 
> _get_uri_from_ha_structure at all. Currently the logic there is to check if 
> ha_nameservice or ha_alias_key is defined, but the comment above it tells 
> that in this case it should "try" to get these properties, i.e. it is still 
> not sure that they are there. Following this logic their absence is not a 
> cause for warning.
> 
> Jonathan Hurley wrote:
> There is really no way to determine if HA is "enabled". That's why we use 
> a combinaton of the nameservice and the alias. If the nameservice is present 
> in the configs, then I think the aliases should be as well. I think the logic 
> as it is today is a little wrong. It should be more like:
> 
> ```
> if ha_nameservice is None:
>   return None
> 
> if ha_nameservice is not None and ha_alias_key is None:
>   return None
> 
> ha_alias_key = ha_alias_key.replace(self.HA_NAMESERVICE_PARAM, 
> ha_nameservice)
> ha_nameservice_alias = self._get_configuration_value(ha_alias_key)
> if ha_nameservice_alias is None:
>   logger.warning("[Alert][{0}] HA nameservice value is present but 
> there are no aliases for {1}".format(
> ```
> 
> Miklos Gergely wrote:
> This is exactly how it is right now, except that the function checks if 
> ha_nameservice is None before replacing the HA_NAMESERVICE_PARAM with it. So 
> if you are right, then there is no issue here, it works fine, so if HA is not 
> "enabled" then the log would be full of warnings about it's absence.
> 
> As I see the presence of a high_availability block in the alert means 
> that if HA is "enabled" then this is how it would affect the urls that the 
> alert should use. And the only way to tell if it is actually enabled is to 
> see if these propreties are there.
> 
> Miklos Gergely wrote:
> Sorry, I just realized that what you've proposed is actually different, 
> and it would solve the warning issue. So if you believe that both 
> ha_nameservice and ha_alias_key must not be None than this is a good fix, 
> although the unit tests suggest that it is valid to have only an ha_alias_key 
> without ha_namespace.

I don't think the unit tests are a good representation of what the correct data 
would be here. They can be changed. From my understanding, the namespace value 
is worthless without aliases as well. Which is why we originally produced a 
warning if we had one but could not resolve the fully-qualified HA keys using 
the alias.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/#review137164
---


On June 11, 2016, 6:24 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48589/
> ---
> 
> (Updated June 11, 2016, 6:24 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17180
> https://issues.apache.org/jira/browse/AMBARI-17180
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> base_alert.py puts a warning into the log if there are properties referenced 
> in the HA nameservice or the alias which are not present in the 
> configuration. The absence of these properties is an indicator that the HA is 
> not enabled, it is not a cause for warning.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
> 
> Diff: https://reviews.apache.org/r/48589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-12 Thread Miklos Gergely



> On June 12, 2016, 11:06 a.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py, lines 
> > 410-411
> > 
> >
> > This still seems like a warning which we want. It means that there was 
> > a name service defined, but no alias properties. If a name service is 
> > defined, then shouldn't there always be valid aliases?
> 
> Miklos Gergely wrote:
> The uri block of some alerts contains a high_availability sub block, 
> which may define a nameservice. If HA is not enabled than the defined 
> property is not present. I don't see any logic to tell earlier whether or not 
> HA is enabled, so the fact that the property the HA nameservice references is 
> missing is a good indicator that it's not. If there is anoyther way to decide 
> if HA is enabled let me know about it, that logic could be used at 
> base_alert:342 to decide whether or not to enter into 
> _get_uri_from_ha_structure at all. Currently the logic there is to check if 
> ha_nameservice or ha_alias_key is defined, but the comment above it tells 
> that in this case it should "try" to get these properties, i.e. it is still 
> not sure that they are there. Following this logic their absence is not a 
> cause for warning.
> 
> Jonathan Hurley wrote:
> There is really no way to determine if HA is "enabled". That's why we use 
> a combinaton of the nameservice and the alias. If the nameservice is present 
> in the configs, then I think the aliases should be as well. I think the logic 
> as it is today is a little wrong. It should be more like:
> 
> ```
> if ha_nameservice is None:
>   return None
> 
> if ha_nameservice is not None and ha_alias_key is None:
>   return None
> 
> ha_alias_key = ha_alias_key.replace(self.HA_NAMESERVICE_PARAM, 
> ha_nameservice)
> ha_nameservice_alias = self._get_configuration_value(ha_alias_key)
> if ha_nameservice_alias is None:
>   logger.warning("[Alert][{0}] HA nameservice value is present but 
> there are no aliases for {1}".format(
> ```
> 
> Miklos Gergely wrote:
> This is exactly how it is right now, except that the function checks if 
> ha_nameservice is None before replacing the HA_NAMESERVICE_PARAM with it. So 
> if you are right, then there is no issue here, it works fine, so if HA is not 
> "enabled" then the log would be full of warnings about it's absence.
> 
> As I see the presence of a high_availability block in the alert means 
> that if HA is "enabled" then this is how it would affect the urls that the 
> alert should use. And the only way to tell if it is actually enabled is to 
> see if these propreties are there.

Sorry, I just realized that what you've proposed is actually different, and it 
would solve the warning issue. So if you believe that both ha_nameservice and 
ha_alias_key must not be None than this is a good fix, although the unit tests 
suggest that it is valid to have only an ha_alias_key without ha_namespace.


- Miklos


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/#review137164
---


On June 11, 2016, 10:24 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48589/
> ---
> 
> (Updated June 11, 2016, 10:24 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17180
> https://issues.apache.org/jira/browse/AMBARI-17180
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> base_alert.py puts a warning into the log if there are properties referenced 
> in the HA nameservice or the alias which are not present in the 
> configuration. The absence of these properties is an indicator that the HA is 
> not enabled, it is not a cause for warning.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
> 
> Diff: https://reviews.apache.org/r/48589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-12 Thread Miklos Gergely



> On June 12, 2016, 11:06 a.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py, lines 
> > 410-411
> > 
> >
> > This still seems like a warning which we want. It means that there was 
> > a name service defined, but no alias properties. If a name service is 
> > defined, then shouldn't there always be valid aliases?
> 
> Miklos Gergely wrote:
> The uri block of some alerts contains a high_availability sub block, 
> which may define a nameservice. If HA is not enabled than the defined 
> property is not present. I don't see any logic to tell earlier whether or not 
> HA is enabled, so the fact that the property the HA nameservice references is 
> missing is a good indicator that it's not. If there is anoyther way to decide 
> if HA is enabled let me know about it, that logic could be used at 
> base_alert:342 to decide whether or not to enter into 
> _get_uri_from_ha_structure at all. Currently the logic there is to check if 
> ha_nameservice or ha_alias_key is defined, but the comment above it tells 
> that in this case it should "try" to get these properties, i.e. it is still 
> not sure that they are there. Following this logic their absence is not a 
> cause for warning.
> 
> Jonathan Hurley wrote:
> There is really no way to determine if HA is "enabled". That's why we use 
> a combinaton of the nameservice and the alias. If the nameservice is present 
> in the configs, then I think the aliases should be as well. I think the logic 
> as it is today is a little wrong. It should be more like:
> 
> ```
> if ha_nameservice is None:
>   return None
> 
> if ha_nameservice is not None and ha_alias_key is None:
>   return None
> 
> ha_alias_key = ha_alias_key.replace(self.HA_NAMESERVICE_PARAM, 
> ha_nameservice)
> ha_nameservice_alias = self._get_configuration_value(ha_alias_key)
> if ha_nameservice_alias is None:
>   logger.warning("[Alert][{0}] HA nameservice value is present but 
> there are no aliases for {1}".format(
> ```

This is exactly how it is right now, except that the function checks if 
ha_nameservice is None before replacing the HA_NAMESERVICE_PARAM with it. So if 
you are right, then there is no issue here, it works fine, so if HA is not 
"enabled" then the log would be full of warnings about it's absence.

As I see the presence of a high_availability block in the alert means that if 
HA is "enabled" then this is how it would affect the urls that the alert should 
use. And the only way to tell if it is actually enabled is to see if these 
propreties are there.


- Miklos


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/#review137164
---


On June 11, 2016, 10:24 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48589/
> ---
> 
> (Updated June 11, 2016, 10:24 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17180
> https://issues.apache.org/jira/browse/AMBARI-17180
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> base_alert.py puts a warning into the log if there are properties referenced 
> in the HA nameservice or the alias which are not present in the 
> configuration. The absence of these properties is an indicator that the HA is 
> not enabled, it is not a cause for warning.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
> 
> Diff: https://reviews.apache.org/r/48589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-12 Thread Jonathan Hurley



> On June 12, 2016, 7:06 a.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py, lines 
> > 410-411
> > 
> >
> > This still seems like a warning which we want. It means that there was 
> > a name service defined, but no alias properties. If a name service is 
> > defined, then shouldn't there always be valid aliases?
> 
> Miklos Gergely wrote:
> The uri block of some alerts contains a high_availability sub block, 
> which may define a nameservice. If HA is not enabled than the defined 
> property is not present. I don't see any logic to tell earlier whether or not 
> HA is enabled, so the fact that the property the HA nameservice references is 
> missing is a good indicator that it's not. If there is anoyther way to decide 
> if HA is enabled let me know about it, that logic could be used at 
> base_alert:342 to decide whether or not to enter into 
> _get_uri_from_ha_structure at all. Currently the logic there is to check if 
> ha_nameservice or ha_alias_key is defined, but the comment above it tells 
> that in this case it should "try" to get these properties, i.e. it is still 
> not sure that they are there. Following this logic their absence is not a 
> cause for warning.

There is really no way to determine if HA is "enabled". That's why we use a 
combinaton of the nameservice and the alias. If the nameservice is present in 
the configs, then I think the aliases should be as well. I think the logic as 
it is today is a little wrong. It should be more like:

```
if ha_nameservice is None:
  return None

if ha_nameservice is not None and ha_alias_key is None:
  return None

ha_alias_key = ha_alias_key.replace(self.HA_NAMESERVICE_PARAM, ha_nameservice)
ha_nameservice_alias = self._get_configuration_value(ha_alias_key)
if ha_nameservice_alias is None:
  logger.warning("[Alert][{0}] HA nameservice value is present but there 
are no aliases for {1}".format(
```


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/#review137164
---


On June 11, 2016, 6:24 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48589/
> ---
> 
> (Updated June 11, 2016, 6:24 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17180
> https://issues.apache.org/jira/browse/AMBARI-17180
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> base_alert.py puts a warning into the log if there are properties referenced 
> in the HA nameservice or the alias which are not present in the 
> configuration. The absence of these properties is an indicator that the HA is 
> not enabled, it is not a cause for warning.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
> 
> Diff: https://reviews.apache.org/r/48589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-12 Thread Miklos Gergely

> On June 12, 2016, 11:06 a.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py, lines 
> > 410-411
> > 
> >
> > This still seems like a warning which we want. It means that there was 
> > a name service defined, but no alias properties. If a name service is 
> > defined, then shouldn't there always be valid aliases?

The uri block of some alerts contains a high_availability sub block, which may 
define a nameservice. If HA is not enabled than the defined property is not 
present. I don't see any logic to tell earlier whether or not HA is enabled, so 
the fact that the property the HA nameservice references is missing is a good 
indicator that it's not. If there is anoyther way to decide if HA is enabled 
let me know about it, that logic could be used at base_alert:342 to decide 
whether or not to enter into _get_uri_from_ha_structure at all. Currently the 
logic there is to check if ha_nameservice or ha_alias_key is defined, but the 
comment above it tells that in this case it should "try" to get these 
properties, i.e. it is still not sure that they are there. Following this logic 
their absence is not a cause for warning.

- Miklos

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/#review137164
---

On June 11, 2016, 10:24 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48589/
> ---
> 
> (Updated June 11, 2016, 10:24 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17180
> https://issues.apache.org/jira/browse/AMBARI-17180
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> base_alert.py puts a warning into the log if there are properties referenced 
> in the HA nameservice or the alias which are not present in the 
> configuration. The absence of these properties is an indicator that the HA is 
> not enabled, it is not a cause for warning.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
> 
> Diff: https://reviews.apache.org/r/48589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 48395: AMBARI-17027: Metrics Collector API: Introduce basic series aggregation functions

2016-06-12 Thread Jungtaek Lim



> On 6 10, 2016, 11:56 오후, Aravindan Vijayan wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java,
> >  line 73
> > 
> >
> > When the data is not timestamp aligned, this will lead to "difficult to 
> > comprehend" graphs.
> > 
> > For example, let's take Sum of Metrics M1,M2,M3 where
> > M1 - t1 - 5.0
> > M1 - t2 - 5.0
> > M2 - t2 - 5.0
> > M3 - t3 - 5.0
> > 
> > The graph will show up like 
> > (t1, 5.0) -> (t2,10.0) -> (t3,5.0)
> > 
> > That is why we do some "interpolation" while we aggregate data.

Yeah, I agree that output could be odd when data is not timestamp aligned. 
I didn't think about this side while working, and also I'm not expert on this.
How AMS interpolate points when aggregating? Does it apply linear interpolation?


- Jungtaek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48395/#review137086
---


On 6 10, 2016, 12:50 오전, Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48395/
> ---
> 
> (Updated 6 10, 2016, 12:50 오전)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Prajwal Rao, 
> Sriharsha Chintalapani, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17027
> https://issues.apache.org/jira/browse/AMBARI-17027
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMS doesn't provide tag so metric is identified by appId, metric name, 
> hostname, instanceId. In this situation metric name is normally consist of 
> origin metric name and tag values, like graphite, but unlike Graphite, AMS 
> also doesn't provide series aggregation functions so aggregation should be 
> done from caller side.
> 
> It would be great if Ambari Metrics Collector provides series aggregation 
> functions, like sumSeries / 
> averageSeries / minSeries / maxSeries on Graphite.
> 
> Query outputs: 
> https://gist.github.com/HeartSaVioR/f4f28b5b8b7bf2e5477e59d7fd56090f
> 
> Attached Grafana screenshots to AMBARI-17027. Please refer 
> https://issues.apache.org/jira/browse/AMBARI-17027 for details.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js 7390aa8 
>   
> ambari-metrics/ambari-metrics-grafana/ambari-metrics/partials/query.editor.html
>  b034c03 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js 2eb3613 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  1b2d02f 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
>  e37bc4d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
>  7d49070 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/SeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunctionFactory.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAvgAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMaxAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMinAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservic

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

2016-06-12 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48348/
---

(Updated June 12, 2016, 12:16 p.m.)


Review request for Ambari, Andrew Onischuk, Don Bosco Durai, Miklos Gergely, 
Robert Nettleton, Sumit Mohanty, and Sebastian Toader.


Changes
---

- fetch_nonlocal_groups from configuration


Bugs: AMBARI-17089
https://issues.apache.org/jira/browse/AMBARI-17089


Repository: ambari


Description
---

- Change logfeeder process/files to use sudo user instead of 
logfeeder/logfeeder user/group (to make sure logfeeder can read any kind of the 
logs).
- solr and logsearch user both moved to hadoop group


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent b38b872 
  
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
 73619a3 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 2525d9f 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 8b7120a 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 0b6e7ab 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 93cf30d 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
 56ce721 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 ff048b4 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
 c0689f3 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 7acdec2 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
 5ca2bd5 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 890e8c6 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 6e71334 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
bfd07b2 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py 981319c 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py bfe6921 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 0590dca 
  ambari-server/src/test/python/stacks/2.4/configs/default.json c3aecff 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 7127451 
  ambari-server/src/test/python/stacks/2.5/configs/default.json 1015593 
  ambari-web/app/data/HDP2/site_properties.js 794da25 

Diff: https://reviews.apache.org/r/48348/diff/


Testing
---

FT: tested locally with 4 node cluster with umask 0027.


Thanks,

Oliver Szabo

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

2016-06-12 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48348/#review137165
---




ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
 (line 42)


Should be from config in 
config['configurations']['cluster-env']["fetch_nonlocal_groups"]


- Andrew Onischuk


On June 12, 2016, 11:52 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48348/
> ---
> 
> (Updated June 12, 2016, 11:52 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Don Bosco Durai, Miklos Gergely, 
> Robert Nettleton, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17089
> https://issues.apache.org/jira/browse/AMBARI-17089
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Change logfeeder process/files to use sudo user instead of 
> logfeeder/logfeeder user/group (to make sure logfeeder can read any kind of 
> the logs).
> - solr and logsearch user both moved to hadoop group
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent b38b872 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  73619a3 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  2525d9f 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  09a86f2 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  0b6e7ab 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  93cf30d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
>  56ce721 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  ff048b4 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
>  c0689f3 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  7acdec2 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
>  5ca2bd5 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  890e8c6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  6e71334 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> bfd07b2 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py 
> 981319c 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> bfe6921 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 0590dca 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 7127451 
>   ambari-server/src/test/python/stacks/2.5/configs/default.json 1015593 
>   ambari-web/app/data/HDP2/site_properties.js 794da25 
> 
> Diff: https://reviews.apache.org/r/48348/diff/
> 
> 
> Testing
> ---
> 
> FT: tested locally with 4 node cluster with umask 0027.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

2016-06-12 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48348/
---

(Updated June 12, 2016, 11:52 a.m.)


Review request for Ambari, Andrew Onischuk, Don Bosco Durai, Miklos Gergely, 
Robert Nettleton, Sumit Mohanty, and Sebastian Toader.


Changes
---

- be consistent with ldap envs


Bugs: AMBARI-17089
https://issues.apache.org/jira/browse/AMBARI-17089


Repository: ambari


Description
---

- Change logfeeder process/files to use sudo user instead of 
logfeeder/logfeeder user/group (to make sure logfeeder can read any kind of the 
logs).
- solr and logsearch user both moved to hadoop group


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent b38b872 
  
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
 73619a3 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 2525d9f 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 09a86f2 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 0b6e7ab 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 93cf30d 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
 56ce721 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 ff048b4 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
 c0689f3 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 7acdec2 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
 5ca2bd5 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 890e8c6 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 6e71334 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
bfd07b2 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py 981319c 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py bfe6921 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 0590dca 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 7127451 
  ambari-server/src/test/python/stacks/2.5/configs/default.json 1015593 
  ambari-web/app/data/HDP2/site_properties.js 794da25 

Diff: https://reviews.apache.org/r/48348/diff/


Testing
---

FT: tested locally with 4 node cluster with umask 0027.


Thanks,

Oliver Szabo

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-12 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/#review137164
---




ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 


This still seems like a warning which we want. It means that there was a 
name service defined, but no alias properties. If a name service is defined, 
then shouldn't there always be valid aliases?


- Jonathan Hurley


On June 11, 2016, 6:24 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48589/
> ---
> 
> (Updated June 11, 2016, 6:24 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17180
> https://issues.apache.org/jira/browse/AMBARI-17180
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> base_alert.py puts a warning into the log if there are properties referenced 
> in the HA nameservice or the alias which are not present in the 
> configuration. The absence of these properties is an indicator that the HA is 
> not enabled, it is not a cause for warning.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
> 
> Diff: https://reviews.apache.org/r/48589/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

2016-06-12 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48348/#review137158
---


Ship it!




Thanks Oviler!

- Andrew Onischuk


On June 11, 2016, 6:17 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48348/
> ---
> 
> (Updated June 11, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Don Bosco Durai, Miklos Gergely, 
> Robert Nettleton, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17089
> https://issues.apache.org/jira/browse/AMBARI-17089
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Change logfeeder process/files to use sudo user instead of 
> logfeeder/logfeeder user/group (to make sure logfeeder can read any kind of 
> the logs).
> - solr and logsearch user both moved to hadoop group
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent b38b872 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  73619a3 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  2525d9f 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  09a86f2 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  0b6e7ab 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  93cf30d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
>  56ce721 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  ff048b4 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
>  c0689f3 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  7acdec2 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
>  5ca2bd5 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  890e8c6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  6e71334 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> bfd07b2 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py 
> 981319c 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> bfe6921 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 0590dca 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 7127451 
>   ambari-server/src/test/python/stacks/2.5/configs/default.json 1015593 
>   ambari-web/app/data/HDP2/site_properties.js 794da25 
> 
> Diff: https://reviews.apache.org/r/48348/diff/
> 
> 
> Testing
> ---
> 
> FT: tested locally with 4 node cluster with umask 0027.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 48490: Last button in the log search pagination panel does not take the user to the last page and few more fixes

Re: Review Request 48395: AMBARI-17027: Metrics Collector API: Introduce basic series aggregation functions

Re: Review Request 48590: AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected queue instead of existing for only 'llap' named queue.

Review Request 48609: AMBARI-17183: client.properties for Falcon should be configurable via Ambari

Review Request 48607: AMBARI-17181: Add some of value-attributes to property files in AMBARI_METRICS

Re: Review Request 48590: AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected queue instead of existing for only 'llap' named queue.

Review Request 48590: AMBARI-17179. Allow 'LLAP related calculations' to be done for any selected queue instead of existing for only 'llap' named queue.

Re: Review Request 48589: Fix HA enabled logic in the alerts

Re: Review Request 48589: Fix HA enabled logic in the alerts

Re: Review Request 48589: Fix HA enabled logic in the alerts

Re: Review Request 48589: Fix HA enabled logic in the alerts

Re: Review Request 48589: Fix HA enabled logic in the alerts

Re: Review Request 48395: AMBARI-17027: Metrics Collector API: Introduce basic series aggregation functions

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

Re: Review Request 48589: Fix HA enabled logic in the alerts

Re: Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder with umask 0027

18 matches

Site Navigation

Mail list logo

Footer information