Review Request 51583: Add ability to add custom grok filters and custom grok patterns with Ambari

2016-09-01 Thread Miklos Gergely 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51583/
---

Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.


Bugs: AMBARI-18298
https://issues.apache.org/jira/browse/AMBARI-18298


Repository: ambari


Description
---

Allow the operators to view the logsearch grok patterns and add new patterns to 
them


Diffs
-

  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-grok.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-log4j.xml
 81e17d8 
  ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml 
e0280b4 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 5c3954f 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
 d3851d3 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/grok-patterns.j2
 49aa357 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-default_grok_patterns.j2
 PRE-CREATION 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py f4dbd63 
  ambari-server/src/test/python/stacks/2.4/configs/default.json a016ce0 
  ambari-web/app/data/HDP2/site_properties.js 1fad1d6 

Diff: https://reviews.apache.org/r/51583/diff/


Testing
---

Tested on local cluster.


Thanks,

Miklos Gergely

Review Request 51587: View for Apache HAWQ

2016-09-01 Thread Stuart Pollock 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51587/
---

Review request for Ambari, Alexander Denissov and Lav Jain.


Bugs: AMBARI-18297
https://issues.apache.org/jira/browse/AMBARI-18297


Repository: ambari


Description
---

View for Apache HAWQ


Diffs
-

  contrib/views/hawq/.gitignore PRE-CREATION 
  contrib/views/hawq/README.md PRE-CREATION 
  contrib/views/hawq/pom.xml PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/HAWQViewServlet.java
 PRE-CREATION 
  contrib/views/hawq/src/main/resources/WEB-INF/web.xml PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/.bowerrc PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/.editorconfig PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/.ember-cli PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/.gitignore PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/.jshintrc PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/adapters/query.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/app.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/components/query-table.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/controllers/.gitkeep 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/helpers/.gitkeep PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/index.html PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/models/query.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/resolver.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/router.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/routes/application.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/routes/main.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/serializers/query.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/styles/app.scss PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/templates/application.hbs 
PRE-CREATION 
  
contrib/views/hawq/src/main/resources/ui/app/templates/components/query-table.hbs
 PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/templates/main.hbs PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/app/utils/utils.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/bower.json PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/config/environment.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/ember-cli-build.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/package.json PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/public/assets/logo-hawq.png 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/testem.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/.jshintrc PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/acceptance/application-test.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/helpers/destroy-app.js 
PRE-CREATION 
  
contrib/views/hawq/src/main/resources/ui/tests/helpers/module-for-acceptance.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/helpers/resolver.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/helpers/start-app.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/helpers/test-helper.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/index.html PRE-CREATION 
  
contrib/views/hawq/src/main/resources/ui/tests/integration/components/query-table-test.js
 PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/test-helper.js PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/unit/adapters/query-test.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/unit/models/query-test.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/unit/routes/main-test.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/unit/serializers/query-test.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/tests/unit/utils/utils-test.js 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/ui/vendor/.gitkeep PRE-CREATION 
  contrib/views/hawq/src/main/resources/view.xml PRE-CREATION 

Diff: https://reviews.apache.org/r/51587/diff/


Testing
---

Manually Tested on live cluster

Rat Check passed
Unit tests passed


Thanks,

Stuart Pollock

Review Request 51570: Enable log search for NIFI_CA component

2016-09-01 Thread Miklos Gergely 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51570/
---

Review request for Ambari, Jayush Luniya, Oliver Szabo, Robert Nettleton, and 
Sumit Mohanty.


Bugs: AMBARI-18293
https://issues.apache.org/jira/browse/AMBARI-18293


Repository: ambari


Description
---

Add NIFI_CA log to logsearch


Diffs
-

  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/HadoopServiceConfig.json.j2
 9ce3fde 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-nifi.json.j2
 99f1740 

Diff: https://reviews.apache.org/r/51570/diff/


Testing
---

Tested on local cluster


Thanks,

Miklos Gergely

Re: Review Request 51570: Enable log search for NIFI_CA component

2016-09-01 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51570/#review147601
---


Ship it!




Ship It!

- Oliver Szabo


On Sept. 1, 2016, 8:44 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51570/
> ---
> 
> (Updated Sept. 1, 2016, 8:44 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Oliver Szabo, Robert Nettleton, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18293
> https://issues.apache.org/jira/browse/AMBARI-18293
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add NIFI_CA log to logsearch
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/HadoopServiceConfig.json.j2
>  9ce3fde 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-nifi.json.j2
>  99f1740 
> 
> Diff: https://reviews.apache.org/r/51570/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 51562: AMBARI-18289 : Invalid negative values for some AMS metrics

2016-09-01 Thread Aravindan Vijayan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51562/
---

(Updated Sept. 1, 2016, 6:11 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18289
https://issues.apache.org/jira/browse/AMBARI-18289


Repository: ambari


Description
---

PROBLEM
Negative values are being reported for IOPS in the "system servers" dashboard

BUG
This was a dockerized environment. Negative rate values were seen because 
read/write counters were dropping below the previous values at random times. On 
further investigation, it revealed that this was due to the docker volume 
groups present on the host. It is expected of docker containers to add/remove 
the volume groups during the lifecyle of a container. So, when a volume group 
goes away, the read/write counters do not contribute to the total counter 
values, thus making the value go below the last seen value.

FIX
Have a provision to discard such "special" disk partitions through a skip 
pattern. If skipped, they will not contribute to the global counter metric. 
Individual disk counter metrics can be used to get disk specific counter values.


Diffs (updated)
-

  ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
3e5d861 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
 02f0ce3 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py 
845b270 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
 4135d32 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
 2503c43 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
 383a0de 

Diff: https://reviews.apache.org/r/51562/diff/


Testing
---

Manually tested.


Thanks,

Aravindan Vijayan

Review Request 51578: AMBARI-18071: Ambari Files View needs to have ability to load security configurations

2016-09-01 Thread Keta Patel 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51578/
---

Review request for Ambari, Di Li and DIPAYAN BHOWMICK.


Bugs: AMBARI-18071
https://issues.apache.org/jira/browse/AMBARI-18071


Repository: ambari


Description
---

When HDFS is configured with Encryption Zones, Files View to browse files will 
give "No KeyProvider" error.

Steps to reproduce this issue:
1. Configure an encrypted zone in HDFS (Transparent Data Encryption). I have 
used Hadoop's KMS (installed tar manually).
2. Create a Files View instance and provide a user/group with the privilege to 
use the instance.
3. Log into the Ambari console as the user with the Files View permission.
4. Open the Files View instance.
5. Go to the folder which is configured as an encrypted zone.
6. Try to open an existing file in this folder.
7. This throws an error - java.io.IOException: No KeyProvider is configured, 
cannot access an encrypted file. 
8. When trying through the shell, opening this file works.

This happens because Files View doesn't have enough configuration set to browse 
secured zone. Files view doesn't even provide an option to add this 
configuration. This is why we see errors "No KeyProvider is configured, cannot 
access an encrypted file". 
To work around this, you could download client configuration from HDFS service 
tab, and copy the core-site.xml and hdfs-site.xml files to 
/etc/ambari-server/conf, then restart ambari-server. After this, the user is 
able to open the file in the encrypted zone. Basically, the property 
"dfs.encryption.key.provider.uri" which provides details of the KeyProvider, is 
obtained from the hdfs-site.xml present in the /etc/ambari-server/conf folder.


Diffs
-

  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
 d505e9a 

Diff: https://reviews.apache.org/r/51578/diff/


Testing
---

**CAUSE:**
==
The error of "No KeyProvider is configured" is seen only for those cases when 
the HDFS uses DistributedFileSystem for its communication. When HDFS uses 
WebHDFSFileSystem for communication, this error is not seen and the Ambari View 
instance is able to open the files in the encrypted zones.

Why Ambari Views use either Distributed or WebHDFS file systems is explained 
below:
Ambari views can be created using one of the 3 modes of configuration:
1. Local cluster
2. Remote cluster
3. Custom configuration (no cluster is associated here).

The HDFS works through abstraction. For Ambari Views, the actual file system 
used during execution depends on whether the view instance was created using a 
Local/Remote cluster or using Custom configuration. For instances created using 
Local/Remote cluster, HDFS uses Distributed File System and for instances 
created using Custom configuration, HDFS uses WebHDFSFileSystem.
WebHDFSFileSystem is an integrated part of the HDFS ecosystem. It is aware of 
all the HDFS configuration. For this reason, when a KMS is configured in HDFS, 
WebHDFSFileSystem is aware of the KeyProvider and no special config mapping is 
needed. Thus, even the view instance created using Custom configuration doesn't 
need any special configuration and can talk to the Encryption Zones 
successfully.

However, for view instances created using Local/Remote cluster configuration, 
HDFS uses the Distributed FileSystem. This Distributed FileSystem works as an 
HDFS client and hence, is not fully aware of all the HDFS configuration. We 
need to explicitly provide HDFS properties like 
"dfs.encryption.key.provider.uri" to these ambari view instances to provide 
details of the KeyProvider. The proposed fix helps in providing this property 
value to the view as follows.

**FIX:**

The proposed fix (attached as "AMBARI-18071.patch") checks if the current view 
instance configuration has any cluster associated in its context. If there is 
an associated cluster then the instance has a Local/Remote cluster 
configuration and needs to be provided with the HDFS KeyProvider information. 
Otherwise, the WebHDFSFileSystem will take care of the KeyProvider if KMS is 
configured.

To provide the property information, the parseProperties() in 
ConfigurationBuilder.java looked best as we also set the defaultFS property 
here. If a cluster is associated with the context, and if the property 
"dfs.encryption.key.provider.uri" is not null, then this property is set in the 
Configuration object and thus made available to Distributed file system of HDFS.
The Ambari VIew instance works successfully with both Local and Remote 
configurations.

One more point to note in the configuration aspect is the addition of proxyuser 
to the kms-site.xml for the ambari-server daemon. Without this proxyuser even 
the custom configuration will not work. (I had installed hadoop's KMS on the 
ambari-server manually)

Review Request 51579: Database Consistency Check Fails With NPE With Missing Service From Stack

2016-09-01 Thread Vitalyi Brodetskyi 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51579/
---

Review request for Ambari, Jonathan Hurley and Sumit Mohanty.


Bugs: AMBARI-18296
https://issues.apache.org/jira/browse/AMBARI-18296


Repository: ambari


Description
---

The database consistency checker can fail if a service configuration exists for 
a service which is no longer on the current stack:


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 9f7419c 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
 d8f538e 

Diff: https://reviews.apache.org/r/51579/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 51562: AMBARI-18289 : Invalid negative values for some AMS metrics

2016-09-01 Thread Aravindan Vijayan 


> On Sept. 1, 2016, 4:59 a.m., Alejandro Fernandez wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py,
> >  line 266
> > 
> >
> > Where is allowed_disks populated?

In line 274


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51562/#review147542
---


On Sept. 1, 2016, 3:51 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51562/
> ---
> 
> (Updated Sept. 1, 2016, 3:51 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18289
> https://issues.apache.org/jira/browse/AMBARI-18289
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> Negative values are being reported for IOPS in the "system servers" dashboard
> 
> BUG
> This was a dockerized environment. Negative rate values were seen because 
> read/write counters were dropping below the previous values at random times. 
> On further investigation, it revealed that this was due to the docker volume 
> groups present on the host. It is expected of docker containers to add/remove 
> the volume groups during the lifecyle of a container. So, when a volume group 
> goes away, the read/write counters do not contribute to the total counter 
> values, thus making the value go below the last seen value.
> 
> FIX
> Have a provision to discard such "special" disk partitions through a skip 
> pattern. If skipped, they will not contribute to the global counter metric. 
> Individual disk counter metrics can be used to get disk specific counter 
> values.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
> 3e5d861 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  02f0ce3 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py
>  845b270 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
>  4135d32 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  2503c43 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
>  383a0de 
> 
> Diff: https://reviews.apache.org/r/51562/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 51452: AMBARI-18261: CapSched View: Showing warning icons to refresh or restart capsched once after configs are saved

2016-09-01 Thread Akhil PB 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51452/
---

(Updated Sept. 1, 2016, 11:36 a.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and Sreenath 
Somarajapuram.


Changes
---

Patch updated with review comments


Bugs: AMBARI-18261
https://issues.apache.org/jira/browse/AMBARI-18261


Repository: ambari


Description
---

Showing warning icons to refresh or restart capsched once after configs are 
saved
Bug fixes


Diffs (updated)
-

  contrib/views/capacity-scheduler/src/main/resources/ui/app/adapters.js 
f300b5b 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/capacityInput.js
 b86719f 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/editLabelCapacity.js
 ed22c2b 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/editQueueCapacity.js
 b244097 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/labelCapacityBar.js
 89e7cac 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueMapping.js
 8118805 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueSummary.js
 c15f100 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/saveConfigDialog.js
 ea5262d 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/advanced.js
 edd8a3b 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
 dd46b2c 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
 a68a2aa 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/scheduler.js
 6638b4c 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/models/queue.js 
acf6465 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/router.js 0b13a8a 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/store.js 01a7d6a 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
 195b55d 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/advanced.hbs
 a9ef778 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/accessControlList.hbs
 1f6fbda 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/labelCapacity.hbs
 9073910 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/preemption.hbs
 73a9e21 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueCapacity.hbs
 7b25af3 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueResources.hbs
 262f380 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf.hbs
 1bc3380 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/scheduler.hbs
 7649512 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/editQueueCapacity.hbs
 a7901b8 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueMapping.hbs
 fa35518 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueSummary.hbs
 9c3ffd0 

Diff: https://reviews.apache.org/r/51452/diff/


Testing
---

Manual Unit Testing


Thanks,

Akhil PB

Review Request 51572: Ambari Server Start/Stop fails on Centos 7.1+

2016-09-01 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51572/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18294
https://issues.apache.org/jira/browse/AMBARI-18294


Repository: ambari


Description
---

Brand new install on Centos 7.2
Ambari Server 'setup' completed successfully.

[root@c7001 ~]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
No errors were found.
Ambari database consistency check finished
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start
Ambari Server 'start' completed successfully.

[root@c7001 ~]# cat /var/run/ambari-server/ambari-server.pid
12302
12303

[root@c7001 ~]# ps aux | grep AmbariServer
root 12302  0.0  0.0 113116   644 pts/0S20:42   0:00 /bin/sh -c 
ulimit -n 1 ; /usr/jdk64/jdk1.8.0_77/bin/java -server -XX:NewRatio=3 
-XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit 
-XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled 
-Dsun.zip.disableMemoryMapping=true   -Xms512m -Xmx2048m -XX:MaxPermSize=128m 
-Djava.security.auth.login.config=$ROOT/etc/ambari-server/conf/krb5JAASLogin.conf
 -Djava.security.krb5.conf=/etc/krb5.conf 
-Djavax.security.auth.useSubjectCredsOnly=false -cp 
'/etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar'
 org.apache.ambari.server.controller.AmbariServer > 
/var/log/ambari-server/ambari-server.out 2>&1 || echo $? > 
/var/run/ambari-server/ambari-server.exitcode &
root 12303 87.7 14.8 4377224 431856 pts/0  Sl   20:42   0:29 
/usr/jdk64/jdk1.8.0_77/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC 
-XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 
-XX:+CMSClassUnloadingEnabled -Dsun.zip.disableMemoryMapping=true -Xms512m 
-Xmx2048m -XX:MaxPermSize=128m 
-Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf 
-Djava.security.krb5.conf=/etc/krb5.conf 
-Djavax.security.auth.useSubjectCredsOnly=false -cp 
/etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar
 org.apache.ambari.server.controller.AmbariServer


Diffs
-

  ambari-server/src/main/python/ambari_server/utils.py c4990f5 
  ambari-server/src/main/python/ambari_server_main.py 17945de 

Diff: https://reviews.apache.org/r/51572/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 51572: Ambari Server Start/Stop fails on Centos 7.1+

2016-09-01 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51572/#review147550
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Sept. 1, 2016, 12:10 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51572/
> ---
> 
> (Updated Sept. 1, 2016, 12:10 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18294
> https://issues.apache.org/jira/browse/AMBARI-18294
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Brand new install on Centos 7.2
> Ambari Server 'setup' completed successfully.
> 
> [root@c7001 ~]# ambari-server start
> Using python  /usr/bin/python
> Starting ambari-server
> Ambari Server running with administrator privileges.
> Organizing resource files at /var/lib/ambari-server/resources...
> Ambari database consistency check started...
> No errors were found.
> Ambari database consistency check finished
> Server PID at: /var/run/ambari-server/ambari-server.pid
> Server out at: /var/log/ambari-server/ambari-server.out
> Server log at: /var/log/ambari-server/ambari-server.log
> Waiting for server start
> Ambari Server 'start' completed successfully.
> 
> [root@c7001 ~]# cat /var/run/ambari-server/ambari-server.pid
> 12302
> 12303
> 
> [root@c7001 ~]# ps aux | grep AmbariServer
> root 12302  0.0  0.0 113116   644 pts/0S20:42   0:00 /bin/sh -c 
> ulimit -n 1 ; /usr/jdk64/jdk1.8.0_77/bin/java -server -XX:NewRatio=3 
> -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit 
> -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled 
> -Dsun.zip.disableMemoryMapping=true   -Xms512m -Xmx2048m -XX:MaxPermSize=128m 
> -Djava.security.auth.login.config=$ROOT/etc/ambari-server/conf/krb5JAASLogin.conf
>  -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -cp 
> '/etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar'
>  org.apache.ambari.server.controller.AmbariServer > 
> /var/log/ambari-server/ambari-server.out 2>&1 || echo $? > 
> /var/run/ambari-server/ambari-server.exitcode &
> root 12303 87.7 14.8 4377224 431856 pts/0  Sl   20:42   0:29 
> /usr/jdk64/jdk1.8.0_77/bin/java -server -XX:NewRatio=3 
> -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit 
> -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled 
> -Dsun.zip.disableMemoryMapping=true -Xms512m -Xmx2048m -XX:MaxPermSize=128m 
> -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -cp 
> /etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar
>  org.apache.ambari.server.controller.AmbariServer
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/utils.py c4990f5 
>   ambari-server/src/main/python/ambari_server_main.py 17945de 
> 
> Diff: https://reviews.apache.org/r/51572/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 51588: AMBARI-18299: Support open files ulimit for ambari-agent

2016-09-01 Thread Nahappan Somasundaram 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51588/
---

Review request for Ambari, Jonathan Hurley and Sumit Mohanty.


Bugs: AMBARI-18299
https://issues.apache.org/jira/browse/AMBARI-18299


Repository: ambari


Description
---

AMBARI-18299: Support open files ulimit for ambari-agent

** Fix **: Added ulimit.open.files property to ambari-agent.ini.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/AmbariConfig.py 
ae938dcd2a94a01fbff0c6412a95910bddf210d7 
  ambari-agent/src/main/python/ambari_agent/main.py 
d7864b3bfb24b0f0f671686b2ec506708c16bd8c 
  ambari-agent/src/test/python/ambari_agent/TestAmbariConfig.py 
78f29fe019e59eea8444d5326246822653e732ec 
  ambari-agent/src/test/python/ambari_agent/TestMain.py 
23cdb13d2d6ec9c6a0343fceb50d2f917eae02e0 

Diff: https://reviews.apache.org/r/51588/diff/


Testing
---

Deployed a VM with the changes and tested manually by providing a value for 
ulimit.open.files and verified that the limit was set. If the property is not 
present, no action is taken.


Thanks,

Nahappan Somasundaram

Review Request 51575: Provide stack-advisor support for Microsoft-R service

2016-09-01 Thread Attila Doroszlai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51575/
---

Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-17891
https://issues.apache.org/jira/browse/AMBARI-17891


Repository: ambari


Description
---

Update existing service_advisor.py for Microsoft-R service to conform to the 
updated interface (changed in AMBARI-16663).  Also fix leftover comment.


Diffs
-

  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/service_advisor.py
 554be2d 

Diff: https://reviews.apache.org/r/51575/diff/


Testing
---

Manual testing on local cluster (created cluster with HDFS, YARN, and 
ZooKeeper; added Microsoft-R service via web UI; verified that service advisor 
is invoked in both recommend-component-layout and validate-component-layout 
requests, and that it correctly recommends/validates service colocation).


Thanks,

Attila Doroszlai

Re: Review Request 51452: AMBARI-18261: CapSched View: Showing warning icons to refresh or restart capsched once after configs are saved

2016-09-01 Thread Akhil PB 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51452/
---

(Updated Sept. 1, 2016, 1:28 p.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and Sreenath 
Somarajapuram.


Changes
---

Patch updated


Bugs: AMBARI-18261
https://issues.apache.org/jira/browse/AMBARI-18261


Repository: ambari


Description
---

Showing warning icons to refresh or restart capsched once after configs are 
saved
Bug fixes


Diffs (updated)
-

  contrib/views/capacity-scheduler/src/main/resources/ui/app/adapters.js 
f300b5b 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/capacityInput.js
 b86719f 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/editLabelCapacity.js
 ed22c2b 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/editQueueCapacity.js
 b244097 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/labelCapacityBar.js
 89e7cac 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueMapping.js
 8118805 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueSummary.js
 c15f100 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/saveConfigDialog.js
 ea5262d 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/advanced.js
 edd8a3b 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
 dd46b2c 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
 a68a2aa 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/scheduler.js
 6638b4c 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/models/queue.js 
acf6465 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/router.js 0b13a8a 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/store.js 01a7d6a 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
 195b55d 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/advanced.hbs
 a9ef778 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/accessControlList.hbs
 1f6fbda 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/labelCapacity.hbs
 9073910 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/preemption.hbs
 73a9e21 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueCapacity.hbs
 7b25af3 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueResources.hbs
 262f380 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf.hbs
 1bc3380 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/scheduler.hbs
 7649512 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/editQueueCapacity.hbs
 a7901b8 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueMapping.hbs
 fa35518 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueSummary.hbs
 9c3ffd0 

Diff: https://reviews.apache.org/r/51452/diff/


Testing
---

Manual Unit Testing


Thanks,

Akhil PB

Re: Review Request 51575: Provide stack-advisor support for Microsoft-R service

2016-09-01 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51575/#review147556
---




contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/service_advisor.py
 (line 37)


Is there any unit test framework for m-pack? If so please add unit tests 
for this class.


- Sebastian Toader


On Sept. 1, 2016, 3:36 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51575/
> ---
> 
> (Updated Sept. 1, 2016, 3:36 p.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Sumit Mohanty, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17891
> https://issues.apache.org/jira/browse/AMBARI-17891
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update existing service_advisor.py for Microsoft-R service to conform to the 
> updated interface (changed in AMBARI-16663).  Also fix leftover comment.
> 
> 
> Diffs
> -
> 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/service_advisor.py
>  554be2d 
> 
> Diff: https://reviews.apache.org/r/51575/diff/
> 
> 
> Testing
> ---
> 
> Manual testing on local cluster (created cluster with HDFS, YARN, and 
> ZooKeeper; added Microsoft-R service via web UI; verified that service 
> advisor is invoked in both recommend-component-layout and 
> validate-component-layout requests, and that it correctly 
> recommends/validates service colocation).
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-01 Thread Tim Thorpe 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147557
---




ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java (line 
138)


The Optional class is only in JDK8. Currently Ambari is supposed to work 
with both JDK7 and 8.


- Tim Thorpe


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
>

Re: Review Request 51557: status commands get all the *-env files irrespective of a service exists or not

2016-09-01 Thread Vitalyi Brodetskyi 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51557/
---

(Updated Вер. 1, 2016, 2:59 після полудня)


Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Dmytro Sen, and 
Sumit Mohanty.


Bugs: AMBARI-18287
https://issues.apache.org/jira/browse/AMBARI-18287


Repository: ambari


Description
---

Ambari should not send env files corresponding to the deleted services for the 
status commands.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
 c353049 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
 a5396d8 

Diff: https://reviews.apache.org/r/51557/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 51557: status commands get all the *-env files irrespective of a service exists or not

2016-09-01 Thread Vitalyi Brodetskyi 


> On Вер. 1, 2016, 2:13 після полудня, Sumit Mohanty wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java,
> >  line 269
> > 
> >
> > Can we add a unit test?

Done


- Vitalyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51557/#review147558
---


On Вер. 1, 2016, 2:59 після полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51557/
> ---
> 
> (Updated Вер. 1, 2016, 2:59 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Dmytro Sen, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18287
> https://issues.apache.org/jira/browse/AMBARI-18287
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari should not send env files corresponding to the deleted services for 
> the status commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
>  c353049 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
>  a5396d8 
> 
> Diff: https://reviews.apache.org/r/51557/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 51562: AMBARI-18289 : Invalid negative values for some AMS metrics

2016-09-01 Thread Dmytro Sen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51562/#review147555
---




ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
 (line 85)


Is "true" a correct default value for pattern ?


- Dmytro Sen


On Сен. 1, 2016, 3:51 д.п., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51562/
> ---
> 
> (Updated Сен. 1, 2016, 3:51 д.п.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18289
> https://issues.apache.org/jira/browse/AMBARI-18289
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> Negative values are being reported for IOPS in the "system servers" dashboard
> 
> BUG
> This was a dockerized environment. Negative rate values were seen because 
> read/write counters were dropping below the previous values at random times. 
> On further investigation, it revealed that this was due to the docker volume 
> groups present on the host. It is expected of docker containers to add/remove 
> the volume groups during the lifecyle of a container. So, when a volume group 
> goes away, the read/write counters do not contribute to the total counter 
> values, thus making the value go below the last seen value.
> 
> FIX
> Have a provision to discard such "special" disk partitions through a skip 
> pattern. If skipped, they will not contribute to the global counter metric. 
> Individual disk counter metrics can be used to get disk specific counter 
> values.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
> 3e5d861 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  02f0ce3 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py
>  845b270 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
>  4135d32 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  2503c43 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
>  383a0de 
> 
> Diff: https://reviews.apache.org/r/51562/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 51557: status commands get all the *-env files irrespective of a service exists or not

2016-09-01 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51557/#review147558
---




ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
 (line 269)


Can we add a unit test?


- Sumit Mohanty


On Aug. 31, 2016, 12:27 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51557/
> ---
> 
> (Updated Aug. 31, 2016, 12:27 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Dmytro Sen, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18287
> https://issues.apache.org/jira/browse/AMBARI-18287
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari should not send env files corresponding to the deleted services for 
> the status commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
>  c353049 
> 
> Diff: https://reviews.apache.org/r/51557/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-01 Thread Tim Thorpe 


> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > I was under the impression that applying an m-pack would DIRECTLY update 
> > repoinfo.xml on the filesystem.  Has the design changed since then?
> > 
> > What is the purpose of carrying service version around with the repo?  When 
> > updating m-packs, that just means that there's that much more work to do to 
> > get it right.
> > 
> > How are you going to prevent an m-pack from overwriting an existing repo? 
> > (What if an m-pack names a repo "HDP-UTILS" or something - then they just 
> > blew over ours)?
> > 
> > You need to fix the code formatting so it's consistent and a bit easier to 
> > read.
> > 
> > In general, I don't like the idea of using service name to tie a repo to a 
> > service.  Maybe name this "reference" or a different indicator of where 
> > this repo came from.
> 
> Jayush Luniya wrote:
> RE: I was under the impression that applying an m-pack would DIRECTLY 
> update repoinfo.xml on the filesystem.
> Personally, it would be better if we can avoid updates to the stack's 
> repoinfo.xml and have the support to define repos at service level.
> 
> Balázs Bence Sári wrote:
> My implementation is based on this comment to the Jira (by Srimanth):
> 
> "Discussed with Nate Cole and Jayush Luniya, and the approach for 
> custom-services to specify their own repo location will be to provide a 
> /repos/repoinfo.xml inside the stack-version they will be in. This repo file 
> will be loaded by Ambari during startup into the 
> /api/v1/stacks/HDP/versions/2.4/repository_versions repos. Service repo files 
> have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> mpacks/custom-services/8.0.0/repos which will be linked into the stacks/ 
> folder.
> 
> ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos"
> 
> Overwriting a stack repo is prohibited in the implementation.
> 
> service_name and service_version come from Srimanth, I can rename or 
> discard them. What should happen with them? Should I delete service_version 
> and rename service_name to reference? My two cents are that it's worth 
> keeping one to indicate that the repo does not come with the stack.
> 
> Nate Cole wrote:
> Association should be with the service itself (I think that's what Jayush 
> is saying), and we can access those as part of the stack endpoint for the 
> service, but should not be integrated with the stack definition.  We should 
> then include that repo info when we create the tasks to agents.
> 
> Balázs Bence Sári wrote:
> I think that's a bit different than how it was in the Jira comment (at 
> least to my interpetation) which I looked like a result of consensus. I can 
> implement it however. 
> 
> Also, we also need to make sure the user can overwrite those repo url's 
> when installing a cluster (Microsoft wants no public repo, repo locatinon in 
> the final mpack will be empty, corporates are expected to create their 
> private repos). With the current implementation, 
> api/v1/stacks/HDP/versions/2.3/operating_systems/redhat6 will return the 
> service repo along the stack repos and the user can override it on the UI. If 
> I move the repo to the service, should this API call still return it? 
> (guessing that UI uses this API to get the repos).
> 
> Nate Cole wrote:
> We can aggregate them in the API response, but I don't know that they 
> should be stored with the stack repos.

I added stack extensions 
(https://cwiki.apache.org/confluence/display/AMBARI/Extensions) for the purpose 
of allowing shared resources for a collection of custom services.  The way you 
are implementing service level repositories seems wrong.  If you have multiple 
services that require a custom repository, then you are randomly selecting 
which service's repoinfo.xml you will use.  If it really is a shared 
repository, it should be included at the extension level.

In general, I think we should be encouraging third parties to deliver their 
custom services in a consistent way (extensions installed with mpacks).


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
>

Re: Review Request 51452: AMBARI-18261: CapSched View: Showing warning icons to refresh or restart capsched once after configs are saved

2016-09-01 Thread Sreenath Somarajapuram 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51452/#review147561
---


Ship it!




Ship It!

- Sreenath Somarajapuram


On Sept. 1, 2016, 1:28 p.m., Akhil PB wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51452/
> ---
> 
> (Updated Sept. 1, 2016, 1:28 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and 
> Sreenath Somarajapuram.
> 
> 
> Bugs: AMBARI-18261
> https://issues.apache.org/jira/browse/AMBARI-18261
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Showing warning icons to refresh or restart capsched once after configs are 
> saved
> Bug fixes
> 
> 
> Diffs
> -
> 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/adapters.js 
> f300b5b 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/capacityInput.js
>  b86719f 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/editLabelCapacity.js
>  ed22c2b 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/editQueueCapacity.js
>  b244097 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/labelCapacityBar.js
>  89e7cac 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueMapping.js
>  8118805 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueSummary.js
>  c15f100 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/saveConfigDialog.js
>  ea5262d 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/advanced.js
>  edd8a3b 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
>  dd46b2c 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
>  a68a2aa 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/scheduler.js
>  6638b4c 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/models/queue.js 
> acf6465 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/router.js 
> 0b13a8a 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/store.js 01a7d6a 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
>  195b55d 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/advanced.hbs
>  a9ef778 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/accessControlList.hbs
>  1f6fbda 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/labelCapacity.hbs
>  9073910 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/preemption.hbs
>  73a9e21 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueCapacity.hbs
>  7b25af3 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueResources.hbs
>  262f380 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf.hbs
>  1bc3380 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/scheduler.hbs
>  7649512 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/editQueueCapacity.hbs
>  a7901b8 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueMapping.hbs
>  fa35518 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueSummary.hbs
>  9c3ffd0 
> 
> Diff: https://reviews.apache.org/r/51452/diff/
> 
> 
> Testing
> ---
> 
> Manual Unit Testing
> 
> 
> Thanks,
> 
> Akhil PB
> 
>

Re: Review Request 51370: DB consistency check should tolerate services with no configuration

2016-09-01 Thread Balazs Bence Sari 
+ Jayush

Hey Nate,

It has not yet been committed.

@Jayush: Should we commit it to trunk only or to 2.5 as well?

Bence

From: Nate Cole > 
on behalf of Nate Cole >
Reply-To: Nate Cole >
Date: Wednesday, August 31, 2016 at 22:34
To: Sebastian Toader >, 
Nate Cole >, Laszlo Puskas 
>, Jonathan Hurley 
>, Sumit Mohanty 
>, Oliver Szabo 
>
Cc: default >, Ambari 
>
Subject: Re: Review Request 51370: DB consistency check should tolerate 
services with no configuration

This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/51370/


What's the state of this review?  If it's pushed can it be closed?


- Nate Cole


On August 25th, 2016, 7:26 a.m. EDT, Balázs Bence Sári wrote:

Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Oliver 
Szabo, Sumit Mohanty, and Sebastian Toader.
By Balázs Bence Sári.

Updated Aug. 25, 2016, 7:26 a.m.

Bugs: AMBARI-18254
Repository: ambari
Description

Services with no configs triggered an error in DB consistency checker during 
Ambari server start. Since the Microsoft R mpack has no configs at all, 
installing the Microsoft R mpack would result in error during server start. 
This patch lowers the check to warning level.


Testing

  1.  Tested manually

  2.  Wrote a new unit tests

  3.  All unit tests in ambari-servers passed.



Diffs

  *   
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 (36a2d99)
  *   
ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
 (a7c77e1)

View Diff

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-01 Thread Balázs Bence Sári 


> On Sept. 1, 2016, 2:10 p.m., Tim Thorpe wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java, 
> > line 138
> > 
> >
> > The Optional class is only in JDK8. Currently Ambari is supposed to 
> > work with both JDK7 and 8.

This is Guava's Optional. I will fix the wildcard imports so it will be visible.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147557
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>

Re: Review Request 51557: status commands get all the *-env files irrespective of a service exists or not

2016-09-01 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51557/#review147567
---


Ship it!




Ship It!

- Sumit Mohanty


On Sept. 1, 2016, 2:59 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51557/
> ---
> 
> (Updated Sept. 1, 2016, 2:59 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Dmytro Sen, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18287
> https://issues.apache.org/jira/browse/AMBARI-18287
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari should not send env files corresponding to the deleted services for 
> the status commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
>  c353049 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
>  a5396d8 
> 
> Diff: https://reviews.apache.org/r/51557/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>