Re: Review Request 54477: Update zeppelin configuration for ambari 2.5

[Prabhjyot Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54477/
---

(Updated Dec. 14, 2016, 7:27 a.m.)


Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Jayush Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and Sumit 
Mohanty.


Changes
---

Replace tab character with 4 spaces


Bugs: AMBARI-19123
https://issues.apache.org/jira/browse/AMBARI-19123


Repository: ambari


Description
---

Update zeppelin configuration for ambari 2.5, and add more configuration 
template for shiro.ini


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
 c03d2dc 

Diff: https://reviews.apache.org/r/54477/diff/


Testing
---

Manually on Centos 6.6


Thanks,

Prabhjyot Singh

Re: Review Request 54477: Update zeppelin configuration for ambari 2.5

[Renjith Kamath]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54477/#review159118
---


Ship it!




Ship It!

- Renjith Kamath


On Dec. 14, 2016, 6:50 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54477/
> ---
> 
> (Updated Dec. 14, 2016, 6:50 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Jayush Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19123
> https://issues.apache.org/jira/browse/AMBARI-19123
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update zeppelin configuration for ambari 2.5, and add more configuration 
> template for shiro.ini
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
>  c03d2dc 
> 
> Diff: https://reviews.apache.org/r/54477/diff/
> 
> 
> Testing
> ---
> 
> Manually on Centos 6.6
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 54477: Update zeppelin configuration for ambari 2.5

[Prabhjyot Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54477/
---

(Updated Dec. 14, 2016, 6:50 a.m.)


Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Jayush Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and Sumit 
Mohanty.


Bugs: AMBARI-19123
https://issues.apache.org/jira/browse/AMBARI-19123


Repository: ambari


Description
---

Update zeppelin configuration for ambari 2.5, and add more configuration 
template for shiro.ini


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
 c03d2dc 

Diff: https://reviews.apache.org/r/54477/diff/


Testing
---

Manually on Centos 6.6


Thanks,

Prabhjyot Singh

Re: Review Request 53686: Stage and Request status should be persisted in the database

[Jaimin Jetly]

> On Nov. 14, 2016, 5:03 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java,
> >  lines 226-230
> > 
> >
> > Won't be performant on large numbers of tasks (like during upgrade).

This code is not part of the new revision of the patch.

With the new work iterating all hostrolecommands of the stage happens in very 
minimal scenarios where entire stage status cannot be figured out from subset 
of updated host role command received from agent as command reports.

For example if the stage current status is non-completed and a particular 
command report received has all tasks in it to be completed state then all host 
role commands of a stage will be iterated. But if any of the state is found to 
be in HOLDING, FAILED, TIMEDOUT or any of in progress states then only comamnds 
in the command report received are iterated.


- Jaimin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53686/#review155819
---


On Dec. 14, 2016, 2:11 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53686/
> ---
> 
> (Updated Dec. 14, 2016, 2:11 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18868
> https://issues.apache.org/jira/browse/AMBARI-18868
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Stage and Request status should be persisted in the database.
> 
> upgrading to ambari-3.0.0 should add status for all present stages and 
> request for the cluster.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  2c87583 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  e80b020 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java
>  3656bfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Request.java
>  31e11c1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> 4a05b32 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3c415df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/TaskCreateEvent.java
>  PRE-CREATION 
>   ambari-server/src/main/java/org/apache/ambari/server/events/TaskEvent.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/TaskUpdateEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListener.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/publishers/TaskEventPublisher.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  02c4091 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
> 1c4d0a3 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
> d2f899f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
>  74271b9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestEntity.java
>  7944d21 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  f9c8810 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  9ca0470 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java
>  4f90ef3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 8cf2c0d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 82ce31e 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql e2c2dd5 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 4e9a535 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0ba7df6 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql d8cad6f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
>  1ca777d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
>  f86c02e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  e23ba62 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeSummaryResourceProviderTest.java
>  baec7df 
>   
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListenerTest.java
>  

Re: Review Request 53686: Stage and Request status should be persisted in the database

[Jaimin Jetly]

> On Nov. 14, 2016, 5:03 p.m., Nate Cole wrote:
> > Is this effort spanning multiple jira?  I was hoping we would be able to 
> > optimize queries against the database if we need to recalculate status.  
> > When tasks are completed can we compute the status of the whole based on 
> > just the status of the single task (for any case)?  Where is the update 
> > code for Stage being run (when we know a task's own status)?

The latest revision of the patch does change StageDAO query to find in progress 
stages by looking into stage status rather tahn subquerying host role command 
statuses.
Although wider scan of the code to leverage stage/request persisted status will 
be done as part of https://issues.apache.org/jira/browse/AMBARI-18889.
For now I have verified that this work does not regress performance by testing 
on 1000 node simulated cluster


- Jaimin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53686/#review155819
---


On Dec. 14, 2016, 2:11 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53686/
> ---
> 
> (Updated Dec. 14, 2016, 2:11 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18868
> https://issues.apache.org/jira/browse/AMBARI-18868
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Stage and Request status should be persisted in the database.
> 
> upgrading to ambari-3.0.0 should add status for all present stages and 
> request for the cluster.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  2c87583 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  e80b020 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java
>  3656bfe 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Request.java
>  31e11c1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> 4a05b32 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3c415df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/TaskCreateEvent.java
>  PRE-CREATION 
>   ambari-server/src/main/java/org/apache/ambari/server/events/TaskEvent.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/TaskUpdateEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListener.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/publishers/TaskEventPublisher.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  02c4091 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
> 1c4d0a3 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
> d2f899f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
>  74271b9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestEntity.java
>  7944d21 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  f9c8810 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  9ca0470 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java
>  4f90ef3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 8cf2c0d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 82ce31e 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql e2c2dd5 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 4e9a535 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0ba7df6 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql d8cad6f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
>  1ca777d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
>  f86c02e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  e23ba62 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeSummaryResourceProviderTest.java
>  baec7df 
>   
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListenerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/UpgradeDAOTest.java
>  

Re: Review Request 53686: Stage and Request status should be persisted in the database

[Jaimin Jetly]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53686/
---

(Updated Dec. 14, 2016, 2:11 a.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle.


Bugs: AMBARI-18868
https://issues.apache.org/jira/browse/AMBARI-18868


Repository: ambari


Description
---

Stage and Request status should be persisted in the database.

upgrading to ambari-3.0.0 should add status for all present stages and request 
for the cluster.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
 2c87583 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 e80b020 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java
 3656bfe 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Request.java 
31e11c1 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
4a05b32 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
 3c415df 
  
ambari-server/src/main/java/org/apache/ambari/server/events/TaskCreateEvent.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/events/TaskEvent.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/events/TaskUpdateEvent.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListener.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/events/publishers/TaskEventPublisher.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 02c4091 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
1c4d0a3 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
d2f899f 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
 74271b9 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestEntity.java
 7944d21 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 f9c8810 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
 9ca0470 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java
 4f90ef3 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 8cf2c0d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 82ce31e 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql e2c2dd5 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 4e9a535 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0ba7df6 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql d8cad6f 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
 1ca777d 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
 f86c02e 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 e23ba62 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeSummaryResourceProviderTest.java
 baec7df 
  
ambari-server/src/test/java/org/apache/ambari/server/events/listeners/tasks/TaskStatusListenerTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/UpgradeDAOTest.java
 cc49cbd 
  
ambari-server/src/test/java/org/apache/ambari/server/state/services/RetryUpgradeActionServiceTest.java
 7dd9932 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java
 d7979e8 

Diff: https://reviews.apache.org/r/53686/diff/


Testing (updated)
---

Verified manually on a cluster by making api requests and upgrading ambari.
Add unit tests.
Verified that the patch does not break any existing unit tests on dev box. 
Jenkins job overall unit test result pending..
Verified on 1000 node cluster that the patch does not regress big operations. 
Executed Stop Services and Start Services API call which gernerated around 9000 
tasks and compared  request completion time for these operations. There was a 
minor performance gain with the patch. As part of 
https://issues.apache.org/jira/browse/AMBARI-18889, I will look if we can use 
the request status and stage status to further enhance performance.


Thanks,

Jaimin Jetly

Re: Review Request 54665: AMBARI-19156 Install Packages fails with the old stack has services removed in the new stack

[Di Li]

> On Dec. 12, 2016, 7:25 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  line 237
> > 
> >
> > Will this work with multi-level inheritence?
> > E.g., 
> > Stack 1 only has Spark
> > Stack 2 only has Spark2, and inherits from Stack 1
> > Stack 3 only has Spark3, and inherits from Stack 2
> > 
> > Technically, Stack 3 has removed both Spark and Spark2.

Hello Alejandro,

I modified the java code to be able to resolve removed services defined in the 
parent stacks.

So a structure like this

Stack 1, Spark
Stack 2, Spark2, Spark(marked as deleted)
Stack 3, Spark3, Spark2(marked as deleted)

would result in Stack3 shows only Spark3 as a valid service, while both Spark2 
and Spark (via inheritance) are marked as deleted.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54665/#review158892
---


On Dec. 12, 2016, 4:31 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54665/
> ---
> 
> (Updated Dec. 12, 2016, 4:31 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-19156
> https://issues.apache.org/jira/browse/AMBARI-19156
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Install packages step (before RU/EU) fails with err "Cannot enumerate 
> services" if the current stack has services that have been removed in the new 
> stack. For example, mystack-1.0 has Spark, but mystakc-2.0 only has Spark2, 
> where Spark's metainfo.xml in mystack-2.0 has the true 
> section.
> 
> Workaround is to remove Spark before attempting the install packages.
> 
> The error is below:
> 02 Dec 2016 12:53:43,928 ERROR [ambari-client-thread-27] 
> BaseManagementHandler:61 - Caught a system exception while attempting to 
> create a resource: Cannot enumerate services
> org.apache.ambari.server.controller.spi.SystemException: Cannot enumerate 
> services
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  23a4a27 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
>  d417ec2 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> ce606de 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> ba5cb42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java
>  8263f72 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  0b7d0ff 
> 
> Diff: https://reviews.apache.org/r/54665/diff/
> 
> 
> Testing
> ---
> 
> 1. unit tests
> 2. patched a cluster with the fixes, update new stack to remove Spark ( 
> leaving only  Spark 2), run Install Packages, verify it didn't fail with the 
> "Cannot enumerate services" error.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 54665: AMBARI-19156 Install Packages fails with the old stack has services removed in the new stack

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54665/
---

(Updated Dec. 13, 2016, 10:38 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tim Thorpe.


Bugs: AMBARI-19156
https://issues.apache.org/jira/browse/AMBARI-19156


Repository: ambari


Description
---

Install packages step (before RU/EU) fails with err "Cannot enumerate services" 
if the current stack has services that have been removed in the new stack. For 
example, mystack-1.0 has Spark, but mystakc-2.0 only has Spark2, where Spark's 
metainfo.xml in mystack-2.0 has the true section.

Workaround is to remove Spark before attempting the install packages.

The error is below:
02 Dec 2016 12:53:43,928 ERROR [ambari-client-thread-27] 
BaseManagementHandler:61 - Caught a system exception while attempting to create 
a resource: Cannot enumerate services
org.apache.ambari.server.controller.spi.SystemException: Cannot enumerate 
services


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
 23a4a27 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 d417ec2 
  ambari-server/src/main/java/org/apache/ambari/server/stack/BaseModule.java 
28b8e46 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionModule.java 
51b6104 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
c027f7f 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
ce606de 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
bd60e3a 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 9509d89 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java
 8263f72 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java 
0b7d0ff 
  ambari-server/src/test/resources/stacks/HDP/2.0.6/services/SPARK/metainfo.xml 
PRE-CREATION 
  ambari-server/src/test/resources/stacks/HDP/2.0.7/services/SPARK/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/stacks/HDP/2.0.7/services/SPARK2/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/stacks/HDP/2.0.8/services/SPARK2/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/stacks/HDP/2.0.8/services/SPARK3/metainfo.xml 
PRE-CREATION 

Diff: https://reviews.apache.org/r/54665/diff/


Testing
---

1. unit tests
2. patched a cluster with the fixes, update new stack to remove Spark ( leaving 
only  Spark 2), run Install Packages, verify it didn't fail with the "Cannot 
enumerate services" error.


Thanks,

Di Li

Re: Review Request 54721: Add common log rotation settings to YARN configs

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54721/#review159066
---


Ship it!




Ship It!

- Sumit Mohanty


On Dec. 13, 2016, 10:31 p.m., Madhuvanthi Radhakrishnan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54721/
> ---
> 
> (Updated Dec. 13, 2016, 10:31 p.m.)
> 
> 
> Review request for Ambari, Andrii Voloshyn and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19193
> https://issues.apache.org/jira/browse/AMBARI-19193
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add common log rotation settings to YARN configs
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  4a5ee25 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-log4j.xml
>  9ac34f3 
> 
> Diff: https://reviews.apache.org/r/54721/diff/
> 
> 
> Testing
> ---
> 
> 1. Check if the parameters are exposed in UI and you can change it
> 2. Check if changed parameter values are reflected in DB - select config_data 
> from clusterconfig where type_name = 'yarn-log4j' and version=2;
> 3. Check if changed param values are reflected in API - 
> http://:8080/api/v1/clusters//configurations?type=hadoop-env=version1
> 4. Check if changed param values are reflected in files - queen 
> /etc/hadoop/conf/log4j.properties log4j.appender.RMSUMMARY.MaxBackupIndex=15
> 
> 
> Thanks,
> 
> Madhuvanthi Radhakrishnan
> 
>

Re: Review Request 54690: Remove HSI host name from UI

[Sumit Mohanty]

> On Dec. 13, 2016, 6:46 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml,
> >  line 46
> > 
> >
> > Should we mark this property as deleted in HDP 2.6?

Thought of that but I do not think it will help. Upon Ambari upgrade, the 
property will no longer be visible in the stack definition. Even if its in the 
DB, its ignored. Marking it as deleted does not auto-delete properties from the 
persisted configs. Usually deleted allows for a property to be valid in a stack 
version and invisible in the next version. We are going for a full delete from 
all versions here.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54690/#review159037
---


On Dec. 13, 2016, 2:30 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54690/
> ---
> 
> (Updated Dec. 13, 2016, 2:30 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Aravindan Vijayan, and Jaimin 
> Jetly.
> 
> 
> Bugs: AMBARI-19184
> https://issues.apache.org/jira/browse/AMBARI-19184
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove HSI host name from UI
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  7b20728 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/themes/theme.json
>  1d50b6a 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 60c1cac 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 54fe42a 
>   ambari-server/src/test/python/stacks/2.5/configs/hsi_ha.json 7f95a35 
> 
> Diff: https://reviews.apache.org/r/54690/diff/
> 
> 
> Testing
> ---
> 
> Manually ran all tests and tested by manually deploying the cluster with 
> patches.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 54640: LogSearch - generate default jks file on startup

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54640/
---

(Updated Dec. 13, 2016, 7:23 p.m.)


Review request for Ambari, Miklos Gergely, Robert Levas, and Robert Nettleton.


Changes
---

rebase the changes


Bugs: AMBARI-19147
https://issues.apache.org/jira/browse/AMBARI-19147


Repository: ambari


Description
---

Generate JKS file on startup. (if keystore is not specified or not exists)


Diffs (updated)
-

  ambari-logsearch/ambari-logsearch-portal/pom.xml 8326683 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
 913d25b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
 257ae3c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 206f793 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties 
c98a482 
  ambari-logsearch/ambari-logsearch-portal/src/main/resources/log4j.xml 60b09cd 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 2f13710 

Diff: https://reviews.apache.org/r/54640/diff/


Testing
---

testing done. manually. (start ambar server, import generated logsearch 
certificate to ambar keystore)


Thanks,

Oliver Szabo

Re: Review Request 54704: Remove UI logic to set or expect hive_server_interactive_host

[Alexandr Antonenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54704/#review159047
---


Ship it!




Ship It!

- Alexandr Antonenko


On Dec. 13, 2016, 6:54 p.m., Andrii Tkach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54704/
> ---
> 
> (Updated Dec. 13, 2016, 6:54 p.m.)
> 
> 
> Review request for Ambari, Aleksandr Kovalenko, Alexandr Antonenko, and 
> Jaimin Jetly.
> 
> 
> Bugs: AMBARI-19190
> https://issues.apache.org/jira/browse/AMBARI-19190
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With the support for HSI HA, the property no longer seems appropriate. Lets 
> remove the dependency on this property and no longer expect or use it.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/service/info/configs.js 94caf4a 
>   ambari-web/app/controllers/wizard/step7/assign_master_controller.js 1ee0c45 
>   ambari-web/app/models/configs/theme/config_action.js 9ac4e7a 
>   ambari-web/test/controllers/wizard/step7/assign_master_controller_test.js 
> 053b900 
> 
> Diff: https://reviews.apache.org/r/54704/diff/
> 
> 
> Testing
> ---
> 
> 19701 passing (32s)
> 153 pending
> 
> 
> Thanks,
> 
> Andrii Tkach
> 
>

Re: Review Request 54675: AMBARI-19178 HostCleanup can't list process details on RHEL7

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54675/#review159046
---


Ship it!




Ship It!

- Tim Thorpe


On Dec. 12, 2016, 9:11 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54675/
> ---
> 
> (Updated Dec. 12, 2016, 9:11 p.m.)
> 
> 
> Review request for Ambari and Tim Thorpe.
> 
> 
> Bugs: AMBARI-19178
> https://issues.apache.org/jira/browse/AMBARI-19178
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> need to use ps auxww instead of just ps aux
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/HostCleanup.py 87cd1b0 
> 
> Diff: https://reviews.apache.org/r/54675/diff/
> 
> 
> Testing
> ---
> 
> existing unit test
> patch a trunk cluster with the change, run HostCleanup.py
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 54640: LogSearch - generate default jks file on startup

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54640/
---

(Updated Dec. 13, 2016, 7:09 p.m.)


Review request for Ambari, Miklos Gergely, Robert Levas, and Robert Nettleton.


Changes
---

small log4j fixes for default log4j.xml


Bugs: AMBARI-19147
https://issues.apache.org/jira/browse/AMBARI-19147


Repository: ambari


Description
---

Generate JKS file on startup. (if keystore is not specified or not exists)


Diffs (updated)
-

  ambari-logsearch/ambari-logsearch-portal/pom.xml 6bacd23 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
 913d25b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
 257ae3c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 206f793 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties 
c98a482 
  ambari-logsearch/ambari-logsearch-portal/src/main/resources/log4j.xml 60b09cd 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 2f13710 

Diff: https://reviews.apache.org/r/54640/diff/


Testing
---

testing done. manually. (start ambar server, import generated logsearch 
certificate to ambar keystore)


Thanks,

Oliver Szabo

Re: Review Request 54640: LogSearch - generate default jks file on startup

[Oliver Szabo]

> On Dec. 13, 2016, 2:44 p.m., Robert Levas wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java,
> >  line 171
> > 
> >
> > Missing JavaDoc

i did not included any javadoc for private methods (as the methods clearly does 
what the method name say. if it does more, that means that method is written 
badly)


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54640/#review159001
---


On Dec. 13, 2016, 6:56 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54640/
> ---
> 
> (Updated Dec. 13, 2016, 6:56 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-19147
> https://issues.apache.org/jira/browse/AMBARI-19147
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Generate JKS file on startup. (if keystore is not specified or not exists)
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml 6bacd23 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  913d25b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
>  257ae3c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
>  206f793 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties
>  c98a482 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  2f13710 
> 
> Diff: https://reviews.apache.org/r/54640/diff/
> 
> 
> Testing
> ---
> 
> testing done. manually. (start ambar server, import generated logsearch 
> certificate to ambar keystore)
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 54704: Remove UI logic to set or expect hive_server_interactive_host

[Andrii Tkach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54704/
---

(Updated Dec. 13, 2016, 6:54 p.m.)


Review request for Ambari, Aleksandr Kovalenko and Jaimin Jetly.


Bugs: AMBARI-19190
https://issues.apache.org/jira/browse/AMBARI-19190


Repository: ambari


Description
---

With the support for HSI HA, the property no longer seems appropriate. Lets 
remove the dependency on this property and no longer expect or use it.


Diffs (updated)
-

  ambari-web/app/controllers/main/service/info/configs.js 94caf4a 
  ambari-web/app/controllers/wizard/step7/assign_master_controller.js 1ee0c45 
  ambari-web/app/models/configs/theme/config_action.js 9ac4e7a 
  ambari-web/test/controllers/wizard/step7/assign_master_controller_test.js 
053b900 

Diff: https://reviews.apache.org/r/54704/diff/


Testing
---

19701 passing (32s)
153 pending


Thanks,

Andrii Tkach

Re: Review Request 54709: Use released Helix 0.6.6 instead of 0.7.1 beta as AMS dependency

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54709/#review159042
---



Please make sure to test kerberized cluster with this version. Thanks.

- Sid Wagle


On Dec. 13, 2016, 4:47 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54709/
> ---
> 
> (Updated Dec. 13, 2016, 4:47 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-19192
> https://issues.apache.org/jira/browse/AMBARI-19192
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Use released Helix 0.6.6 instead of 0.7.1 beta as AMS dependency
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml 6896c6b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/AggregationTaskRunner.java
>  ceab754 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  26702b6 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/OnlineOfflineStateModelFactory.java
>  eb63327 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAControllerTest.java
>  875f013 
> 
> Diff: https://reviews.apache.org/r/54709/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 53980: AMBARI-18945 Allowing user to edit service name while enabling ranger plugin

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53980/#review159040
---


Ship it!




Ship It!

- Alejandro Fernandez


On Dec. 13, 2016, 12:40 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53980/
> ---
> 
> (Updated Dec. 13, 2016, 12:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18945
> https://issues.apache.org/jira/browse/AMBARI-18945
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Need to add a provision for creating default repository with static name or 
> user provided name.
> 
> This will help user to provided service name under Ranger supported plugins 
> config tab and can take dump of latest snapshot of all policies and use it 
> while recreating another/multiple clusters with same service name.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  32e1510 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  f47dc8f 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  d86e360 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  e9436c1 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  5635fe3 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  4cefbda 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  c20bdde 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  4859534 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  1347864 
> 
> Diff: https://reviews.apache.org/r/53980/diff/
> 
> 
> Testing
> ---
> 
> mvn -DskipPythonTests -Dtest=UpgradeCatalog250Test test
> 
> ---
>  T E S T S
> ---
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support 
> was removed in 8.0
> Running org.apache.ambari.server.upgrade.UpgradeCatalog250Test
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.548 sec - 
> in org.apache.ambari.server.upgrade.UpgradeCatalog250Test
> 
> Results :
> 
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 30.347 s
> [INFO] Finished at: 2016-11-22T11:51:23+00:00
> [INFO] Final Memory: 63M/1107M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 54690: Remove HSI host name from UI

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54690/#review159037
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 


Should we mark this property as deleted in HDP 2.6?


- Alejandro Fernandez


On Dec. 13, 2016, 2:30 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54690/
> ---
> 
> (Updated Dec. 13, 2016, 2:30 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Aravindan Vijayan, and Jaimin 
> Jetly.
> 
> 
> Bugs: AMBARI-19184
> https://issues.apache.org/jira/browse/AMBARI-19184
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove HSI host name from UI
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  7b20728 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/themes/theme.json
>  1d50b6a 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 60c1cac 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 54fe42a 
>   ambari-server/src/test/python/stacks/2.5/configs/hsi_ha.json 7f95a35 
> 
> Diff: https://reviews.apache.org/r/54690/diff/
> 
> 
> Testing
> ---
> 
> Manually ran all tests and tested by manually deploying the cluster with 
> patches.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 54674: AMBARI-19170. NPE during Ambari server schema upgrade

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54674/#review159032
---


Ship it!




Ship It!

- Jonathan Hurley


On Dec. 12, 2016, 3:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54674/
> ---
> 
> (Updated Dec. 12, 2016, 3:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19170
> https://issues.apache.org/jira/browse/AMBARI-19170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Removed @Transactional
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  063c29539e135433b7b9ce37fa5c32d0de2234f8 
> 
> Diff: https://reviews.apache.org/r/54674/diff/
> 
> 
> Testing
> ---
> 
> Installed 2.2.2.0-460, upgraded to 2.5.0.0-453.
> 
> ```
> [root@c6401 ~]# ambari-server upgrade
> Using python  /usr/bin/python
> Upgrading ambari-server
> INFO: Upgrade Ambari Server
> INFO: Updating Ambari Server properties in ambari.properties ...
> INFO: Updating Ambari Server properties in ambari-env.sh ...
> WARNING: Original file ambari-env.sh kept
> INFO: Fixing database objects owner
> Ambari Server configured for Embedded Postgres. Confirm you have made a 
> backup of the Ambari Server database [y/n] (y)? y
> INFO: Upgrading database schema
> INFO: Return code from schema upgrade command, retcode = 0
> INFO: Schema upgrade completed
> Adjusting ambari-server permissions and ownership...
> Ambari Server 'upgrade' completed successfully.
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 54698: Disable security hook

[Attila Magyar]

> On Dec. 13, 2016, 3:26 p.m., Robert Levas wrote:
> > ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java, 
> > line 38
> > 
> >
> > This test suite should use mocks (example, `org.easymock.EasyMock`) 
> > rather than create a server and perform operations on it.  This is more of 
> > an intergration test than a unit test.

I strongly disagree with this one. This code talks to a 3rd party zookeeper 
API. It would be required to mock objects we don't own and can't control. The 
test would mimic the exact same behaviour than the code, leading to brittle and 
implementation specific tests without adding too much -if any- value to the 
correctness of the code. I wanted to check with this test that the 3rd party 
API works the way I expected.
Introducing an ambari specific abstraction on top of the zookeeper API and 
mocking that would be ok, but the integration test still would be needed i 
think and there is not too much code there besides which talks to zookeeper.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54698/#review159008
---


On Dec. 13, 2016, 5:30 p.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54698/
> ---
> 
> (Updated Dec. 13, 2016, 5:30 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jaimin Jetly, Laszlo Puskas, 
> Oliver Szabo, Robert Levas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19187
> https://issues.apache.org/jira/browse/AMBARI-19187
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hadoop components need to establish a secure connection with ZooKeeper when 
> Kerberos is enabled. This involves the setup of the correct authentication 
> (JAAS config file) and authorization (per-component Kerberos-backed ACLs on 
> the znodes) between the service and ZooKeeper. Most services are able to set 
> these ACLs based on their config when the user enable kerberos.
> When we disable kerberos again, the sasl ACL should be removed otherwise the 
> services won't be able to access their znodes.
> 
> This issue is about introducing a new command (DISABLE_SECURITY) that will be 
> sent by the ambari server to the services upon the dekerberiztion process. 
> When a service receives this command it will be able to do the zookeeper 
> secure to unsecure migration process (e.g. removing sasl ACLs).
> 
> Notable changes:
>  - Added a java command line tool to the agent project that can setAcls 
> recursively on a znode
>  - Modified the dekerberization workflow:
>  - 1. UI stops all services but zookeeper 
>  - 2. 2 new stages was introduced in the backend (send DISABLE_SECURITY 
> command to the services, start zookeeper)
> 
> 
> Diffs
> -
> 
>   ambari-agent/pom.xml a8ed7f1 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java 
> PRE-CREATION 
>   ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java 
> PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/core/resources/zkmigrator.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 584775e 
>   ambari-server/pom.xml 48ddb52 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  3261a56 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/ActionMetadata.java
>  0064662 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  90f8098 
>   ambari-web/app/controllers/main/admin/kerberos/disable_controller.js 
> cec4503 
> 
> Diff: https://reviews.apache.org/r/54698/diff/
> 
> 
> Testing
> ---
> 
> Added unittests for ZkMigrator, KerberosHelperImpl
> 
> Manual testings: 
>  - created cluster with ambari
>  - enabled kerberos
>  - disabled kerberos
>  - checked if the DISABLE_SECURITY command was sent to the services
> 
> 
> Ambari agent:
> --
> Ran 450 tests in 10.634s
> 
> Ambari server:
> --
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 34:44.448s
> [INFO] Finished at: Tue Dec 13 14:29:00 CET 2016
> [INFO] Final Memory: 160M/798M
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Re: Review Request 54698: Disable security hook

[Jaimin Jetly]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54698/#review159027
---




ambari-web/app/controllers/main/admin/kerberos/disable_controller.js (line 26)


It is possible that zookeeper is alreay stopped when user starts 
de-kerberizing. To handle that situation additional step to start zookeeper 
will be required. 
['stopServices', 'startZk',...]


- Jaimin Jetly


On Dec. 13, 2016, 1:51 p.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54698/
> ---
> 
> (Updated Dec. 13, 2016, 1:51 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Laszlo Puskas, Oliver Szabo, Robert 
> Levas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19187
> https://issues.apache.org/jira/browse/AMBARI-19187
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hadoop components need to establish a secure connection with ZooKeeper when 
> Kerberos is enabled. This involves the setup of the correct authentication 
> (JAAS config file) and authorization (per-component Kerberos-backed ACLs on 
> the znodes) between the service and ZooKeeper. Most services are able to set 
> these ACLs based on their config when the user enable kerberos.
> When we disable kerberos again, the sasl ACL should be removed otherwise the 
> services won't be able to access their znodes.
> 
> This issue is about introducing a new command (DISABLE_SECURITY) that will be 
> sent by the ambari server to the services upon the dekerberiztion process. 
> When a service receives this command it will be able to do the zookeeper 
> secure to unsecure migration process (e.g. removing sasl ACLs).
> 
> Notable changes:
>  - Added a java command line tool to the agent project that can setAcls 
> recursively on a znode
>  - Modified the dekerberization workflow:
>  - 1. UI stops all services but zookeeper 
>  - 2. 2 new stages was introduced in the backend (send DISABLE_SECURITY 
> command to the services, start zookeeper)
> 
> 
> Diffs
> -
> 
>   ambari-agent/pom.xml a8ed7f1 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java 
> PRE-CREATION 
>   ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java 
> PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/core/resources/zkmigrator.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 584775e 
>   ambari-server/pom.xml 48ddb52 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  3261a56 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/ActionMetadata.java
>  0064662 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  90f8098 
>   ambari-web/app/controllers/main/admin/kerberos/disable_controller.js 
> cec4503 
> 
> Diff: https://reviews.apache.org/r/54698/diff/
> 
> 
> Testing
> ---
> 
> Added unittests for ZkMigrator, KerberosHelperImpl
> 
> Manual testings: 
>  - created cluster with ambari
>  - enabled kerberos
>  - disabled kerberos
>  - checked if the DISABLE_SECURITY command was sent to the services
> 
> 
> Ambari agent:
> --
> Ran 450 tests in 10.634s
> 
> Ambari server:
> --
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 34:44.448s
> [INFO] Finished at: Tue Dec 13 14:29:00 CET 2016
> [INFO] Final Memory: 160M/798M
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Re: Review Request 54704: Remove UI logic to set or expect hive_server_interactive_host

[Andrii Tkach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54704/
---

(Updated Dec. 13, 2016, 4:52 p.m.)


Review request for Ambari, Aleksandr Kovalenko and Jaimin Jetly.


Bugs: AMBARI-19190
https://issues.apache.org/jira/browse/AMBARI-19190


Repository: ambari


Description
---

With the support for HSI HA, the property no longer seems appropriate. Lets 
remove the dependency on this property and no longer expect or use it.


Diffs (updated)
-

  ambari-web/app/controllers/wizard/step7/assign_master_controller.js 1ee0c45 
  ambari-web/app/mixins/main/service/configs/component_actions_by_configs.js 
347bdde 
  ambari-web/app/models/configs/theme/config_action.js 9ac4e7a 
  ambari-web/app/views/common/configs/widgets/config_widget_view.js 2c90cc3 
  ambari-web/test/controllers/wizard/step7/assign_master_controller_test.js 
053b900 
  
ambari-web/test/mixins/main/service/configs/component_actions_by_configs_test.js
 1a93a26 

Diff: https://reviews.apache.org/r/54704/diff/


Testing
---

19701 passing (32s)
153 pending


Thanks,

Andrii Tkach

Review Request 54709: Use released Helix 0.6.6 instead of 0.7.1 beta as AMS dependency

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54709/
---

Review request for Ambari, Aravindan Vijayan, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-19192
https://issues.apache.org/jira/browse/AMBARI-19192


Repository: ambari


Description
---

Use released Helix 0.6.6 instead of 0.7.1 beta as AMS dependency


Diffs
-

  ambari-metrics/ambari-metrics-timelineservice/pom.xml 6896c6b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/AggregationTaskRunner.java
 ceab754 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
 26702b6 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/OnlineOfflineStateModelFactory.java
 eb63327 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAControllerTest.java
 875f013 

Diff: https://reviews.apache.org/r/54709/diff/


Testing
---

Unit tests passed.


Thanks,

Dmytro Sen

Re: Review Request 54674: AMBARI-19170. NPE during Ambari server schema upgrade

[Attila Doroszlai]

> On Dec. 12, 2016, 9:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java,
> >  line 1555
> > 
> >
> > Because this is changing data, should the data manipulation be wrapped 
> > in an `executeInTransaction(Runnable)` ?
> 
> Dmytro Grinenko wrote:
> moreover, i'm not sure that @Transactional annotation will have any 
> effect on DDL changes via dbAccessor or any other way to make schema changes 
> in scope of database transaction. (not all database engines support this)
> 
> Dmytro Grinenko wrote:
> P.S. i mean revert of schema alteration, as MySQL for exmple done 
> explicit commit on alter commands
> 
> Attila Doroszlai wrote:
> `@Transactional` on these 2 methods was ignored by Guice until they were 
> changed from `private` to default visibility recently (AMBARI-18966).  The 
> cause of the NPE is that the persistence service, which creates 
> `EntityManagerFactory`, is only 
> [started](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L426)
>  after [performing DDL 
> upgrade](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L424).
> 
> As Dmytro mentioned, not all databases support transactional DDL:
> 
>  * [PostgreSQL: 
> OK](https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis)
>  * [MySQL: 
> NO](http://dev.mysql.com/doc/refman/5.7/en/cannot-roll-back.html)
>  * [Oracle: 
> NO](https://docs.oracle.com/cd/B14117_01/server.101/b10759/statements_1001.htm)
> 
> Attila Doroszlai wrote:
> ReviewBoard ate my link for PostgreSQL: 
> https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis
> 
> Jonathan Hurley wrote:
> I understand about the DDL changes - but that's OK since we dont need to 
> roll those back. This method makes DML changes down below which should be in 
> a transaction.

These DML statements never ran in a single transaction.  In this bug I'm only 
fixing the NPE, since it's a blocker.  I'll create a follow-up JIRA.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54674/#review158900
---


On Dec. 12, 2016, 9:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54674/
> ---
> 
> (Updated Dec. 12, 2016, 9:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19170
> https://issues.apache.org/jira/browse/AMBARI-19170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Removed @Transactional
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  063c29539e135433b7b9ce37fa5c32d0de2234f8 
> 
> Diff: https://reviews.apache.org/r/54674/diff/
> 
> 
> Testing
> ---
> 
> Installed 2.2.2.0-460, upgraded to 2.5.0.0-453.
> 
> ```
> [root@c6401 ~]# ambari-server upgrade
> Using python  /usr/bin/python
> Upgrading ambari-server
> INFO: Upgrade Ambari Server
> INFO: Updating Ambari Server properties in ambari.properties ...
> INFO: Updating Ambari Server properties in ambari-env.sh ...
> WARNING: Original file ambari-env.sh kept
> INFO: Fixing database objects owner
> Ambari Server configured for Embedded Postgres. Confirm you have made a 
> backup of the Ambari Server database [y/n] (y)? y
> INFO: Upgrading database schema
> INFO: Return code from schema upgrade command, retcode = 0
> INFO: Schema upgrade completed
> Adjusting ambari-server permissions and ownership...
> Ambari Server 'upgrade' completed successfully.
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 53686: Stage and Request status should be persisted in the database

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53686/#review158848
---




ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (line 190)


just "taskStatusLoadedOnce" or "taskStatusLoaded" (since it's boolean and 
can only be false "once")



ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
(lines 51 - 52)


Are you using these?



ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
(lines 303 - 305)


Even though these are simple getters and setters, we should still javadoc 
them



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java
 (lines 110 - 115)


not null, etc



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java
 (lines 187 - 188)


no need to break this line up



ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql (lines 346 - 347)


Should probably make status non-null as well.



ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql (lines 363 - 364)


Make not-nullable with a default value of 'PENDING' (a HostRoleCommand's 
initial status is PENDING).



ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
 (line 58)


org.junit.Assert


- Nate Cole


On Dec. 7, 2016, 2:45 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53686/
> ---
> 
> (Updated Dec. 7, 2016, 2:45 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18868
> https://issues.apache.org/jira/browse/AMBARI-18868
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Stage and Request status should be persisted in the database.
> 
> upgrading to ambari-3.0.0 should add status for all present stages and 
> request for the cluster.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  2c87583 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  e80b020 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Request.java
>  31e11c1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> 319d72d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
>  3c415df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/TaskUpdateEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/tasks/TaskUpdateListener.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/publishers/TaskUpdateEventPublisher.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  02c4091 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 
> 1c4d0a3 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 
> d2f899f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RequestEntity.java
>  b1aad00 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  f9c8810 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntityPK.java
>  0c227bc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java
>  4f90ef3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 8cf2c0d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 82ce31e 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql e2c2dd5 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 4e9a535 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0ba7df6 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql d8cad6f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
>  f86c02e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8efcc61 
>   
> 

Re: Review Request 54665: AMBARI-19156 Install Packages fails with the old stack has services removed in the new stack

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54665/#review159015
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(lines 237 - 239)


Agree with @Alejandro's comment.  Once fixed, then I'm ok to Ship.


- Nate Cole


On Dec. 12, 2016, 11:31 a.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54665/
> ---
> 
> (Updated Dec. 12, 2016, 11:31 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-19156
> https://issues.apache.org/jira/browse/AMBARI-19156
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Install packages step (before RU/EU) fails with err "Cannot enumerate 
> services" if the current stack has services that have been removed in the new 
> stack. For example, mystack-1.0 has Spark, but mystakc-2.0 only has Spark2, 
> where Spark's metainfo.xml in mystack-2.0 has the true 
> section.
> 
> Workaround is to remove Spark before attempting the install packages.
> 
> The error is below:
> 02 Dec 2016 12:53:43,928 ERROR [ambari-client-thread-27] 
> BaseManagementHandler:61 - Caught a system exception while attempting to 
> create a resource: Cannot enumerate services
> org.apache.ambari.server.controller.spi.SystemException: Cannot enumerate 
> services
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  23a4a27 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
>  d417ec2 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> ce606de 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> ba5cb42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java
>  8263f72 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  0b7d0ff 
> 
> Diff: https://reviews.apache.org/r/54665/diff/
> 
> 
> Testing
> ---
> 
> 1. unit tests
> 2. patched a cluster with the fixes, update new stack to remove Spark ( 
> leaving only  Spark 2), run Install Packages, verify it didn't fail with the 
> "Cannot enumerate services" error.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Review Request 54707: Metrics Collector Move Wizard should no longer Restart All Services

[Aleksandr Kovalenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54707/
---

Review request for Ambari and Alexandr Antonenko.


Bugs: AMBARI-19191
https://issues.apache.org/jira/browse/AMBARI-19191


Repository: ambari


Description
---

Metrics Collector Move Wizard should no longer Restart All Services


Diffs
-

  ambari-web/app/controllers/main/service/reassign/step4_controller.js da7d2d9 

Diff: https://reviews.apache.org/r/54707/diff/


Testing
---

19881 tests complete (28 seconds)
  153 tests pending


Thanks,

Aleksandr Kovalenko

Re: Review Request 54698: Disable security hook

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54698/#review159008
---



Most of this looks pretty good, however the flow seems like it might be 
problematic and not clear to users using the Ambari REST API to disable 
Kerberos.  This can be helped with documentation and 
`ambari-server/docs/security/kerberos/enabling_kerberos.md` should be updated 
to reflect any changes. 

Would it be possible to have the worflow orchestrated by the caller rather than 
_hide_ some steps. The REST API calls might look something like:

- Stop all services excecpt for Zookeeper
- Execute command on all Services - DISABLE_KERBEROS (like one would execute 
SERVICE_CHECK)
- Stop all services (this would really be just Zookeeper at this point)
- _Continue with the rest of the workflow as is_

This seens to be more explicit.


ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java (line 12)


Interesting wording.  Maybe this should read

```
"ZkACL represents a ZooKeeper ACL (scheme, id and permissions)"
```



ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java (line 28)


surround with `{` and `}` - Ambari coding style.



ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java (line 48)


surround with `{` and `}` - Ambari coding style.



ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java (line 60)


surround with `{` and `}` - Ambari coding style.



ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java (line 
53)


surround with `{` and `}` - Ambari coding style.



ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java (line 36)


surround with `{` and `}` - Ambari coding style.



ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java (line 
38)


This test suite should use mocks (example, `org.easymock.EasyMock`) rather 
than create a server and perform operations on it.  This is more of an 
intergration test than a unit test.



ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java 
(lines 110 - 115)


It seems like this is unnecessary and mocks should be used instead.



ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 (line 3033)


Add docs to clarify what is going on here.  A quick scan of the code is not 
sufficient to understand what is going on any why.



ambari-web/app/controllers/main/admin/kerberos/disable_controller.js (line 26)


Rather than change this step, I think it would be better to add a new on to 
prepare to disable Kerberos then proceed as it did before.


- Robert Levas


On Dec. 13, 2016, 8:51 a.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54698/
> ---
> 
> (Updated Dec. 13, 2016, 8:51 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Laszlo Puskas, Oliver Szabo, Robert 
> Levas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19187
> https://issues.apache.org/jira/browse/AMBARI-19187
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hadoop components need to establish a secure connection with ZooKeeper when 
> Kerberos is enabled. This involves the setup of the correct authentication 
> (JAAS config file) and authorization (per-component Kerberos-backed ACLs on 
> the znodes) between the service and ZooKeeper. Most services are able to set 
> these ACLs based on their config when the user enable kerberos.
> When we disable kerberos again, the sasl ACL should be removed otherwise the 
> services won't be able to access their znodes.
> 
> This issue is about introducing a new command (DISABLE_SECURITY) that will be 
> sent by the ambari server to the services upon the dekerberiztion process. 
> When a service receives this command it will be able to do the zookeeper 
> secure to unsecure migration process (e.g. removing sasl ACLs).
> 
> Notable changes:
>  - Added a java command line tool to the agent project that can setAcls 
> recursively on a znode
>  - Modified the dekerberization workflow:
>  - 1. UI stops all services but zookeeper 
>  - 2. 2 new stages was introduced in the backend (send DISABLE_SECURITY 
> command 

Re: Review Request 54706: AMBARI-19188 :: Provide Atlas Hook support for Hive2

[Vishal Suvagia]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54706/
---

(Updated Dec. 13, 2016, 3:20 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, 
Mugdha Varadkar, and Swapan Shridhar.


Bugs: AMBARI-19188
https://issues.apache.org/jira/browse/AMBARI-19188


Repository: ambari


Description
---

Need to provide Atlas hook configuration support for HiveServer-Interactive.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 7fbd6bd 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_interactive.py
 af2a05d 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 33f1056 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 dde3e00 

Diff: https://reviews.apache.org/r/54706/diff/


Testing
---

Tested Hive and Hive2 with Atlas installation on cent-os 6.


test_make_valid_json_6 (test_hive_server_int.TestHiveServerInteractive) ... ok
test_make_valid_json_7 (test_hive_server_int.TestHiveServerInteractive) ... ok
test_make_valid_json_8 (test_hive_server_int.TestHiveServerInteractive) ... ok
test_make_valid_json_9 (test_hive_server_int.TestHiveServerInteractive) ... ok
test_start_default_with_llap_multi_line_output 
(test_hive_server_int.TestHiveServerInteractive) ... ok
test_start_default_with_llap_single_line_output 
(test_hive_server_int.TestHiveServerInteractive) ... ok
test_start_hsi_ha_with_llap_single_line_output 
(test_hive_server_int.TestHiveServerInteractive) ... ok
test_start_with_ha_and_llap_running 
(test_hive_server_int.TestHiveServerInteractive) ... ok


Thanks,

Vishal Suvagia

Re: Review Request 54640: LogSearch - generate default jks file on startup

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54640/#review159001
---


Fix it, then Ship it!




Ship It!


ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
 (line 83)


Maybe create a JIRA for this (and note it here) so that it really gets 
done, else it may be a TODO for a really long time.



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
 (line 235)


Missing JavaDoc



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
 (lines 243 - 266)


Too bad we dont have a common library that we can share between Ambari 
projects.  There is already a class in `ambari-server` that does something like 
this - `org.apache.ambari.server.security.CertificateManager`. Maybe there 
should be an effort to consolicate such features.



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 (line 133)


Missing JavaDoc



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 (line 146)


Missing JavaDoc



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 (line 153)


Missing JavaDoc



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 (line 161)


`MD5WithRSAEncryption` may not be good enough. This should probably be 
configurable or set to something like `sha256WithRSAEncryption`.  Ambari was 
dinged for this a while back since MD5 and SHA1 are considered insecure. 

Ambari handles this by using OpenSSL to create certificates. The `ca.conf` 
file has the digest algorithm set to sha256, and users can change this by 
editting the config file if they want:
```
default_md = sha256
```



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 (line 171)


Missing JavaDoc



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 (line 181)


Missing JavaDoc



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
 (line 194)


Missing JavaDoc


- Robert Levas


On Dec. 12, 2016, 10:51 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54640/
> ---
> 
> (Updated Dec. 12, 2016, 10:51 a.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-19147
> https://issues.apache.org/jira/browse/AMBARI-19147
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Generate JKS file on startup. (if keystore is not specified or not exists)
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml 6bacd23 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  14f83cf 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java
>  206f793 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  2f13710 
> 
> Diff: https://reviews.apache.org/r/54640/diff/
> 
> 
> Testing
> ---
> 
> testing done. manually. (start ambar server, import generated logsearch 
> certificate to ambar keystore)
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 54674: AMBARI-19170. NPE during Ambari server schema upgrade

[Jonathan Hurley]

> On Dec. 12, 2016, 3:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java,
> >  line 1555
> > 
> >
> > Because this is changing data, should the data manipulation be wrapped 
> > in an `executeInTransaction(Runnable)` ?
> 
> Dmytro Grinenko wrote:
> moreover, i'm not sure that @Transactional annotation will have any 
> effect on DDL changes via dbAccessor or any other way to make schema changes 
> in scope of database transaction. (not all database engines support this)
> 
> Dmytro Grinenko wrote:
> P.S. i mean revert of schema alteration, as MySQL for exmple done 
> explicit commit on alter commands
> 
> Attila Doroszlai wrote:
> `@Transactional` on these 2 methods was ignored by Guice until they were 
> changed from `private` to default visibility recently (AMBARI-18966).  The 
> cause of the NPE is that the persistence service, which creates 
> `EntityManagerFactory`, is only 
> [started](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L426)
>  after [performing DDL 
> upgrade](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L424).
> 
> As Dmytro mentioned, not all databases support transactional DDL:
> 
>  * [PostgreSQL: 
> OK](https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis)
>  * [MySQL: 
> NO](http://dev.mysql.com/doc/refman/5.7/en/cannot-roll-back.html)
>  * [Oracle: 
> NO](https://docs.oracle.com/cd/B14117_01/server.101/b10759/statements_1001.htm)
> 
> Attila Doroszlai wrote:
> ReviewBoard ate my link for PostgreSQL: 
> https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis

I understand about the DDL changes - but that's OK since we dont need to roll 
those back. This method makes DML changes down below which should be in a 
transaction.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54674/#review158900
---


On Dec. 12, 2016, 3:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54674/
> ---
> 
> (Updated Dec. 12, 2016, 3:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19170
> https://issues.apache.org/jira/browse/AMBARI-19170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Removed @Transactional
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  063c29539e135433b7b9ce37fa5c32d0de2234f8 
> 
> Diff: https://reviews.apache.org/r/54674/diff/
> 
> 
> Testing
> ---
> 
> Installed 2.2.2.0-460, upgraded to 2.5.0.0-453.
> 
> ```
> [root@c6401 ~]# ambari-server upgrade
> Using python  /usr/bin/python
> Upgrading ambari-server
> INFO: Upgrade Ambari Server
> INFO: Updating Ambari Server properties in ambari.properties ...
> INFO: Updating Ambari Server properties in ambari-env.sh ...
> WARNING: Original file ambari-env.sh kept
> INFO: Fixing database objects owner
> Ambari Server configured for Embedded Postgres. Confirm you have made a 
> backup of the Ambari Server database [y/n] (y)? y
> INFO: Upgrading database schema
> INFO: Return code from schema upgrade command, retcode = 0
> INFO: Schema upgrade completed
> Adjusting ambari-server permissions and ownership...
> Ambari Server 'upgrade' completed successfully.
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 54698: Disable security hook

[Oliver Szabo]

> On Dec. 13, 2016, 2:15 p.m., Laszlo Puskas wrote:
> > ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java, line 
> > 31
> > 
> >
> > You could use the org.apache.commons.cli framework to handle the 
> > arguments. (There are examples in the code base)

+1, you can check how it was done in 
ambari-logsearch/ambari-logsearch-solr-client (AmbariSolrClientCLI.java)


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54698/#review159000
---


On Dec. 13, 2016, 1:51 p.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54698/
> ---
> 
> (Updated Dec. 13, 2016, 1:51 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Laszlo Puskas, Oliver Szabo, Robert 
> Levas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19187
> https://issues.apache.org/jira/browse/AMBARI-19187
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hadoop components need to establish a secure connection with ZooKeeper when 
> Kerberos is enabled. This involves the setup of the correct authentication 
> (JAAS config file) and authorization (per-component Kerberos-backed ACLs on 
> the znodes) between the service and ZooKeeper. Most services are able to set 
> these ACLs based on their config when the user enable kerberos.
> When we disable kerberos again, the sasl ACL should be removed otherwise the 
> services won't be able to access their znodes.
> 
> This issue is about introducing a new command (DISABLE_SECURITY) that will be 
> sent by the ambari server to the services upon the dekerberiztion process. 
> When a service receives this command it will be able to do the zookeeper 
> secure to unsecure migration process (e.g. removing sasl ACLs).
> 
> Notable changes:
>  - Added a java command line tool to the agent project that can setAcls 
> recursively on a znode
>  - Modified the dekerberization workflow:
>  - 1. UI stops all services but zookeeper 
>  - 2. 2 new stages was introduced in the backend (send DISABLE_SECURITY 
> command to the services, start zookeeper)
> 
> 
> Diffs
> -
> 
>   ambari-agent/pom.xml a8ed7f1 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java 
> PRE-CREATION 
>   ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java 
> PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/core/resources/zkmigrator.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 584775e 
>   ambari-server/pom.xml 48ddb52 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  3261a56 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/ActionMetadata.java
>  0064662 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  90f8098 
>   ambari-web/app/controllers/main/admin/kerberos/disable_controller.js 
> cec4503 
> 
> Diff: https://reviews.apache.org/r/54698/diff/
> 
> 
> Testing
> ---
> 
> Added unittests for ZkMigrator, KerberosHelperImpl
> 
> Manual testings: 
>  - created cluster with ambari
>  - enabled kerberos
>  - disabled kerberos
>  - checked if the DISABLE_SECURITY command was sent to the services
> 
> 
> Ambari agent:
> --
> Ran 450 tests in 10.634s
> 
> Ambari server:
> --
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 34:44.448s
> [INFO] Finished at: Tue Dec 13 14:29:00 CET 2016
> [INFO] Final Memory: 160M/798M
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Re: Review Request 54698: Disable security hook

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54698/#review159000
---




ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java (line 31)


You could use the org.apache.commons.cli framework to handle the arguments. 
(There are examples in the code base)


- Laszlo Puskas


On Dec. 13, 2016, 1:51 p.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54698/
> ---
> 
> (Updated Dec. 13, 2016, 1:51 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Laszlo Puskas, Oliver Szabo, Robert 
> Levas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-19187
> https://issues.apache.org/jira/browse/AMBARI-19187
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hadoop components need to establish a secure connection with ZooKeeper when 
> Kerberos is enabled. This involves the setup of the correct authentication 
> (JAAS config file) and authorization (per-component Kerberos-backed ACLs on 
> the znodes) between the service and ZooKeeper. Most services are able to set 
> these ACLs based on their config when the user enable kerberos.
> When we disable kerberos again, the sasl ACL should be removed otherwise the 
> services won't be able to access their znodes.
> 
> This issue is about introducing a new command (DISABLE_SECURITY) that will be 
> sent by the ambari server to the services upon the dekerberiztion process. 
> When a service receives this command it will be able to do the zookeeper 
> secure to unsecure migration process (e.g. removing sasl ACLs).
> 
> Notable changes:
>  - Added a java command line tool to the agent project that can setAcls 
> recursively on a znode
>  - Modified the dekerberization workflow:
>  - 1. UI stops all services but zookeeper 
>  - 2. 2 new stages was introduced in the backend (send DISABLE_SECURITY 
> command to the services, start zookeeper)
> 
> 
> Diffs
> -
> 
>   ambari-agent/pom.xml a8ed7f1 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java 
> PRE-CREATION 
>   ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java 
> PRE-CREATION 
>   ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java 
> PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/core/resources/zkmigrator.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 584775e 
>   ambari-server/pom.xml 48ddb52 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  3261a56 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/ActionMetadata.java
>  0064662 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  90f8098 
>   ambari-web/app/controllers/main/admin/kerberos/disable_controller.js 
> cec4503 
> 
> Diff: https://reviews.apache.org/r/54698/diff/
> 
> 
> Testing
> ---
> 
> Added unittests for ZkMigrator, KerberosHelperImpl
> 
> Manual testings: 
>  - created cluster with ambari
>  - enabled kerberos
>  - disabled kerberos
>  - checked if the DISABLE_SECURITY command was sent to the services
> 
> 
> Ambari agent:
> --
> Ran 450 tests in 10.634s
> 
> Ambari server:
> --
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 34:44.448s
> [INFO] Finished at: Tue Dec 13 14:29:00 CET 2016
> [INFO] Final Memory: 160M/798M
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Review Request 54695: NameNode JVM has many parameters repeated 3 or more times

[Yuanbo Liu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54695/
---

Review request for Ambari.


Bugs: AMBARI-19185
https://issues.apache.org/jira/browse/AMBARI-19185


Repository: ambari


Description
---

When using Ambari to start NameNode, then looking at namenode process running 
in Linux, we find that there are many repeated jvm options in the start 
command. It will be good if we can get rid of the redundant options.


Diffs
-

  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
 c2f37c1 

Diff: https://reviews.apache.org/r/54695/diff/


Testing
---


Thanks,

Yuanbo Liu

Re: Review Request 54674: AMBARI-19170. NPE during Ambari server schema upgrade

[Attila Doroszlai]

> On Dec. 12, 2016, 9:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java,
> >  line 1555
> > 
> >
> > Because this is changing data, should the data manipulation be wrapped 
> > in an `executeInTransaction(Runnable)` ?
> 
> Dmytro Grinenko wrote:
> moreover, i'm not sure that @Transactional annotation will have any 
> effect on DDL changes via dbAccessor or any other way to make schema changes 
> in scope of database transaction. (not all database engines support this)
> 
> Dmytro Grinenko wrote:
> P.S. i mean revert of schema alteration, as MySQL for exmple done 
> explicit commit on alter commands
> 
> Attila Doroszlai wrote:
> `@Transactional` on these 2 methods was ignored by Guice until they were 
> changed from `private` to default visibility recently (AMBARI-18966).  The 
> cause of the NPE is that the persistence service, which creates 
> `EntityManagerFactory`, is only 
> [started](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L426)
>  after [performing DDL 
> upgrade](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L424).
> 
> As Dmytro mentioned, not all databases support transactional DDL:
> 
>  * [PostgreSQL: 
> OK](https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis)
>  * [MySQL: 
> NO](http://dev.mysql.com/doc/refman/5.7/en/cannot-roll-back.html)
>  * [Oracle: 
> NO](https://docs.oracle.com/cd/B14117_01/server.101/b10759/statements_1001.htm)

ReviewBoard ate my link for PostgreSQL: 
https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54674/#review158900
---


On Dec. 12, 2016, 9:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54674/
> ---
> 
> (Updated Dec. 12, 2016, 9:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19170
> https://issues.apache.org/jira/browse/AMBARI-19170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Removed @Transactional
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  063c29539e135433b7b9ce37fa5c32d0de2234f8 
> 
> Diff: https://reviews.apache.org/r/54674/diff/
> 
> 
> Testing
> ---
> 
> Installed 2.2.2.0-460, upgraded to 2.5.0.0-453.
> 
> ```
> [root@c6401 ~]# ambari-server upgrade
> Using python  /usr/bin/python
> Upgrading ambari-server
> INFO: Upgrade Ambari Server
> INFO: Updating Ambari Server properties in ambari.properties ...
> INFO: Updating Ambari Server properties in ambari-env.sh ...
> WARNING: Original file ambari-env.sh kept
> INFO: Fixing database objects owner
> Ambari Server configured for Embedded Postgres. Confirm you have made a 
> backup of the Ambari Server database [y/n] (y)? y
> INFO: Upgrading database schema
> INFO: Return code from schema upgrade command, retcode = 0
> INFO: Schema upgrade completed
> Adjusting ambari-server permissions and ownership...
> Ambari Server 'upgrade' completed successfully.
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 54674: AMBARI-19170. NPE during Ambari server schema upgrade

[Attila Doroszlai]

> On Dec. 12, 2016, 9:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java,
> >  line 1555
> > 
> >
> > Because this is changing data, should the data manipulation be wrapped 
> > in an `executeInTransaction(Runnable)` ?
> 
> Dmytro Grinenko wrote:
> moreover, i'm not sure that @Transactional annotation will have any 
> effect on DDL changes via dbAccessor or any other way to make schema changes 
> in scope of database transaction. (not all database engines support this)
> 
> Dmytro Grinenko wrote:
> P.S. i mean revert of schema alteration, as MySQL for exmple done 
> explicit commit on alter commands

`@Transactional` on these 2 methods was ignored by Guice until they were 
changed from `private` to default visibility recently (AMBARI-18966).  The 
cause of the NPE is that the persistence service, which creates 
`EntityManagerFactory`, is only 
[started](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L426)
 after [performing DDL 
upgrade](https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java#L424).

As Dmytro mentioned, not all databases support transactional DDL:

 * [PostgreSQL: 
OK](https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis)
 * [MySQL: NO](http://dev.mysql.com/doc/refman/5.7/en/cannot-roll-back.html)
 * [Oracle: 
NO](https://docs.oracle.com/cd/B14117_01/server.101/b10759/statements_1001.htm)


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54674/#review158900
---


On Dec. 12, 2016, 9:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54674/
> ---
> 
> (Updated Dec. 12, 2016, 9:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-19170
> https://issues.apache.org/jira/browse/AMBARI-19170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Removed @Transactional
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  063c29539e135433b7b9ce37fa5c32d0de2234f8 
> 
> Diff: https://reviews.apache.org/r/54674/diff/
> 
> 
> Testing
> ---
> 
> Installed 2.2.2.0-460, upgraded to 2.5.0.0-453.
> 
> ```
> [root@c6401 ~]# ambari-server upgrade
> Using python  /usr/bin/python
> Upgrading ambari-server
> INFO: Upgrade Ambari Server
> INFO: Updating Ambari Server properties in ambari.properties ...
> INFO: Updating Ambari Server properties in ambari-env.sh ...
> WARNING: Original file ambari-env.sh kept
> INFO: Fixing database objects owner
> Ambari Server configured for Embedded Postgres. Confirm you have made a 
> backup of the Ambari Server database [y/n] (y)? y
> INFO: Upgrading database schema
> INFO: Return code from schema upgrade command, retcode = 0
> INFO: Schema upgrade completed
> Adjusting ambari-server permissions and ownership...
> Ambari Server 'upgrade' completed successfully.
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 54698: Disable security hook

[Attila Magyar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54698/
---

Review request for Ambari, Jaimin Jetly, Laszlo Puskas, Oliver Szabo, Robert 
Levas, and Sebastian Toader.


Bugs: AMBARI-19187
https://issues.apache.org/jira/browse/AMBARI-19187


Repository: ambari


Description
---

Hadoop components need to establish a secure connection with ZooKeeper when 
Kerberos is enabled. This involves the setup of the correct authentication 
(JAAS config file) and authorization (per-component Kerberos-backed ACLs on the 
znodes) between the service and ZooKeeper. Most services are able to set these 
ACLs based on their config when the user enable kerberos.
When we disable kerberos again, the sasl ACL should be removed otherwise the 
services won't be able to access their znodes.

This issue is about introducing a new command (DISABLE_SECURITY) that will be 
sent by the ambari server to the services upon the dekerberiztion process. When 
a service receives this command it will be able to do the zookeeper secure to 
unsecure migration process (e.g. removing sasl ACLs).

Notable changes:
 - Added a java command line tool to the agent project that can setAcls 
recursively on a znode
 - Modified the dekerberization workflow:
 - 1. UI stops all services but zookeeper 
 - 2. 2 new stages was introduced in the backend (send DISABLE_SECURITY command 
to the services, start zookeeper)


Diffs
-

  ambari-agent/pom.xml a8ed7f1 
  ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkAcl.java PRE-CREATION 
  ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java 
PRE-CREATION 
  ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java 
PRE-CREATION 
  ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkMigratorTest.java 
PRE-CREATION 
  
ambari-common/src/main/python/resource_management/core/resources/zkmigrator.py 
PRE-CREATION 
  ambari-common/src/main/python/resource_management/libraries/script/script.py 
584775e 
  ambari-server/pom.xml 48ddb52 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 3261a56 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/ActionMetadata.java
 0064662 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 90f8098 
  ambari-web/app/controllers/main/admin/kerberos/disable_controller.js cec4503 

Diff: https://reviews.apache.org/r/54698/diff/


Testing
---

Added unittests for ZkMigrator, KerberosHelperImpl

Manual testings: 
 - created cluster with ambari
 - enabled kerberos
 - disabled kerberos
 - checked if the DISABLE_SECURITY command was sent to the services


Ambari agent:
--
Ran 450 tests in 10.634s

Ambari server:
--
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 34:44.448s
[INFO] Finished at: Tue Dec 13 14:29:00 CET 2016
[INFO] Final Memory: 160M/798M


Thanks,

Attila Magyar

Review Request 54704: Remove UI logic to set or expect hive_server_interactive_host

[Andrii Tkach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54704/
---

Review request for Ambari, Aleksandr Kovalenko and Jaimin Jetly.


Bugs: AMBARI-19190
https://issues.apache.org/jira/browse/AMBARI-19190


Repository: ambari


Description
---

With the support for HSI HA, the property no longer seems appropriate. Lets 
remove the dependency on this property and no longer expect or use it.


Diffs
-

  ambari-web/app/controllers/wizard/step7/assign_master_controller.js 1ee0c45 
  ambari-web/app/mixins/main/service/configs/component_actions_by_configs.js 
347bdde 
  ambari-web/app/models/configs/theme/config_action.js 9ac4e7a 
  ambari-web/app/views/common/configs/widgets/config_widget_view.js 2c90cc3 
  ambari-web/test/controllers/wizard/step7/assign_master_controller_test.js 
053b900 
  
ambari-web/test/mixins/main/service/configs/component_actions_by_configs_test.js
 1a93a26 

Diff: https://reviews.apache.org/r/54704/diff/


Testing
---

19701 passing (32s)
153 pending


Thanks,

Andrii Tkach

Re: Review Request 54477: Update zeppelin configuration for ambari 2.5

[Renjith Kamath]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54477/#review158995
---




ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/metainfo.xml
 (line 1)


Change the version to 0.6.0.2.5 instead of 0.7.0.2.6. These are not new 
config files.


- Renjith Kamath


On Dec. 10, 2016, 3:13 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54477/
> ---
> 
> (Updated Dec. 10, 2016, 3:13 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Jayush Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19123
> https://issues.apache.org/jira/browse/AMBARI-19123
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update zeppelin configuration for ambari 2.5, and add more configuration 
> template for shiro.ini
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/configuration/zeppelin-config.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/configuration/zeppelin-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/configuration/zeppelin-logsearch-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/metainfo.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54477/diff/
> 
> 
> Testing
> ---
> 
> Manually on Centos 6.6
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 54703: Add host component should show configs being changed in recommendation popup modal

[Alexandr Antonenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54703/#review158994
---


Ship it!




Ship It!

- Alexandr Antonenko


On Dec. 13, 2016, 1:10 p.m., Andriy Babiichuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54703/
> ---
> 
> (Updated Dec. 13, 2016, 1:10 p.m.)
> 
> 
> Review request for Ambari and Alexandr Antonenko.
> 
> 
> Bugs: AMBARI-19189
> https://issues.apache.org/jira/browse/AMBARI-19189
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **STR:**
> 1. Navigate to Hosts details page and click on Add button
> 2. While adding components like ZooKeeper Server and some other host 
> components, ambari-web also changes some configurations
> 
> **Actual Result:** Before changing configurations, ambari-web warns user that 
> a list of configurations will be changed in a modal popup. But UI does not 
> show current and the new value that will be changed to the end-user in the 
> popup, neither does it allow user to add component without changing 
> configurations if user wants to do that
> **Expected Result:** Similar popup with the ux as the one shown for the stack 
> advisor recommendation response should be shown which shows current value and 
> new value of the config with checkboxes which user can choose to uncheck.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/host/details.js ecb3219 
>   ambari-web/app/messages.js 26f2ce5 
>   ambari-web/app/templates/common/modal_popups/dependent_configs_list.hbs 
> d117dd1 
>   
> ambari-web/app/templates/main/host/details/addComponentWithConfigsChanges.hbs 
> PRE-CREATION 
>   ambari-web/app/views/common/modal_popups/dependent_configs_list_popup.js 
> 728634e 
>   ambari-web/test/controllers/main/host/details_test.js 9a56ab2 
> 
> Diff: https://reviews.apache.org/r/54703/diff/
> 
> 
> Testing
> ---
> 
> 19881 tests complete (22 seconds)
>   153 tests pending
> 
> 
> Thanks,
> 
> Andriy Babiichuk
> 
>

Review Request 54703: Add host component should show configs being changed in recommendation popup modal

[Andriy Babiichuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54703/
---

Review request for Ambari and Alexandr Antonenko.


Bugs: AMBARI-19189
https://issues.apache.org/jira/browse/AMBARI-19189


Repository: ambari


Description
---

**STR:**
1. Navigate to Hosts details page and click on Add button
2. While adding components like ZooKeeper Server and some other host 
components, ambari-web also changes some configurations

**Actual Result:** Before changing configurations, ambari-web warns user that a 
list of configurations will be changed in a modal popup. But UI does not show 
current and the new value that will be changed to the end-user in the popup, 
neither does it allow user to add component without changing configurations if 
user wants to do that
**Expected Result:** Similar popup with the ux as the one shown for the stack 
advisor recommendation response should be shown which shows current value and 
new value of the config with checkboxes which user can choose to uncheck.


Diffs
-

  ambari-web/app/controllers/main/host/details.js ecb3219 
  ambari-web/app/messages.js 26f2ce5 
  ambari-web/app/templates/common/modal_popups/dependent_configs_list.hbs 
d117dd1 
  ambari-web/app/templates/main/host/details/addComponentWithConfigsChanges.hbs 
PRE-CREATION 
  ambari-web/app/views/common/modal_popups/dependent_configs_list_popup.js 
728634e 
  ambari-web/test/controllers/main/host/details_test.js 9a56ab2 

Diff: https://reviews.apache.org/r/54703/diff/


Testing
---

19881 tests complete (22 seconds)
  153 tests pending


Thanks,

Andriy Babiichuk

Re: Review Request 53980: AMBARI-18945 Allowing user to edit service name while enabling ranger plugin

[Mugdha Varadkar]

> On Dec. 8, 2016, 2:37 p.m., Jonathan Hurley wrote:
> > One thing I'm not clear on is how this will affect an existing 
> > installation. Won't Ranger have a problem if you change the name of a repo 
> > on an already running cluster?

Reverting the logic


> On Dec. 8, 2016, 2:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java,
> >  line 384
> > 
> >
> > I don't think this can be null/empty - ever.

Reverting the logic


> On Dec. 8, 2016, 2:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py, 
> > lines 221-229
> > 
> >
> > This is a lot of logic repeated over and over - can this be placed in a 
> > common function and just re-used in order to get the name of the repo? 
> > Similar to what you did in the upgrade?

Reverting the logic


> On Dec. 8, 2016, 2:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java,
> >  line 377
> > 
> >
> > Doc.

Reverting the logic


> On Dec. 8, 2016, 2:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java,
> >  line 362
> > 
> >
> > Doc.

Reverting the logic


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53980/#review158525
---


On Dec. 13, 2016, 12:40 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53980/
> ---
> 
> (Updated Dec. 13, 2016, 12:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18945
> https://issues.apache.org/jira/browse/AMBARI-18945
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Need to add a provision for creating default repository with static name or 
> user provided name.
> 
> This will help user to provided service name under Ranger supported plugins 
> config tab and can take dump of latest snapshot of all policies and use it 
> while recreating another/multiple clusters with same service name.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  32e1510 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  f47dc8f 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  d86e360 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  e9436c1 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  5635fe3 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  4cefbda 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  c20bdde 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  4859534 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  1347864 
> 
> Diff: https://reviews.apache.org/r/53980/diff/
> 
> 
> Testing
> ---
> 
> mvn -DskipPythonTests -Dtest=UpgradeCatalog250Test test
> 
> ---
>  T E S T S
> ---
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support 
> was removed in 8.0
> Running org.apache.ambari.server.upgrade.UpgradeCatalog250Test
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.548 sec - 
> in org.apache.ambari.server.upgrade.UpgradeCatalog250Test
> 
> Results :
> 
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 30.347 s
> [INFO] Finished at: 2016-11-22T11:51:23+00:00
> [INFO] Final Memory: 63M/1107M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 53980: AMBARI-18945 Allowing user to edit service name while enabling ranger plugin

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53980/
---

(Updated Dec. 13, 2016, 12:40 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan Hurley, 
Sumit Mohanty, and Velmurugan Periasamy.


Changes
---

Reverting the logic for UpgradeCatalog250.java and stack_advisor.py, because 
cluster_name key is not available under clusterData['referenceHost'] during 
first installation of cluster.

This does not fail for UI based installation because Ranger has to be installed 
after cluster setup(by which time the cluster_name is available)

However, this will fail at fetching the cluster_name for blue-print based 
installation(with config_recommendation_strategy set as 
ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES) where ranger-plugin is enabled.


Bugs: AMBARI-18945
https://issues.apache.org/jira/browse/AMBARI-18945


Repository: ambari


Description
---

Need to add a provision for creating default repository with static name or 
user provided name.

This will help user to provided service name under Ranger supported plugins 
config tab and can take dump of latest snapshot of all policies and use it 
while recreating another/multiple clusters with same service name.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 32e1510 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 f47dc8f 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 d86e360 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 e9436c1 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
 5635fe3 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
 4cefbda 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 c20bdde 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 4859534 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 1347864 

Diff: https://reviews.apache.org/r/53980/diff/


Testing
---

mvn -DskipPythonTests -Dtest=UpgradeCatalog250Test test

---
 T E S T S
---
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was 
removed in 8.0
Running org.apache.ambari.server.upgrade.UpgradeCatalog250Test
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.548 sec - in 
org.apache.ambari.server.upgrade.UpgradeCatalog250Test

Results :

Tests run: 13, Failures: 0, Errors: 0, Skipped: 0


[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 30.347 s
[INFO] Finished at: 2016-11-22T11:51:23+00:00
[INFO] Final Memory: 63M/1107M
[INFO] 


Thanks,

Mugdha Varadkar

Re: Review Request 54477: Update zeppelin configuration for ambari 2.5

[Renjith Kamath]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54477/#review158993
---


Ship it!




Ship It!

- Renjith Kamath


On Dec. 10, 2016, 3:13 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54477/
> ---
> 
> (Updated Dec. 10, 2016, 3:13 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Jayush Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19123
> https://issues.apache.org/jira/browse/AMBARI-19123
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update zeppelin configuration for ambari 2.5, and add more configuration 
> template for shiro.ini
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/configuration/zeppelin-config.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/configuration/zeppelin-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/configuration/zeppelin-logsearch-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0.2.6/metainfo.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54477/diff/
> 
> 
> Testing
> ---
> 
> Manually on Centos 6.6
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 54543: AMBARI-19140 Ambari Server symbolic links has double // when root dir is just /

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54543/#review158988
---


Ship it!




Ship It!

- Andrew Onischuk


On Dec. 12, 2016, 7:58 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54543/
> ---
> 
> (Updated Dec. 12, 2016, 7:58 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Tim Thorpe.
> 
> 
> Bugs: AMBARI-19140
> https://issues.apache.org/jira/browse/AMBARI-19140
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Notice the "//" in the symbolic link , where it could have been trimmed to be 
> just one leading "/"
> 
> [root]# ll /usr/sbin/ambari-server
> lrwxrwxrwx 1 root root 26 Dec 5 11:27 /usr/sbin/ambari-server -> 
> //etc/init.d/ambari-server
> [root]# clear
> [root]# ll
> total 20
> drwxrwxr-x 4 root root 8192 Dec 5 11:32 ambari_agent
> lrwxrwxrwx 1 root root 42 Dec 5 11:27 ambari_commons -> 
> //usr/lib/ambari-server/lib/ambari_commons
> lrwxrwxrwx 1 root root 41 Dec 5 11:27 ambari_jinja2 -> 
> //usr/lib/ambari-server/lib/ambari_jinja2
> drwxrwxr-x 2 root root 4096 Dec 5 11:28 ambari_server
> lrwxrwxrwx 1 root root 45 Dec 5 11:27 ambari_simplejson -> 
> //usr/lib/ambari-server/lib/ambari_simplejson
> lrwxrwxrwx 1 root root 47 Dec 5 11:27 resource_management -> 
> //usr/lib/ambari-server/lib/resource_management
> drwxr-xr-x 5 ams hadoop 83 Dec 6 09:26 resource_monitoring
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/install-helper.sh 34ec0e9 
>   ambari-server/src/main/package/rpm/posttrans_server.sh a719cae 
>   ambari-server/src/main/package/rpm/preinstall.sh 524fea8 
> 
> Diff: https://reviews.apache.org/r/54543/diff/
> 
> 
> Testing
> ---
> 
> build Ambari RPMs, install a trunk cluster with my own RPMs, verify the 
> double "/"s are trimmed now.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 54666: DB configs consistency check failed after disable kerberos or Ambari upgrade

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54666/#review158984
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Гру. 13, 2016, 10:55 до полудня, Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54666/
> ---
> 
> (Updated Гру. 13, 2016, 10:55 до полудня)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-19175
> https://issues.apache.org/jira/browse/AMBARI-19175
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> DB configs consistency check failed after disable kerberos.
> Services start successful and service check passed.
> 
> Added more detailed logging. Also made "configs not mapped to any service" a 
> warning that does not affect server start
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  d40dfd2124 
> 
> Diff: https://reviews.apache.org/r/54666/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 54413: AMBARI-19031 : UI Enhancements, import/export assets and smart version configuration

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54413/#review158983
---


Ship it!




Ship It!

- Nitiraj Rathore


On Dec. 9, 2016, 11:05 a.m., Padma Priya N wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54413/
> ---
> 
> (Updated Dec. 9, 2016, 11:05 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Nitiraj Rathore, 
> and Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19031
> https://issues.apache.org/jira/browse/AMBARI-19031
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> UX Enhancemenets 
>  - Dashboard as a tab
>  - Coordinator UI changes
>  - Bundle smart name autofil
> Features 
>  - Version setting picked up from Oozie
>  - Ability to import/export assets(actions)
> 
> 
> Diffs
> -
> 
>   contrib/views/wfmanager/pom.xml d4aedf3 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/AmbariIOUtil.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/HDFSFileUtils.java
>  58c3980 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/OozieProxyImpersonator.java
>  0533f04 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/OozieUtils.java
>  170132f 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/WorkflowFileInfo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/WorkflowFilesService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetRepo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/model/ActionAsset.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/model/BaseModel.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowManagerService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowsManagerResource.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowsRepo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/model/Workflow.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-config.js 
> 2799db5 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-coord-config.js
>  229b2cc 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-version-settings.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-config.js 
> 57fcdf8 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-version-settings.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/date-with-expr.js
>  dd9a9ae 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
>  a3d64b0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> ed743f4 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/hdfs-browser.js 
> 232df16 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/import-from-stream.js
>  PRE-CREATION 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-config.js 
> 4fa2666 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-details.js 
> e403dc4 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-row.js 
> 297244a 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/name-value-config.js
>  ae453a0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/preview-dialog.js
>  31f9df8 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/save-wf.js 
> b41f743 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/search-create-new-bar.js
>  420d3b0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-actions.js
>  7fc1c94 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-icon.js 
> PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/controllers/design/dashboardtab.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/controllers/design/jobtab.js
>  PRE-CREATION 
>   contrib/views/wfmanager/src/main/resources/ui/app/controllers/job.js 
> 2c90be4 
>   
> 

Re: Review Request 54413: AMBARI-19031 : UI Enhancements, import/export assets and smart version configuration

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54413/#review158982
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On Dec. 9, 2016, 11:05 a.m., Padma Priya N wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54413/
> ---
> 
> (Updated Dec. 9, 2016, 11:05 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Nitiraj Rathore, 
> and Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19031
> https://issues.apache.org/jira/browse/AMBARI-19031
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> UX Enhancemenets 
>  - Dashboard as a tab
>  - Coordinator UI changes
>  - Bundle smart name autofil
> Features 
>  - Version setting picked up from Oozie
>  - Ability to import/export assets(actions)
> 
> 
> Diffs
> -
> 
>   contrib/views/wfmanager/pom.xml d4aedf3 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/AmbariIOUtil.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/HDFSFileUtils.java
>  58c3980 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/OozieProxyImpersonator.java
>  0533f04 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/OozieUtils.java
>  170132f 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/WorkflowFileInfo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/WorkflowFilesService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetRepo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/model/ActionAsset.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/model/BaseModel.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowManagerService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowsManagerResource.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowsRepo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/model/Workflow.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-config.js 
> 2799db5 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-coord-config.js
>  229b2cc 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-version-settings.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-config.js 
> 57fcdf8 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-version-settings.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/date-with-expr.js
>  dd9a9ae 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
>  a3d64b0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> ed743f4 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/hdfs-browser.js 
> 232df16 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/import-from-stream.js
>  PRE-CREATION 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-config.js 
> 4fa2666 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-details.js 
> e403dc4 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-row.js 
> 297244a 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/name-value-config.js
>  ae453a0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/preview-dialog.js
>  31f9df8 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/save-wf.js 
> b41f743 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/search-create-new-bar.js
>  420d3b0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-actions.js
>  7fc1c94 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-icon.js 
> PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/controllers/design/dashboardtab.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/controllers/design/jobtab.js
>  PRE-CREATION 
>   contrib/views/wfmanager/src/main/resources/ui/app/controllers/job.js 
> 2c90be4 
>   
> 

Re: Review Request 54413: AMBARI-19031 : UI Enhancements, import/export assets and smart version configuration

[DIPAYAN BHOWMICK]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54413/#review158981
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On Dec. 9, 2016, 11:05 a.m., Padma Priya N wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54413/
> ---
> 
> (Updated Dec. 9, 2016, 11:05 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Nitiraj Rathore, 
> and Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-19031
> https://issues.apache.org/jira/browse/AMBARI-19031
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> UX Enhancemenets 
>  - Dashboard as a tab
>  - Coordinator UI changes
>  - Bundle smart name autofil
> Features 
>  - Version setting picked up from Oozie
>  - Ability to import/export assets(actions)
> 
> 
> Diffs
> -
> 
>   contrib/views/wfmanager/pom.xml d4aedf3 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/AmbariIOUtil.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/HDFSFileUtils.java
>  58c3980 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/OozieProxyImpersonator.java
>  0533f04 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/OozieUtils.java
>  170132f 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/WorkflowFileInfo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/WorkflowFilesService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetRepo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/AssetService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/assets/model/ActionAsset.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/model/BaseModel.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowManagerService.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowsManagerResource.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/WorkflowsRepo.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/workflowmanager/model/Workflow.java
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-config.js 
> 2799db5 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-coord-config.js
>  229b2cc 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/bundle-version-settings.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-config.js 
> 57fcdf8 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/coord-version-settings.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/date-with-expr.js
>  dd9a9ae 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/designer-workspace.js
>  a3d64b0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> ed743f4 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/hdfs-browser.js 
> 232df16 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/import-from-stream.js
>  PRE-CREATION 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-config.js 
> 4fa2666 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-details.js 
> e403dc4 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/job-row.js 
> 297244a 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/name-value-config.js
>  ae453a0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/preview-dialog.js
>  31f9df8 
>   contrib/views/wfmanager/src/main/resources/ui/app/components/save-wf.js 
> b41f743 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/search-create-new-bar.js
>  420d3b0 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-actions.js
>  7fc1c94 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-icon.js 
> PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/controllers/design/dashboardtab.js
>  PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/controllers/design/jobtab.js
>  PRE-CREATION 
>   contrib/views/wfmanager/src/main/resources/ui/app/controllers/job.js 
> 2c90be4 
>   
> 

Re: Review Request 54583: New REST API in Log Search Portal that returns the log files of the host components

[Miklos Gergely]

> On Dec. 12, 2016, 4:01 p.m., Robert Nettleton wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java,
> >  line 95
> > 
> >
> > Minor spelling issue:
> > 
> > "componentes" should be "components"

Thanks


- Miklos


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54583/#review158862
---


On Dec. 13, 2016, 10:19 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54583/
> ---
> 
> (Updated Dec. 13, 2016, 10:19 a.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19151
> https://issues.apache.org/jira/browse/AMBARI-19151
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari Server fetches the log file names of the host components by fetching 
> the log entries, and get the log file field of the first entry it founds. 
> This new API replaces this logic, it explicitly returns the log files of a 
> host component, and thus giving the control to the Log Search server.
> 
> Also removed a few unused imports.
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml 6bacd23 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  230ca8c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/converter/HostLogFilesRequestQueryConverter.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/converter/ServiceLogAnyGraphRequestQueryConverter.java
>  57bac10 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
>  805c5a0 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
>  75ac819 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
>  3ee1d17 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
>  fc2f0f7 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/HostComponentParamDefinition.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/ServiceLogParamDefinition.java
>  752f786 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/BaseServiceLogRequest.java
>  bafca37 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/HostLogFilesRequest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/ServiceAnyGraphRequest.java
>  1fb4bf6 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/HostLogFilesResponse.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/NodeData.java
>  41d8a31 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsResource.java
>  3e1e120 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/ResponseDataGenerator.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/HostLogFilesRequestQueryConverterTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/ServiceLogTreeRequestFacetQueryConverterTest.java
>  107a99d 
>   
> ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/UserConfigRequestQueryConverterTest.java
>  adeaf5b 
> 
> Diff: https://reviews.apache.org/r/54583/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster, new unit test added.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 54583: New REST API in Log Search Portal that returns the log files of the host components

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54583/
---

(Updated Dec. 13, 2016, 10:19 a.m.)


Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.


Changes
---

typo fixed


Bugs: AMBARI-19151
https://issues.apache.org/jira/browse/AMBARI-19151


Repository: ambari


Description
---

Ambari Server fetches the log file names of the host components by fetching the 
log entries, and get the log file field of the first entry it founds. This new 
API replaces this logic, it explicitly returns the log files of a host 
component, and thus giving the control to the Log Search server.

Also removed a few unused imports.


Diffs (updated)
-

  ambari-logsearch/ambari-logsearch-portal/pom.xml 6bacd23 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
 230ca8c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/converter/HostLogFilesRequestQueryConverter.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/converter/ServiceLogAnyGraphRequestQueryConverter.java
 57bac10 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
 805c5a0 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
 75ac819 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
 3ee1d17 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
 fc2f0f7 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/HostComponentParamDefinition.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/ServiceLogParamDefinition.java
 752f786 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/BaseServiceLogRequest.java
 bafca37 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/HostLogFilesRequest.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/ServiceAnyGraphRequest.java
 1fb4bf6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/HostLogFilesResponse.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/NodeData.java
 41d8a31 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsResource.java
 3e1e120 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/ResponseDataGenerator.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/HostLogFilesRequestQueryConverterTest.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/ServiceLogTreeRequestFacetQueryConverterTest.java
 107a99d 
  
ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/UserConfigRequestQueryConverterTest.java
 adeaf5b 

Diff: https://reviews.apache.org/r/54583/diff/


Testing
---

Tested on local cluster, new unit test added.


Thanks,

Miklos Gergely

Re: Review Request 54679: Perf: Deploy 3000 Agent cluster and find perf bugs.Part 3

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54679/#review158978
---


Ship it!




Ship It!

- Dmytro Sen


On Дек. 12, 2016, 10:43 п.п., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54679/
> ---
> 
> (Updated Дек. 12, 2016, 10:43 п.п.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19058
> https://issues.apache.org/jira/browse/AMBARI-19058
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Sometimes alerts are not available and service scripts disappear.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/agent-multiplier.py 7c0774c 
> 
> Diff: https://reviews.apache.org/r/54679/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 54583: New REST API in Log Search Portal that returns the log files of the host components

[Miklos Gergely]

> On Dec. 11, 2016, 11:15 p.m., Oliver Szabo wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/HostLogFilesRequestQueryConverterTest.java,
> >  line 54
> > 
> >
> > although test name is without data, you set component name and hostname 
> > as well

Fair point.


- Miklos


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54583/#review158808
---


On Dec. 12, 2016, 9:48 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54583/
> ---
> 
> (Updated Dec. 12, 2016, 9:48 a.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-19151
> https://issues.apache.org/jira/browse/AMBARI-19151
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari Server fetches the log file names of the host components by fetching 
> the log entries, and get the log file field of the first entry it founds. 
> This new API replaces this logic, it explicitly returns the log files of a 
> host component, and thus giving the control to the Log Search server.
> 
> Also removed a few unused imports.
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml 6bacd23 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  230ca8c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/converter/HostLogFilesRequestQueryConverter.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/converter/ServiceLogAnyGraphRequestQueryConverter.java
>  57bac10 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
>  805c5a0 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
>  75ac819 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
>  3ee1d17 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
>  fc2f0f7 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/HostComponentParamDefinition.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/ServiceLogParamDefinition.java
>  752f786 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/BaseServiceLogRequest.java
>  bafca37 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/HostLogFilesRequest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/request/impl/ServiceAnyGraphRequest.java
>  1fb4bf6 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/HostLogFilesResponse.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/NodeData.java
>  41d8a31 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsResource.java
>  3e1e120 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/ResponseDataGenerator.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/HostLogFilesRequestQueryConverterTest.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/ServiceLogTreeRequestFacetQueryConverterTest.java
>  107a99d 
>   
> ambari-logsearch/ambari-logsearch-portal/src/test/java/org/apache/ambari/logsearch/converter/UserConfigRequestQueryConverterTest.java
>  adeaf5b 
> 
> Diff: https://reviews.apache.org/r/54583/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster, new unit test added.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>