Re: Review Request 60249: AMBARI-21234 Ambari rack awareness for Kafka

[Ambud Sharma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60249/
---

(Updated July 11, 2017, 6:18 a.m.)


Review request for Ambari, Alejandro Fernandez and Sriharsha Chintalapani.


Changes
---

2.5 branch patch


Repository: ambari


Description
---

Ambari rack awareness for Kafka. 
https://issues.apache.org/jira/browse/AMBARI-21234

As an operations person it would be nice to manage Kafka rack awareness via 
Ambari. Ambari allows node rack information to be configured and this 
information can then be pulled in the Kafka stack and populated in the 
server.properties file for Kafka.
Design:
This stack change uses the /clusterHostInfo/all_hosts and 
/clusterHostInfo/all_racks paths and materializes them to a variable. Then it 
uses linear search to find this node in the list of all hosts and it's 
corresponding rack id. This information is then stored in a variable called 
rack and which is materialized during the configure method of the broker 
scripts.
This stack change relies on the node rack information stored in Ambari 
therefore will enable both Ambari UI and Blueprints to be used for setting up 
Kafka broker rack information.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/KAFKA/0.10.0/configuration/ranger-kafka-audit.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 1327090aa7 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
 c36a10ff28 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/KAFKA/metainfo.xml 
12f6c45396 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/KAFKA/metainfo.xml 
fb281361a1 


Diff: https://reviews.apache.org/r/60249/diff/7/

Changes: https://reviews.apache.org/r/60249/diff/6-7/


Testing
---

Manually deployed Kafka cluster and verified the broker.rack property is 
correctly populated.


Thanks,

Ambud Sharma

Re: Review Request 60756: AMBARI-21440. Cross-stack migration, package supplemental stack in RPM

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60756/#review180147
---


Ship it!




Ship It!

- Sid Wagle


On July 10, 2017, 11:38 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60756/
> ---
> 
> (Updated July 10, 2017, 11:38 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Grinenko, Jonathan 
> Hurley, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21440
> https://issues.apache.org/jira/browse/AMBARI-21440
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When trying to upgrade ambari-server, it fails with error: 
> ```
> Exception in thread "main" org.apache.ambari.server.AmbariException: Stack 
> data, Stack BigInsights 4.2.5 is not found in Ambari metainfo
> ```
> 
> For the server, package /var/lib/ambari-server/resources/stacks/BigInsights
> For the agent, package /var/lib/ambari-agent/cache/stacks/BigInsights
> 
> 
> Diffs
> -
> 
>   ambari-agent/pom.xml 475adf2 
>   ambari-server/pom.xml 63e704c 
>   ambari-server/src/main/assemblies/server.xml 74d96a4 
>   dev-support/docker/docker/bin/ambaribuild.py dfb7d60 
>   dev-support/docker/docker/bin/test/ambaribuild_test.py a324f5f 
>   pom.xml 8a17956 
> 
> 
> Diff: https://reviews.apache.org/r/60756/diff/1/
> 
> 
> Testing
> ---
> 
> Verified the new RPMs,
> 
> ```
> # Generate the RPM
> mvn -B install package rpm:rpm -pl 
> utility,ambari-views,ambari-metrics/ambari-metrics-common,ambari-server,ambari-agent
>  -Dmaven.clover.skip=true -Dfindbugs.skip=true -DskipTests=true 
> -Dstack.distribution=HDP -Dsupplemental.distribution=BigInsights 
> -Dpython.ver="python >= 2.6" -Drat.ignoreErrors=true
> 
> # Verify the contents
> rpm -qlp rpm -qlp ambari-server-2.5.1.0-0.x86_64.rpm | grep 
> "var/lib/ambari-server/resources/stacks/BigInsights"
> 
> rpm -qlp rpm -qlp ambari-agent-2.5.1.0-0.x86_64.rpm | grep 
> "/var/lib/ambari-agent/cache/stacks/BigInsights"
> ```
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 60756: AMBARI-21440. Cross-stack migration, package supplemental stack in RPM

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60756/
---

Review request for Ambari, Attila Doroszlai, Dmytro Grinenko, Jonathan Hurley, 
and Sid Wagle.


Bugs: AMBARI-21440
https://issues.apache.org/jira/browse/AMBARI-21440


Repository: ambari


Description
---

When trying to upgrade ambari-server, it fails with error: 
```
Exception in thread "main" org.apache.ambari.server.AmbariException: Stack 
data, Stack BigInsights 4.2.5 is not found in Ambari metainfo
```

For the server, package /var/lib/ambari-server/resources/stacks/BigInsights
For the agent, package /var/lib/ambari-agent/cache/stacks/BigInsights


Diffs
-

  ambari-agent/pom.xml 475adf2 
  ambari-server/pom.xml 63e704c 
  ambari-server/src/main/assemblies/server.xml 74d96a4 
  dev-support/docker/docker/bin/ambaribuild.py dfb7d60 
  dev-support/docker/docker/bin/test/ambaribuild_test.py a324f5f 
  pom.xml 8a17956 


Diff: https://reviews.apache.org/r/60756/diff/1/


Testing
---

Verified the new RPMs,

```
# Generate the RPM
mvn -B install package rpm:rpm -pl 
utility,ambari-views,ambari-metrics/ambari-metrics-common,ambari-server,ambari-agent
 -Dmaven.clover.skip=true -Dfindbugs.skip=true -DskipTests=true 
-Dstack.distribution=HDP -Dsupplemental.distribution=BigInsights 
-Dpython.ver="python >= 2.6" -Drat.ignoreErrors=true

# Verify the contents
rpm -qlp rpm -qlp ambari-server-2.5.1.0-0.x86_64.rpm | grep 
"var/lib/ambari-server/resources/stacks/BigInsights"

rpm -qlp rpm -qlp ambari-agent-2.5.1.0-0.x86_64.rpm | grep 
"/var/lib/ambari-agent/cache/stacks/BigInsights"
```


Thanks,

Alejandro Fernandez

Re: Review Request 60249: AMBARI-21234 Ambari rack awareness for Kafka

[Ambud Sharma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60249/
---

(Updated July 10, 2017, 11:36 p.m.)


Review request for Ambari, Alejandro Fernandez and Sriharsha Chintalapani.


Repository: ambari


Description
---

Ambari rack awareness for Kafka. 
https://issues.apache.org/jira/browse/AMBARI-21234

As an operations person it would be nice to manage Kafka rack awareness via 
Ambari. Ambari allows node rack information to be configured and this 
information can then be pulled in the Kafka stack and populated in the 
server.properties file for Kafka.
Design:
This stack change uses the /clusterHostInfo/all_hosts and 
/clusterHostInfo/all_racks paths and materializes them to a variable. Then it 
uses linear search to find this node in the list of all hosts and it's 
corresponding rack id. This information is then stored in a variable called 
rack and which is materialized during the configure method of the broker 
scripts.
This stack change relies on the node rack information stored in Ambari 
therefore will enable both Ambari UI and Blueprints to be used for setting up 
Kafka broker rack information.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/KAFKA/0.10.0/configuration/ranger-kafka-audit.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 1327090aa7 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
 c36a10ff28 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/KAFKA/metainfo.xml 
12f6c45396 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/KAFKA/metainfo.xml 
fb281361a1 


Diff: https://reviews.apache.org/r/60249/diff/6/

Changes: https://reviews.apache.org/r/60249/diff/5-6/


Testing
---

Manually deployed Kafka cluster and verified the broker.rack property is 
correctly populated.


Thanks,

Ambud Sharma

Re: Review Request 60198: Handle scenario when host in-memory aggregation is not working

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60198/#review180140
---


Ship it!




Ship It!

- Aravindan Vijayan


On July 10, 2017, 4:17 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60198/
> ---
> 
> (Updated July 10, 2017, 4:17 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-21279
> https://issues.apache.org/jira/browse/AMBARI-21279
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If monitor aggregation is enabled we do not do collector side 5 minutes 
> aggregation. If a monitor is down we do not get metrics from that host. 
> Metrics collector should check that metrics from a host are available per 
> minute but aggregates are not.
> This could be based on liveliness check of a monitor. More like a heartbeat 
> ping. When we loose ping for > 1 min < 5 min we can aggregate metrics for 
> that time period and hostId.
> Sinks and monitor should switch back to collector reporting state if local 
> aggregation app is not available.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricsService.java
>  66c46dbea3 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
>  081e610028 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricFilteringHostAggregator.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricFilteringHostAggregator.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
>  e00c04537a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/Condition.java
>  9714e1ab9b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java
>  3c03dcab08 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/EmptyCondition.java
>  b667df3287 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  51c96c682f 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/SplitByMetricNamesCondition.java
>  45ea74c9bc 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java
>  dd73a8aecf 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITClusterAggregator.java
>  e66e65db1a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultConditionTest.java
>  e4e922554b 
> 
> 
> Diff: https://reviews.apache.org/r/60198/diff/2/
> 
> 
> Testing
> ---
> 
> In progress
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 60754: AMBARI-21439 Hive/MapRed2 restart fails after Ambari upgrade

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60754/#review180115
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 10, 2017, 9:40 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60754/
> ---
> 
> (Updated July 10, 2017, 9:40 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-21439
> https://issues.apache.org/jira/browse/AMBARI-21439
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The BI stacks copied to HDP uses old copy_to_hdfs Python API from 
> ambari-common that uses the now-renamed "host_system_prepped", It's renamed 
> to "skip".
> This issue only happens on BI 4.2 GA clusters. References to the copy_to_hdfs 
> API need to use the new parameter - skip
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/YARN/package/scripts/historyserver.py
>  bb384e4 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/package/scripts/hive.py
>  f6fddd0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/package/scripts/hive_server.py
>  9c336eb 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SLIDER/package/scripts/service_check.py
>  d56f612 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/YARN/package/scripts/historyserver.py
>  40ca374 
> 
> 
> Diff: https://reviews.apache.org/r/60754/diff/1/
> 
> 
> Testing
> ---
> 
> patched a cluster with the changes, restart Hive, Yarn, run service check for 
> Slider
> 
> 
> Thanks,
> 
> Di Li
> 
>

Review Request 60754: AMBARI-21439 Hive/MapRed2 restart fails after Ambari upgrade

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60754/
---

Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe.


Bugs: AMBARI-21439
https://issues.apache.org/jira/browse/AMBARI-21439


Repository: ambari


Description
---

The BI stacks copied to HDP uses old copy_to_hdfs Python API from ambari-common 
that uses the now-renamed "host_system_prepped", It's renamed to "skip".
This issue only happens on BI 4.2 GA clusters. References to the copy_to_hdfs 
API need to use the new parameter - skip


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/YARN/package/scripts/historyserver.py
 bb384e4 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/package/scripts/hive.py
 f6fddd0 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/package/scripts/hive_server.py
 9c336eb 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SLIDER/package/scripts/service_check.py
 d56f612 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/YARN/package/scripts/historyserver.py
 40ca374 


Diff: https://reviews.apache.org/r/60754/diff/1/


Testing
---

patched a cluster with the changes, restart Hive, Yarn, run service check for 
Slider


Thanks,

Di Li

Re: Review Request 60707: BE: Extend Ambari REST API to Support User Account Management Improvements

[Robert Levas]

> On July 10, 2017, 2:57 a.m., Sebastian Toader wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java
> > Lines 367-369 (patched)
> > 
> >
> > Shouldn't be an error message returned here in case no pwd passed in 
> > for local users?

The password is optional.  If set, the local authentication source will be 
added (for backward compatibilty).  If not set, there is no issue becuase the 
user did not want to create a local authentication source for the new resource.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60707/#review180009
---


On July 7, 2017, 9:30 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60707/
> ---
> 
> (Updated July 7, 2017, 9:30 a.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-20861
> https://issues.apache.org/jira/browse/AMBARI-20861
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update the Ambari REST API to allow for GET, POST, PUT, and DELETE operations 
> on the authentication sources related to an Ambari user account.
> 
> Add the following entry points:
> 
> - `/api/v1/users/:USER_ID`
> -- Get a user
> -- Modify a user
> 
> - `/api/v1/users/:USERNAME/sources`
> -- List a user’s authentication sources
> -- Add a new authentication source for a user
> 
> - `/api/v1/users/:USERNAME/sources/:SOURCE_ID`
> -- Get details on a specific authentication source for a user
> -- Modify details for a specific authentication source for a user
> 
> - `/api/v1/users/:USER_ID/sources`
> -- List a user’s authentication sources
> -- Add a new authentication source for a user
> 
> - `/api/v1/users/:USER_ID/sources/:SOURCE_ID`
> -- Get details on a specific authentication source for a user
> -- Modify details for a specific authentication source for a user
> 
> Update the following entry points, ensuring backwards compatibility where 
> possible:
> 
> - `/api/v1/users`
> -- List all users
> -- Add a new user
> -- Backward compatibility: Set password should create or update the 
> appropriate user_authentication record.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/api/v1/authentication-source-create.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-delete.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-get.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-list.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-resources.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-update.md PRE-CREATION 
>   ambari-server/docs/api/v1/index.md da05d75bc3 
>   ambari-server/docs/api/v1/user-create.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-delete.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-get.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-list.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-resources.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-update.md PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  96e288fee0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/UserResourceDefinition.java
>  b228c82aa9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/users/UserAuthenticationSourceService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  25d12c7dd6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java
>  2454bf7694 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/UserAuthenticationSourceRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/UserAuthenticationSourceResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/UserResponse.java
>  bcb3aaf3e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
>  595b7f996f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserAuthenticationSourceResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java
>  45b733b042 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
>  362b4e631c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/o

Re: Review Request 60691: AMBARI-21420 HBase master crashed during/post EU on an IOP4.2.5/HDP2.6 migrated cluster

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60691/
---

(Updated July 10, 2017, 7:55 p.m.)


Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe.


Changes
---

backup the old wal dir instead of removing it.


Bugs: AMBARI-21420
https://issues.apache.org/jira/browse/AMBARI-21420


Repository: ambari


Description
---

HBase in IOP 4.2.5 is of higher version than the one shipped in HDP 2.6. HBase 
master appeared as if it had started according to the Ambari action log but 
later crashed with error:
2017-07-05 09:28:08,952 FATAL [hats1:16000.activeMasterManager] master.HMaster: 
Failed to become active masterjava.io.IOException: The procedure class 
org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure must be 
accessible and have an empty constructorat 
org.apache.hadoop.hbase.procedure2.Procedure.newInstance(Procedure.java:663)at 
org.apache.hadoop.hbase.procedure2.Procedure.convert(Procedure.java:789)at 
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.read(ProcedureWALFormatReader.java:111)at
 
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:81)at
 
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:374)at
 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:297)at
 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:464)at
 
org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1297)at
 org.apache.
 hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1196)
Attach one or more files to this issue
Attachments


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py
 a1003dc 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 43c6036 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 42999b2 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 e9c8541 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 f9e3e15 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 e9c8541 


Diff: https://reviews.apache.org/r/60691/diff/5/

Changes: https://reviews.apache.org/r/60691/diff/4-5/


Testing
---

will be tested in the integration tests.


Thanks,

Di Li

Re: Review Request 60745: Allow Services To Be Stopped During an EU Between Stack Vendors

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60745/
---

(Updated July 10, 2017, 2:44 p.m.)


Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Nate Cole, 
and Sid Wagle.


Changes
---

My prior changes where I removed HDP 2.2-specific code caused way too many 
tests to fail b/c they assumed old directories. I tried to fix them up, but it 
was too much for this release and should be done separately.


Bugs: AMBARI-21432
https://issues.apache.org/jira/browse/AMBARI-21432


Repository: ambari


Description
---

With stack features allowed to co-exist for cross stack upgrade, once 
AMBARI-21430 is completed we should be able to do Stop All services from the 
upgrade pack.

This will allow EU to look seamless and similar to any intra-stack upgrade with 
pre-checks and stop enabled.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 facf186f0e 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 46260058cf 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 46260058cf 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 
60022e10e4 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 
dfeb4be734 


Diff: https://reviews.apache.org/r/60745/diff/2/

Changes: https://reviews.apache.org/r/60745/diff/1-2/


Testing (updated)
---

Performed an EU with Core services from BI to HDP.

--
Total run:1161
Total errors:0
Total failures:0
OK


Thanks,

Jonathan Hurley

Re: Review Request 60741: AMBARI-21431. Update BigInsight configuration files to be compliant with XSD

[Sumit Mohanty]

> On July 10, 2017, 5:15 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
> > Lines 211 (patched)
> > 
> >
> > We should actually set all of these to on-ambari-upgrade add="false" to 
> > minimize the risk of new properties being added before the EU migration.
> 
> Sumit Mohanty wrote:
> +1. It will good to not have Ambari upgrade add any config or an absolute 
> minimum.
> 
> Jonathan Hurley wrote:
> I disagree - this should match what is in trunk as closely as possible 
> (which is true in many cases). The reason for this is that the Python code 
> could be assuming these properties exist and if they don't get added, then it 
> could cause problems during restarts.

Looks like the properties did not have any "ambari-upgrade" directive. What is 
the default behavior? Was it true?
Anyway, lets see how many properties get added post Ambari upgrade. Usually, 
extra properties do not hurt other than indicating restart and restart does not 
seem to be a problem here.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60741/#review180057
---


On July 10, 2017, 12:30 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60741/
> ---
> 
> (Updated July 10, 2017, 12:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21431
> https://issues.apache.org/jira/browse/AMBARI-21431
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhancements in `StackManager`:
>  * print line number of schema violation
>  * print canonical path if possible: eg. 
> `ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
>instead of 
> `ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
> 
> Added `` where missing:
>  * value of the attribute `add` matches that in `common-services` where 
> possible
>  * some properties were marked with 
> `DONT_ADD_ON_UPGRADE`, which is not 
> schema-valid, hence removed -- added `` for 
> these
>  * added with `add="false"` for a few properties in BI stack that are marked 
> `true`
>  * conversely, some properties marked as deleted in `common-services`, but 
> still present in BI stack are added with `add="true"`
> 
> Some properties had other schema problems:
>  * `` instead of ``
>  * duplicate tags
>  * wrong tag `` instead of ``
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> d9a3ac4f956b397661587f13fece80b74ddd6f32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
>  2d11ef38005262f38d59dd24b223c5b8c6c42530 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-env.xml
>  59102538b589ebf3bb792d0067ea7d29d6ef1672 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-env.xml
>  379297f8e3df2c462e7a5b9f65d683dd53a5ba9e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-log4j.xml
>  52ead29872f1f30837da9c68dba66d98bb96b3e1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-policy.xml
>  febbd44af15519b26ba0c35173207b5b51fa0fba 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-security-site.xml
>  5e7bc51809c809a56ac2c9592c611cf2a7c927e0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-site.xml
>  00d0a2c39a1834e908eb4766e7ad7587444280c2 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-log4j.xml
>  2b0a4cf20d94cdeca4320845f3d733e57f460dbe 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-site.xml
>  cc9c27a67d0213903391074f82c132fc8a50 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-conf.xml
>  582e181aeb42bbda0c330a86570e163f292fc3df 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-env.xml
>  a448a668d26c159cfff97618cb72fabbb532eac1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-env.xml
>  47a2bb65bc48b84c77ff97e2eabb26436899aa8a 
>   
> ambari-serv

Re: Review Request 60745: Allow Services To Be Stopped During an EU Between Stack Vendors

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60745/#review180049
---


Fix it, then Ship it!




Some trivial issues


ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
Line 472 (original), 441 (patched)


"targetted" -> "targeted"



ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
Line 47 (original), 45 (patched)


trailing whitespace



ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
Lines 124 (patched)


trailing whitespace


- Attila Doroszlai


On July 10, 2017, 4:31 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60745/
> ---
> 
> (Updated July 10, 2017, 4:31 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-21432
> https://issues.apache.org/jira/browse/AMBARI-21432
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With stack features allowed to co-exist for cross stack upgrade, once 
> AMBARI-21430 is completed we should be able to do Stop All services from the 
> upgrade pack.
> 
> This will allow EU to look seamless and similar to any intra-stack upgrade 
> with pre-checks and stop enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  facf186f0e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
> 
> 
> Diff: https://reviews.apache.org/r/60745/diff/1/
> 
> 
> Testing
> ---
> 
> Performed an EU with Core services from BI to HDP.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60691: AMBARI-21420 HBase master crashed during/post EU on an IOP4.2.5/HDP2.6 migrated cluster

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60691/#review180063
---




ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
Lines 445 (patched)


Lines 445 and 446 are no longer needed now.


- Alejandro Fernandez


On July 10, 2017, 3:54 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60691/
> ---
> 
> (Updated July 10, 2017, 3:54 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-21420
> https://issues.apache.org/jira/browse/AMBARI-21420
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase in IOP 4.2.5 is of higher version than the one shipped in HDP 2.6. 
> HBase master appeared as if it had started according to the Ambari action log 
> but later crashed with error:
> 2017-07-05 09:28:08,952 FATAL [hats1:16000.activeMasterManager] 
> master.HMaster: Failed to become active masterjava.io.IOException: The 
> procedure class org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure 
> must be accessible and have an empty constructorat 
> org.apache.hadoop.hbase.procedure2.Procedure.newInstance(Procedure.java:663)at
>  org.apache.hadoop.hbase.procedure2.Procedure.convert(Procedure.java:789)at 
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.read(ProcedureWALFormatReader.java:111)at
>  
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:81)at
>  
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:374)at
>  
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:297)at
>  
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:464)at
>  
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1297)at
>  org.apach
 e.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1196)
> Attach one or more files to this issue
> Attachments
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py
>  a1003dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  43c6036 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  bff6672 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  e9c8541 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  d931093 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  e9c8541 
> 
> 
> Diff: https://reviews.apache.org/r/60691/diff/4/
> 
> 
> Testing
> ---
> 
> will be tested in the integration tests.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Review Request 60751: Log Search UI: implement sorting of logs list

[Andrii Babiichuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60751/
---

Review request for Ambari and Aleksandr Kovalenko.


Bugs: AMBARI-21438
https://issues.apache.org/jira/browse/AMBARI-21438


Repository: ambari


Description
---

User should be able to sort logs list content by component name/level/time etc.


Diffs
-

  ambari-logsearch/ambari-logsearch-web-new/src/app/app.module.ts 72b5ad1 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.html
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.less
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.spec.ts
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.ts
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-list/dropdown-list.component.spec.ts
 445ee97 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-list/dropdown-list.component.ts
 6426f6a 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-button/filter-button.component.spec.ts
 dda81e6 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-button/filter-button.component.ts
 b529686 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.html
 bb7a206 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.less
 ec8e84b 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.spec.ts
 f6d1294 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.ts
 1cf7d41 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.html
 8fb7659 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.spec.ts
 9123969 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.ts
 b05000b 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.html
 450ce5a 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.less
 46a157c 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.spec.ts
 1917f7f 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.ts
 f42a2bb 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.html
 09b2e1e 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.less
 6ed0463 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.spec.ts
 ffcfb83 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.ts
 bf80445 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.html
 597 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.ts
 0c1f840 
  ambari-logsearch/ambari-logsearch-web-new/src/app/components/variables.less 
9a8ea09 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/filtering.service.spec.ts
 0ad3795 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/filtering.service.ts 
36bb756 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/mock-api-data.service.ts
 40645f0 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/utils.service.spec.ts
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-web-new/src/app/services/utils.service.ts 
PRE-CREATION 
  ambari-logsearch/ambari-logsearch-web-new/src/assets/i18n/en.json 755f307 


Diff: https://reviews.apache.org/r/60751/diff/1/


Testing
---

Executed 47 of 47 SUCCESS (1.178 secs / 1.147 secs)


Thanks,

Andrii Babiichuk

Re: Review Request 60741: AMBARI-21431. Update BigInsight configuration files to be compliant with XSD

[Jonathan Hurley]

> On July 10, 2017, 1:15 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
> > Lines 211 (patched)
> > 
> >
> > We should actually set all of these to on-ambari-upgrade add="false" to 
> > minimize the risk of new properties being added before the EU migration.
> 
> Sumit Mohanty wrote:
> +1. It will good to not have Ambari upgrade add any config or an absolute 
> minimum.

I disagree - this should match what is in trunk as closely as possible (which 
is true in many cases). The reason for this is that the Python code could be 
assuming these properties exist and if they don't get added, then it could 
cause problems during restarts.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60741/#review180057
---


On July 10, 2017, 8:30 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60741/
> ---
> 
> (Updated July 10, 2017, 8:30 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21431
> https://issues.apache.org/jira/browse/AMBARI-21431
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhancements in `StackManager`:
>  * print line number of schema violation
>  * print canonical path if possible: eg. 
> `ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
>instead of 
> `ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
> 
> Added `` where missing:
>  * value of the attribute `add` matches that in `common-services` where 
> possible
>  * some properties were marked with 
> `DONT_ADD_ON_UPGRADE`, which is not 
> schema-valid, hence removed -- added `` for 
> these
>  * added with `add="false"` for a few properties in BI stack that are marked 
> `true`
>  * conversely, some properties marked as deleted in `common-services`, but 
> still present in BI stack are added with `add="true"`
> 
> Some properties had other schema problems:
>  * `` instead of ``
>  * duplicate tags
>  * wrong tag `` instead of ``
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> d9a3ac4f956b397661587f13fece80b74ddd6f32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
>  2d11ef38005262f38d59dd24b223c5b8c6c42530 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-env.xml
>  59102538b589ebf3bb792d0067ea7d29d6ef1672 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-env.xml
>  379297f8e3df2c462e7a5b9f65d683dd53a5ba9e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-log4j.xml
>  52ead29872f1f30837da9c68dba66d98bb96b3e1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-policy.xml
>  febbd44af15519b26ba0c35173207b5b51fa0fba 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-security-site.xml
>  5e7bc51809c809a56ac2c9592c611cf2a7c927e0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-site.xml
>  00d0a2c39a1834e908eb4766e7ad7587444280c2 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-log4j.xml
>  2b0a4cf20d94cdeca4320845f3d733e57f460dbe 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-site.xml
>  cc9c27a67d0213903391074f82c132fc8a50 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-conf.xml
>  582e181aeb42bbda0c330a86570e163f292fc3df 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-env.xml
>  a448a668d26c159cfff97618cb72fabbb532eac1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-env.xml
>  47a2bb65bc48b84c77ff97e2eabb26436899aa8a 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-javaopts-properties.xml
>  3eca77a1171499bff9fe930862d01a9b266296da 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-log4j.xml
>  2e8fb3d22c95159323a1e3f691a3ef68cd3bc8b0 
>   
> ambari-server/src/main/resource

Re: Review Request 60751: Log Search UI: implement sorting of logs list

[Aleksandr Kovalenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60751/#review180070
---


Ship it!




Ship It!

- Aleksandr Kovalenko


On Июль 10, 2017, 5:57 п.п., Andrii Babiichuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60751/
> ---
> 
> (Updated Июль 10, 2017, 5:57 п.п.)
> 
> 
> Review request for Ambari and Aleksandr Kovalenko.
> 
> 
> Bugs: AMBARI-21438
> https://issues.apache.org/jira/browse/AMBARI-21438
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> User should be able to sort logs list content by component name/level/time 
> etc.
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-web-new/src/app/app.module.ts 72b5ad1 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.html
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.less
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.spec.ts
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.ts
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-list/dropdown-list.component.spec.ts
>  445ee97 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-list/dropdown-list.component.ts
>  6426f6a 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-button/filter-button.component.spec.ts
>  dda81e6 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-button/filter-button.component.ts
>  b529686 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.html
>  bb7a206 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.less
>  ec8e84b 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.spec.ts
>  f6d1294 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.ts
>  1cf7d41 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.html
>  8fb7659 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.spec.ts
>  9123969 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.ts
>  b05000b 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.html
>  450ce5a 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.less
>  46a157c 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.spec.ts
>  1917f7f 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.ts
>  f42a2bb 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.html
>  09b2e1e 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.less
>  6ed0463 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.spec.ts
>  ffcfb83 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.ts
>  bf80445 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.html
>  597 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.ts
>  0c1f840 
>   ambari-logsearch/ambari-logsearch-web-new/src/app/components/variables.less 
> 9a8ea09 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/filtering.service.spec.ts
>  0ad3795 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/filtering.service.ts
>  36bb756 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/mock-api-data.service.ts
>  40645f0 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/utils.service.spec.ts
>  PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-web-new/src/app/services/utils.service.ts 
> PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-web-new/src/assets/i18n/en.json 755f307 
> 
> 
> Diff: https://reviews.apache.org/r/60751/diff/1/
> 
> 
> Testing
> ---
> 
> Executed 47 of 47 SUCCESS (1.178 secs / 1.147 secs)
> 
> 
> Thanks,
> 
> Andrii Babiichuk
> 
>

Re: Review Request 60745: Allow Services To Be Stopped During an EU Between Stack Vendors

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60745/#review180069
---


Ship it!




Ship It!

- Nate Cole


On July 10, 2017, 10:31 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60745/
> ---
> 
> (Updated July 10, 2017, 10:31 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-21432
> https://issues.apache.org/jira/browse/AMBARI-21432
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With stack features allowed to co-exist for cross stack upgrade, once 
> AMBARI-21430 is completed we should be able to do Stop All services from the 
> upgrade pack.
> 
> This will allow EU to look seamless and similar to any intra-stack upgrade 
> with pre-checks and stop enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  facf186f0e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
> 
> 
> Diff: https://reviews.apache.org/r/60745/diff/1/
> 
> 
> Testing
> ---
> 
> Performed an EU with Core services from BI to HDP.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60744: Add python functions to get the Ambari version of the agent

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/#review180062
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 10, 2017, 2:37 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60744/
> ---
> 
> (Updated July 10, 2017, 2:37 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, 
> and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21435
> https://issues.apache.org/jira/browse/AMBARI-21435
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the mpacks we may want to know the version of Ambari, for example the Log 
> Search handles it's configuration in different ways depending on it's version
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60744/diff/3/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 60738: HdfsResource can not handle S3 URL when hbase.rootdir is set to S3 URL

[Alejandro Fernandez]

> On July 10, 2017, 5:08 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
> > Lines 197 (patched)
> > 
> >
> > This is already needed in HBase for 3.0
> > Please add a comment as well explaining why this was needed.

Typo, "This is also needed in HBase for 3.0"


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60738/#review180055
---


On July 10, 2017, 9:47 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60738/
> ---
> 
> (Updated July 10, 2017, 9:47 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20950
> https://issues.apache.org/jira/browse/AMBARI-20950
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBASE-17437 enables HBASE to put its rootdir on S3. Ambari calls HdfsResource
> to create the dir for HBase, but it can not handle S3.  
> Fix should be similar to AMBARI-15157 for Hive.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
>  68856ce 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  43c6036 
> 
> 
> Diff: https://reviews.apache.org/r/60738/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 60745: Allow Services To Be Stopped During an EU Between Stack Vendors

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60745/#review180060
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 10, 2017, 2:31 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60745/
> ---
> 
> (Updated July 10, 2017, 2:31 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-21432
> https://issues.apache.org/jira/browse/AMBARI-21432
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With stack features allowed to co-exist for cross stack upgrade, once 
> AMBARI-21430 is completed we should be able to do Stop All services from the 
> upgrade pack.
> 
> This will allow EU to look seamless and similar to any intra-stack upgrade 
> with pre-checks and stop enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  facf186f0e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
> 
> 
> Diff: https://reviews.apache.org/r/60745/diff/1/
> 
> 
> Testing
> ---
> 
> Performed an EU with Core services from BI to HDP.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60741: AMBARI-21431. Update BigInsight configuration files to be compliant with XSD

[Sumit Mohanty]

> On July 10, 2017, 5:15 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
> > Lines 211 (patched)
> > 
> >
> > We should actually set all of these to on-ambari-upgrade add="false" to 
> > minimize the risk of new properties being added before the EU migration.

+1. It will good to not have Ambari upgrade add any config or an absolute 
minimum.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60741/#review180057
---


On July 10, 2017, 12:30 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60741/
> ---
> 
> (Updated July 10, 2017, 12:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21431
> https://issues.apache.org/jira/browse/AMBARI-21431
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhancements in `StackManager`:
>  * print line number of schema violation
>  * print canonical path if possible: eg. 
> `ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
>instead of 
> `ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
> 
> Added `` where missing:
>  * value of the attribute `add` matches that in `common-services` where 
> possible
>  * some properties were marked with 
> `DONT_ADD_ON_UPGRADE`, which is not 
> schema-valid, hence removed -- added `` for 
> these
>  * added with `add="false"` for a few properties in BI stack that are marked 
> `true`
>  * conversely, some properties marked as deleted in `common-services`, but 
> still present in BI stack are added with `add="true"`
> 
> Some properties had other schema problems:
>  * `` instead of ``
>  * duplicate tags
>  * wrong tag `` instead of ``
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> d9a3ac4f956b397661587f13fece80b74ddd6f32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
>  2d11ef38005262f38d59dd24b223c5b8c6c42530 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-env.xml
>  59102538b589ebf3bb792d0067ea7d29d6ef1672 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-env.xml
>  379297f8e3df2c462e7a5b9f65d683dd53a5ba9e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-log4j.xml
>  52ead29872f1f30837da9c68dba66d98bb96b3e1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-policy.xml
>  febbd44af15519b26ba0c35173207b5b51fa0fba 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-security-site.xml
>  5e7bc51809c809a56ac2c9592c611cf2a7c927e0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-site.xml
>  00d0a2c39a1834e908eb4766e7ad7587444280c2 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-log4j.xml
>  2b0a4cf20d94cdeca4320845f3d733e57f460dbe 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-site.xml
>  cc9c27a67d0213903391074f82c132fc8a50 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-conf.xml
>  582e181aeb42bbda0c330a86570e163f292fc3df 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-env.xml
>  a448a668d26c159cfff97618cb72fabbb532eac1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-env.xml
>  47a2bb65bc48b84c77ff97e2eabb26436899aa8a 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-javaopts-properties.xml
>  3eca77a1171499bff9fe930862d01a9b266296da 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-log4j.xml
>  2e8fb3d22c95159323a1e3f691a3ef68cd3bc8b0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-policy.xml
>  b0807b6cf2a008a053992eec103fcb4b7f7a0d67 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-site.xml
>  d4cca6c7bd1c127d92e00fff10622a720405197b 
>   
> ambari-server/src/main/re

Re: Review Request 60741: AMBARI-21431. Update BigInsight configuration files to be compliant with XSD

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60741/#review180057
---




ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
Lines 211 (patched)


We should actually set all of these to on-ambari-upgrade add="false" to 
minimize the risk of new properties being added before the EU migration.


- Alejandro Fernandez


On July 10, 2017, 12:30 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60741/
> ---
> 
> (Updated July 10, 2017, 12:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21431
> https://issues.apache.org/jira/browse/AMBARI-21431
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhancements in `StackManager`:
>  * print line number of schema violation
>  * print canonical path if possible: eg. 
> `ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
>instead of 
> `ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
> 
> Added `` where missing:
>  * value of the attribute `add` matches that in `common-services` where 
> possible
>  * some properties were marked with 
> `DONT_ADD_ON_UPGRADE`, which is not 
> schema-valid, hence removed -- added `` for 
> these
>  * added with `add="false"` for a few properties in BI stack that are marked 
> `true`
>  * conversely, some properties marked as deleted in `common-services`, but 
> still present in BI stack are added with `add="true"`
> 
> Some properties had other schema problems:
>  * `` instead of ``
>  * duplicate tags
>  * wrong tag `` instead of ``
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> d9a3ac4f956b397661587f13fece80b74ddd6f32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
>  2d11ef38005262f38d59dd24b223c5b8c6c42530 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-env.xml
>  59102538b589ebf3bb792d0067ea7d29d6ef1672 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-env.xml
>  379297f8e3df2c462e7a5b9f65d683dd53a5ba9e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-log4j.xml
>  52ead29872f1f30837da9c68dba66d98bb96b3e1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-policy.xml
>  febbd44af15519b26ba0c35173207b5b51fa0fba 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-security-site.xml
>  5e7bc51809c809a56ac2c9592c611cf2a7c927e0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-site.xml
>  00d0a2c39a1834e908eb4766e7ad7587444280c2 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-log4j.xml
>  2b0a4cf20d94cdeca4320845f3d733e57f460dbe 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-site.xml
>  cc9c27a67d0213903391074f82c132fc8a50 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-conf.xml
>  582e181aeb42bbda0c330a86570e163f292fc3df 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-env.xml
>  a448a668d26c159cfff97618cb72fabbb532eac1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-env.xml
>  47a2bb65bc48b84c77ff97e2eabb26436899aa8a 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-javaopts-properties.xml
>  3eca77a1171499bff9fe930862d01a9b266296da 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-log4j.xml
>  2e8fb3d22c95159323a1e3f691a3ef68cd3bc8b0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-policy.xml
>  b0807b6cf2a008a053992eec103fcb4b7f7a0d67 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-site.xml
>  d4cca6c7bd1c127d92e00fff10622a720405197b 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/core-site.xml
>  f833896330028f2e2854d166b9fb107cb56a4cfe 
>   
> ambari-server/src/main/resources/stacks/BigInsights

Re: Review Request 60715: AMBARI-21421 EU upgrade pack to update Hive and Yarn properties to remove IOP/BigSQL properties

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60715/#review180054
---




ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
Lines 38 (patched)


Should be `https://reviews.apache.org/r/60715/#comment255058>

Same as above.



ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
Lines 60 (patched)


Missing close tag, ie. change to ``


- Attila Doroszlai


On July 7, 2017, 9:16 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60715/
> ---
> 
> (Updated July 7, 2017, 9:16 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-21421
> https://issues.apache.org/jira/browse/AMBARI-21421
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> IOP Yarn has property "" that requires Spark Shuffle Jar. HDP does not have 
> this jar. The property needs to be removed during EU.
> BigSQL also inserts a property to hive-site that causes Hive to crash 
> during/post EU due to the unique BigSQL upgrade/migrate process (that 
> requires BigSQL to be removed before EU). This property needs to be removed 
> as well.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  6597055 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  af87794 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  6597055 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  af87794 
> 
> 
> Diff: https://reviews.apache.org/r/60715/diff/1/
> 
> 
> Testing
> ---
> 
> integrated tests during EU
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 60738: HdfsResource can not handle S3 URL when hbase.rootdir is set to S3 URL

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60738/#review180055
---




ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
Lines 197 (patched)


This is already needed in HBase for 3.0
Please add a comment as well explaining why this was needed.


- Alejandro Fernandez


On July 10, 2017, 9:47 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60738/
> ---
> 
> (Updated July 10, 2017, 9:47 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20950
> https://issues.apache.org/jira/browse/AMBARI-20950
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBASE-17437 enables HBASE to put its rootdir on S3. Ambari calls HdfsResource
> to create the dir for HBase, but it can not handle S3.  
> Fix should be similar to AMBARI-15157 for Hive.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
>  68856ce 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  43c6036 
> 
> 
> Diff: https://reviews.apache.org/r/60738/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 60198: Handle scenario when host in-memory aggregation is not working

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60198/
---

(Updated Июль 10, 2017, 4:17 п.п.)


Review request for Ambari, Aravindan Vijayan and Sid Wagle.


Bugs: AMBARI-21279
https://issues.apache.org/jira/browse/AMBARI-21279


Repository: ambari


Description
---

If monitor aggregation is enabled we do not do collector side 5 minutes 
aggregation. If a monitor is down we do not get metrics from that host. Metrics 
collector should check that metrics from a host are available per minute but 
aggregates are not.
This could be based on liveliness check of a monitor. More like a heartbeat 
ping. When we loose ping for > 1 min < 5 min we can aggregate metrics for that 
time period and hostId.
Sinks and monitor should switch back to collector reporting state if local 
aggregation app is not available.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricsService.java
 66c46dbea3 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
 081e610028 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricFilteringHostAggregator.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricFilteringHostAggregator.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
 e00c04537a 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/Condition.java
 9714e1ab9b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java
 3c03dcab08 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/EmptyCondition.java
 b667df3287 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
 51c96c682f 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/SplitByMetricNamesCondition.java
 45ea74c9bc 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java
 dd73a8aecf 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITClusterAggregator.java
 e66e65db1a 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultConditionTest.java
 e4e922554b 


Diff: https://reviews.apache.org/r/60198/diff/2/

Changes: https://reviews.apache.org/r/60198/diff/1-2/


Testing
---

In progress


Thanks,

Dmytro Sen

Re: Review Request 60691: AMBARI-21420 HBase master crashed during/post EU on an IOP4.2.5/HDP2.6 migrated cluster

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60691/
---

(Updated July 10, 2017, 3:54 p.m.)


Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe.


Bugs: AMBARI-21420
https://issues.apache.org/jira/browse/AMBARI-21420


Repository: ambari


Description
---

HBase in IOP 4.2.5 is of higher version than the one shipped in HDP 2.6. HBase 
master appeared as if it had started according to the Ambari action log but 
later crashed with error:
2017-07-05 09:28:08,952 FATAL [hats1:16000.activeMasterManager] master.HMaster: 
Failed to become active masterjava.io.IOException: The procedure class 
org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure must be 
accessible and have an empty constructorat 
org.apache.hadoop.hbase.procedure2.Procedure.newInstance(Procedure.java:663)at 
org.apache.hadoop.hbase.procedure2.Procedure.convert(Procedure.java:789)at 
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.read(ProcedureWALFormatReader.java:111)at
 
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:81)at
 
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:374)at
 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:297)at
 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:464)at
 
org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1297)at
 org.apache.
 hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1196)
Attach one or more files to this issue
Attachments


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py
 a1003dc 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 43c6036 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 bff6672 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 e9c8541 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 d931093 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 e9c8541 


Diff: https://reviews.apache.org/r/60691/diff/4/

Changes: https://reviews.apache.org/r/60691/diff/3-4/


Testing
---

will be tested in the integration tests.


Thanks,

Di Li

Re: Review Request 60745: Allow Services To Be Stopped During an EU Between Stack Vendors

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60745/#review180040
---


Fix it, then Ship it!





ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
Lines 416 (patched)


Spelling.


- Sid Wagle


On July 10, 2017, 2:31 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60745/
> ---
> 
> (Updated July 10, 2017, 2:31 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-21432
> https://issues.apache.org/jira/browse/AMBARI-21432
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With stack features allowed to co-exist for cross stack upgrade, once 
> AMBARI-21430 is completed we should be able to do Stop All services from the 
> upgrade pack.
> 
> This will allow EU to look seamless and similar to any intra-stack upgrade 
> with pre-checks and stop enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  facf186f0e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
> 
> 
> Diff: https://reviews.apache.org/r/60745/diff/1/
> 
> 
> Testing
> ---
> 
> Performed an EU with Core services from BI to HDP.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60744: Add python functions to get the Ambari version of the agent

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/#review180041
---


Ship it!




Ship It!

- Robert Nettleton


On July 10, 2017, 2:37 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60744/
> ---
> 
> (Updated July 10, 2017, 2:37 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, 
> and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21435
> https://issues.apache.org/jira/browse/AMBARI-21435
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the mpacks we may want to know the version of Ambari, for example the Log 
> Search handles it's configuration in different ways depending on it's version
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60744/diff/3/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 60745: Allow BI stack services to be stopped as a part of the upgrade pack

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60745/#review180037
---




ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
Lines 406-445 (original), 400-417 (patched)


It turns out that when adding STOP commands back, we encountered failures 
all over the place. The problem is that the conf-select tool installed by HDP 
doesn't take a stack root - it hard codes /usr/hdp (just like the one for BI 
hard coded /usr/iop)

When sending down any command, the status_params.py consult this function 
to get the hadoop directory. The logic is like this:
- If there is not an upgrade in progress, then just return 
/usr/hdp/current/hadoop-client/conf

- If there is an upgrade in progress, then use the hadoop conf directory 
which matches this component's version

*The odd part here is that we were also doing a conf_select.select() which 
failed b/c the stack component was still using a BigInsight number in its patch 
during the STOP command*

This seems to have been a relic from the ancient HDP 2.2 days where 
conf-select didn't exist yet and we had to cover the initial upgrade/downgrade 
situation between 2.2 and 2.3 when conf-select was introduced.

I decided to simply get rid of calling conf-select.select() during the 
upgrade and I left the exist code which says to invoke it if 
/usr/hdp/current/hadoop-client/conf is not a link. Technically this can never 
happen anymore, but I'd rather not change too many tangential areas of the 
upgrade code.


- Jonathan Hurley


On July 10, 2017, 10:31 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60745/
> ---
> 
> (Updated July 10, 2017, 10:31 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-21432
> https://issues.apache.org/jira/browse/AMBARI-21432
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With stack features allowed to co-exist for cross stack upgrade, once 
> AMBARI-21430 is completed we should be able to do Stop All services from the 
> upgrade pack.
> 
> This will allow EU to look seamless and similar to any intra-stack upgrade 
> with pre-checks and stop enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  facf186f0e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  46260058cf 
> 
> 
> Diff: https://reviews.apache.org/r/60745/diff/1/
> 
> 
> Testing
> ---
> 
> Performed an EU with Core services from BI to HDP.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60744: Add python functions to get the Ambari version of the agent

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/
---

(Updated July 10, 2017, 2:37 p.m.)


Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, and 
Robert Nettleton.


Changes
---

fix summary


Summary (updated)
-

Add python functions to get the Ambari version of the agent


Bugs: AMBARI-21435
https://issues.apache.org/jira/browse/AMBARI-21435


Repository: ambari


Description
---

In the mpacks we may want to know the version of Ambari, for example the Log 
Search handles it's configuration in different ways depending on it's version


Diffs
-

  ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
PRE-CREATION 


Diff: https://reviews.apache.org/r/60744/diff/3/


Testing
---

Tested on local cluster


Thanks,

Miklos Gergely

Review Request 60745: Allow BI stack services to be stopped as a part of the upgrade pack

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60745/
---

Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Nate Cole, 
and Sid Wagle.


Bugs: AMBARI-21432
https://issues.apache.org/jira/browse/AMBARI-21432


Repository: ambari


Description
---

With stack features allowed to co-exist for cross stack upgrade, once 
AMBARI-21430 is completed we should be able to do Stop All services from the 
upgrade pack.

This will allow EU to look seamless and similar to any intra-stack upgrade with 
pre-checks and stop enabled.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 facf186f0e 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 46260058cf 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 46260058cf 


Diff: https://reviews.apache.org/r/60745/diff/1/


Testing
---

Performed an EU with Core services from BI to HDP.


Thanks,

Jonathan Hurley

Re: Review Request 60744: Add python functions to get the Ambari version of the server / agent

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/
---

(Updated July 10, 2017, 2:27 p.m.)


Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, and 
Robert Nettleton.


Changes
---

added documentation

removed get_ambari_version_server, it can be obtained like this:

>>> from ambari_server.serverConfiguration import get_ambari_properties, 
>>> get_ambari_version
>>> p = get_ambari_properties()
>>> get_ambari_version(p)


Bugs: AMBARI-21435
https://issues.apache.org/jira/browse/AMBARI-21435


Repository: ambari


Description
---

In the mpacks we may want to know the version of Ambari, for example the Log 
Search handles it's configuration in different ways depending on it's version


Diffs (updated)
-

  ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
PRE-CREATION 


Diff: https://reviews.apache.org/r/60744/diff/3/

Changes: https://reviews.apache.org/r/60744/diff/2-3/


Testing
---

Tested on local cluster


Thanks,

Miklos Gergely

Re: Review Request 60741: AMBARI-21431. Update BigInsight configuration files to be compliant with XSD

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60741/#review180035
---


Ship it!




Ship It!

- Jonathan Hurley


On July 10, 2017, 8:30 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60741/
> ---
> 
> (Updated July 10, 2017, 8:30 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21431
> https://issues.apache.org/jira/browse/AMBARI-21431
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhancements in `StackManager`:
>  * print line number of schema violation
>  * print canonical path if possible: eg. 
> `ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
>instead of 
> `ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
> 
> Added `` where missing:
>  * value of the attribute `add` matches that in `common-services` where 
> possible
>  * some properties were marked with 
> `DONT_ADD_ON_UPGRADE`, which is not 
> schema-valid, hence removed -- added `` for 
> these
>  * added with `add="false"` for a few properties in BI stack that are marked 
> `true`
>  * conversely, some properties marked as deleted in `common-services`, but 
> still present in BI stack are added with `add="true"`
> 
> Some properties had other schema problems:
>  * `` instead of ``
>  * duplicate tags
>  * wrong tag `` instead of ``
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> d9a3ac4f956b397661587f13fece80b74ddd6f32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
>  2d11ef38005262f38d59dd24b223c5b8c6c42530 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-env.xml
>  59102538b589ebf3bb792d0067ea7d29d6ef1672 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-env.xml
>  379297f8e3df2c462e7a5b9f65d683dd53a5ba9e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-log4j.xml
>  52ead29872f1f30837da9c68dba66d98bb96b3e1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-policy.xml
>  febbd44af15519b26ba0c35173207b5b51fa0fba 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-security-site.xml
>  5e7bc51809c809a56ac2c9592c611cf2a7c927e0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-site.xml
>  00d0a2c39a1834e908eb4766e7ad7587444280c2 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-log4j.xml
>  2b0a4cf20d94cdeca4320845f3d733e57f460dbe 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-site.xml
>  cc9c27a67d0213903391074f82c132fc8a50 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-conf.xml
>  582e181aeb42bbda0c330a86570e163f292fc3df 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-env.xml
>  a448a668d26c159cfff97618cb72fabbb532eac1 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-env.xml
>  47a2bb65bc48b84c77ff97e2eabb26436899aa8a 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-javaopts-properties.xml
>  3eca77a1171499bff9fe930862d01a9b266296da 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-log4j.xml
>  2e8fb3d22c95159323a1e3f691a3ef68cd3bc8b0 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-policy.xml
>  b0807b6cf2a008a053992eec103fcb4b7f7a0d67 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-site.xml
>  d4cca6c7bd1c127d92e00fff10622a720405197b 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/core-site.xml
>  f833896330028f2e2854d166b9fb107cb56a4cfe 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hadoop-env.xml
>  ba575b0ff894ad5a55e1c8c4065fec0548dc6236 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hadoop-policy.xml
>  41bde16a375d161aee4f531ab6b3728b9521c2cf 
>   
> ambari-server/src/main/resources/stacks/Bi

Re: Review Request 60744: Add python functions to get the Ambari version of the server / agent

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/#review180029
---




ambari-common/src/main/python/ambari_commons/get_ambari_version.py
Lines 28 (patched)


Version file may be changed in `ambari.properties`:

```
server.version.file=/var/lib/ambari-server/resources/version
```



ambari-common/src/main/python/ambari_commons/get_ambari_version.py
Lines 33 (patched)


Please add "server" to the message



ambari-common/src/main/python/ambari_commons/get_ambari_version.py
Lines 41 (patched)


Move inside the `if`



ambari-common/src/main/python/ambari_commons/get_ambari_version.py
Lines 51 (patched)


Please add "agent" to the message


- Attila Doroszlai


On July 10, 2017, 3:48 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60744/
> ---
> 
> (Updated July 10, 2017, 3:48 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, 
> and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21435
> https://issues.apache.org/jira/browse/AMBARI-21435
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the mpacks we may want to know the version of Ambari, for example the Log 
> Search handles it's configuration in different ways depending on it's version
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60744/diff/2/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 60744: Add python functions to get the Ambari version of the server / agent

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/
---

(Updated July 10, 2017, 1:48 p.m.)


Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, and 
Robert Nettleton.


Changes
---

fix file handling


Bugs: AMBARI-21435
https://issues.apache.org/jira/browse/AMBARI-21435


Repository: ambari


Description
---

In the mpacks we may want to know the version of Ambari, for example the Log 
Search handles it's configuration in different ways depending on it's version


Diffs (updated)
-

  ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
PRE-CREATION 


Diff: https://reviews.apache.org/r/60744/diff/2/

Changes: https://reviews.apache.org/r/60744/diff/1-2/


Testing
---

Tested on local cluster


Thanks,

Miklos Gergely

Re: Review Request 60744: Add python functions to get the Ambari version of the server / agent

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/#review180028
---




ambari-common/src/main/python/ambari_commons/get_ambari_version.py
Lines 29 (patched)


Use
"with open" 
to avoid not closing the file during exception.


- Andrew Onischuk


On July 10, 2017, 1:04 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60744/
> ---
> 
> (Updated July 10, 2017, 1:04 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, 
> and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21435
> https://issues.apache.org/jira/browse/AMBARI-21435
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the mpacks we may want to know the version of Ambari, for example the Log 
> Search handles it's configuration in different ways depending on it's version
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60744/diff/1/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Review Request 60744: Add python functions to get the Ambari version of the server / agent

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60744/
---

Review request for Ambari, Oliver Szabo and Robert Nettleton.


Bugs: AMBARI-21435
https://issues.apache.org/jira/browse/AMBARI-21435


Repository: ambari


Description
---

In the mpacks we may want to know the version of Ambari, for example the Log 
Search handles it's configuration in different ways depending on it's version


Diffs
-

  ambari-common/src/main/python/ambari_commons/get_ambari_version.py 
PRE-CREATION 


Diff: https://reviews.apache.org/r/60744/diff/1/


Testing
---

Tested on local cluster


Thanks,

Miklos Gergely

Re: Review Request 60741: AMBARI-21431. Update BigInsight configuration files to be compliant with XSD

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60741/
---

(Updated July 10, 2017, 2:30 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit Mohanty, 
and Sid Wagle.


Changes
---

unit test results


Bugs: AMBARI-21431
https://issues.apache.org/jira/browse/AMBARI-21431


Repository: ambari


Description
---

Enhancements in `StackManager`:
 * print line number of schema violation
 * print canonical path if possible: eg. 
`ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
   instead of 
`ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`

Added `` where missing:
 * value of the attribute `add` matches that in `common-services` where possible
 * some properties were marked with 
`DONT_ADD_ON_UPGRADE`, which is not 
schema-valid, hence removed -- added `` for these
 * added with `add="false"` for a few properties in BI stack that are marked 
`true`
 * conversely, some properties marked as deleted in `common-services`, but 
still present in BI stack are added with `add="true"`

Some properties had other schema problems:
 * `` instead of ``
 * duplicate tags
 * wrong tag `` instead of ``


Diffs
-

  ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
d9a3ac4f956b397661587f13fece80b74ddd6f32 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
 2d11ef38005262f38d59dd24b223c5b8c6c42530 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-env.xml
 59102538b589ebf3bb792d0067ea7d29d6ef1672 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-env.xml
 379297f8e3df2c462e7a5b9f65d683dd53a5ba9e 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-log4j.xml
 52ead29872f1f30837da9c68dba66d98bb96b3e1 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-policy.xml
 febbd44af15519b26ba0c35173207b5b51fa0fba 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-security-site.xml
 5e7bc51809c809a56ac2c9592c611cf2a7c927e0 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-site.xml
 00d0a2c39a1834e908eb4766e7ad7587444280c2 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-log4j.xml
 2b0a4cf20d94cdeca4320845f3d733e57f460dbe 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-site.xml
 cc9c27a67d0213903391074f82c132fc8a50 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-conf.xml
 582e181aeb42bbda0c330a86570e163f292fc3df 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-env.xml
 a448a668d26c159cfff97618cb72fabbb532eac1 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-env.xml
 47a2bb65bc48b84c77ff97e2eabb26436899aa8a 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-javaopts-properties.xml
 3eca77a1171499bff9fe930862d01a9b266296da 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-log4j.xml
 2e8fb3d22c95159323a1e3f691a3ef68cd3bc8b0 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-policy.xml
 b0807b6cf2a008a053992eec103fcb4b7f7a0d67 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-site.xml
 d4cca6c7bd1c127d92e00fff10622a720405197b 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/core-site.xml
 f833896330028f2e2854d166b9fb107cb56a4cfe 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hadoop-env.xml
 ba575b0ff894ad5a55e1c8c4065fec0548dc6236 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hadoop-policy.xml
 41bde16a375d161aee4f531ab6b3728b9521c2cf 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hdfs-log4j.xml
 08822eb72f598ee5b9d6299f6066c71a37c826a3 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hdfs-site.xml
 fc510fae1271d8b8c676a80bf548d4528be39c24 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/ssl-client.xml
 4513fdd2180e01e42abd7ea5988de1b46ccd946d 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/ssl-server.xml
 f95793e1ffa0a38a5f2c653a0b89

Review Request 60741: AMBARI-21431. Update BigInsight configuration files to be compliant with XSD

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60741/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit Mohanty, 
and Sid Wagle.


Bugs: AMBARI-21431
https://issues.apache.org/jira/browse/AMBARI-21431


Repository: ambari


Description
---

Enhancements in `StackManager`:
 * print line number of schema violation
 * print canonical path if possible: eg. 
`ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`
   instead of 
`ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml`

Added `` where missing:
 * value of the attribute `add` matches that in `common-services` where possible
 * some properties were marked with 
`DONT_ADD_ON_UPGRADE`, which is not 
schema-valid, hence removed -- added `` for these
 * added with `add="false"` for a few properties in BI stack that are marked 
`true`
 * conversely, some properties marked as deleted in `common-services`, but 
still present in BI stack are added with `add="true"`

Some properties had other schema problems:
 * `` instead of ``
 * duplicate tags
 * wrong tag `` instead of ``


Diffs
-

  ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
d9a3ac4f956b397661587f13fece80b74ddd6f32 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml
 2d11ef38005262f38d59dd24b223c5b8c6c42530 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-env.xml
 59102538b589ebf3bb792d0067ea7d29d6ef1672 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-env.xml
 379297f8e3df2c462e7a5b9f65d683dd53a5ba9e 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-log4j.xml
 52ead29872f1f30837da9c68dba66d98bb96b3e1 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-policy.xml
 febbd44af15519b26ba0c35173207b5b51fa0fba 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-security-site.xml
 5e7bc51809c809a56ac2c9592c611cf2a7c927e0 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-hbase-site.xml
 00d0a2c39a1834e908eb4766e7ad7587444280c2 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-log4j.xml
 2b0a4cf20d94cdeca4320845f3d733e57f460dbe 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/AMBARI_METRICS/configuration/ams-site.xml
 cc9c27a67d0213903391074f82c132fc8a50 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-conf.xml
 582e181aeb42bbda0c330a86570e163f292fc3df 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/FLUME/configuration/flume-env.xml
 a448a668d26c159cfff97618cb72fabbb532eac1 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-env.xml
 47a2bb65bc48b84c77ff97e2eabb26436899aa8a 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-javaopts-properties.xml
 3eca77a1171499bff9fe930862d01a9b266296da 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-log4j.xml
 2e8fb3d22c95159323a1e3f691a3ef68cd3bc8b0 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-policy.xml
 b0807b6cf2a008a053992eec103fcb4b7f7a0d67 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HBASE/configuration/hbase-site.xml
 d4cca6c7bd1c127d92e00fff10622a720405197b 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/core-site.xml
 f833896330028f2e2854d166b9fb107cb56a4cfe 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hadoop-env.xml
 ba575b0ff894ad5a55e1c8c4065fec0548dc6236 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hadoop-policy.xml
 41bde16a375d161aee4f531ab6b3728b9521c2cf 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hdfs-log4j.xml
 08822eb72f598ee5b9d6299f6066c71a37c826a3 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/hdfs-site.xml
 fc510fae1271d8b8c676a80bf548d4528be39c24 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/ssl-client.xml
 4513fdd2180e01e42abd7ea5988de1b46ccd946d 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/configuration/ssl-server.xml
 f95793e1ffa0a38a5f2c653a0b89c85b13d3e989 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/s

Re: Review Request 60738: HdfsResource can not handle S3 URL when hbase.rootdir is set to S3 URL

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60738/#review180016
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Липень 10, 2017, 9:47 до полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60738/
> ---
> 
> (Updated Липень 10, 2017, 9:47 до полудня)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20950
> https://issues.apache.org/jira/browse/AMBARI-20950
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBASE-17437 enables HBASE to put its rootdir on S3. Ambari calls HdfsResource
> to create the dir for HBase, but it can not handle S3.  
> Fix should be similar to AMBARI-15157 for Hive.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
>  68856ce 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  43c6036 
> 
> 
> Diff: https://reviews.apache.org/r/60738/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 60738: HdfsResource can not handle S3 URL when hbase.rootdir is set to S3 URL

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60738/#review180017
---


Ship it!




Ship It!

- Dmytro Sen


On Июль 10, 2017, 9:47 д.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60738/
> ---
> 
> (Updated Июль 10, 2017, 9:47 д.п.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-20950
> https://issues.apache.org/jira/browse/AMBARI-20950
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBASE-17437 enables HBASE to put its rootdir on S3. Ambari calls HdfsResource
> to create the dir for HBase, but it can not handle S3.  
> Fix should be similar to AMBARI-15157 for Hive.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
>  68856ce 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  43c6036 
> 
> 
> Diff: https://reviews.apache.org/r/60738/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 60738: HdfsResource can not handle S3 URL when hbase.rootdir is set to S3 URL

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60738/
---

Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-20950
https://issues.apache.org/jira/browse/AMBARI-20950


Repository: ambari


Description
---

HBASE-17437 enables HBASE to put its rootdir on S3. Ambari calls HdfsResource
to create the dir for HBase, but it can not handle S3.  
Fix should be similar to AMBARI-15157 for Hive.


Diffs
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
 68856ce 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 43c6036 


Diff: https://reviews.apache.org/r/60738/diff/1/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 60711: Assigning hosts concurrently to same config group may fail with "org.apache.ambari.server.controller.spi.ResourceAlreadyExistsException: Config group already exist'

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60711/
---

(Updated July 10, 2017, 10:44 a.m.)


Review request for Ambari, Attila Magyar, Laszlo Puskas, Robert Levas, and 
Sandor Magyari.


Changes
---

Rebased on trunk


Bugs: AMBARI-21427
https://issues.apache.org/jira/browse/AMBARI-21427


Repository: ambari


Description
---

PROBLEM:
Assigning hosts concurrently to the same config group may fail due to a race 
condition causing the config group to be created twice:

```
05 Jul 2017 18:50:44,742  INFO [pool-5-thread-2] RegisterWithConfigGroupTask:45 
- HostRequest: Executing CONFIGURE task for host: w1.internal

05 Jul 2017 18:50:44,884 ERROR [pool-5-thread-2] AmbariContext:655 - Failed to 
create new configuration group: 
org.apache.ambari.server.controller.spi.ResourceAlreadyExistsException: Config 
group already exists with the same name and tag, clusterName = mycluster, 
groupName = myblueprint:group1, tag = HDFS
05 Jul 2017 18:50:44,885 ERROR [pool-5-thread-2] AmbariContext:342 - Unable to 
register config group for host: 
java.lang.RuntimeException: Failed to create new configuration group: 
org.apache.ambari.server.controller.spi.ResourceAlreadyExistsException: Config 
group already exists with the same name and tag, clusterName = mycluster, 
groupName = myblueprint:group1, tag = HDFS
at 
org.apache.ambari.server.topology.AmbariContext.createConfigGroupsAndRegisterHost(AmbariContext.java:656)
at 
org.apache.ambari.server.topology.AmbariContext.registerHostWithConfigGroup(AmbariContext.java:339)
at 
org.apache.ambari.server.topology.tasks.RegisterWithConfigGroupTask.runTask(RegisterWithConfigGroupTask.java:46)
at 
org.apache.ambari.server.topology.tasks.TopologyHostTask.run(TopologyHostTask.java:51)
at 
org.apache.ambari.server.topology.HostOfferResponse$1.run(HostOfferResponse.java:83)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: 
org.apache.ambari.server.controller.spi.ResourceAlreadyExistsException: Config 
group already exists with the same name and tag, clusterName = mycluster, 
groupName = myblueprint:group1, tag = HDFS
at 
org.apache.ambari.server.controller.internal.AbstractResourceProvider.createResources(AbstractResourceProvider.java:282)
at 
org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider.createResources(ConfigGroupResourceProvider.java:276)
at 
org.apache.ambari.server.topology.AmbariContext.createConfigGroupsAndRegisterHost(AmbariContext.java:653)
... 7 more
```

SOLUTION:
Synchronize the verification of the config group existence and creation of a 
config group with the same name.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
 9b64edc 


Diff: https://reviews.apache.org/r/60711/diff/2/

Changes: https://reviews.apache.org/r/60711/diff/1-2/


Testing
---

Manual testing:
Created a cluster using Blueprint with configurations specified in the host 
groups to trigger config group creation.
ambari.properties included ```topology.task.creation.parallel=true``` to 
trigger parallel host assignment to config groups.


Unit tests:
Failed tests: 
  
RestMetricsPropertyProviderTest.testRestMetricsPropertyProviderAsAdministrator:222->testPopulateResourcesMany:457
 expected:<400> but was:<121>

Tests run: 4797, Failures: 1, Errors: 0, Skipped: 35

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Main  SUCCESS [ 53.583 s]
[INFO] Apache Ambari Project POM .. SUCCESS [  0.012 s]
[INFO] Ambari Views ... SUCCESS [  5.241 s]
[INFO] ambari-utility . SUCCESS [  7.855 s]
[INFO] ambari-metrics . SUCCESS [  1.334 s]
[INFO] Ambari Metrics Common .. SUCCESS [ 10.795 s]
[INFO] Ambari Service Advisor . SUCCESS [  0.886 s]
[INFO] Ambari Server .. FAILURE [  01:05 h]
[INFO] 


One metrics test failed and is not related to this change.


Thanks,

Sebastian Toader

Re: Review Request 60707: BE: Extend Ambari REST API to Support User Account Management Improvements

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60707/#review180009
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserAuthenticationSourceResourceProvider.java
Lines 76-78 (patched)


You could use guava's ImmutableSet.of(...) here.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserAuthenticationSourceResourceProvider.java
Lines 80-88 (patched)


You could use guava's ImmutableSet.builder() here.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserAuthenticationSourceResourceProvider.java
Lines 90-93 (patched)


You could use guava's ImmutableMap.of(...) here.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java
Lines 101-123 (patched)


You could use guava here.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java
Lines 367-369 (patched)


Shouldn't be an error message returned here in case no pwd passed in for 
local users?



ambari-server/src/test/java/org/apache/ambari/server/api/resources/UserResourceDefinitionTest.java
Line 42 (original), 44 (patched)


It worth adding a test that ensures that none breaks the equals method of 
UserDefinition in the future by adding a new field to the class but forgetting 
to update the equals and hashcode methods accordingly.

There a test library that does that for you ```EqualsVerifier``` that can 
be used for that. (look at ```OperationStatusAuditEventTest.testEquals()``` as 
an example how currently is used.


- Sebastian Toader


On July 7, 2017, 3:30 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60707/
> ---
> 
> (Updated July 7, 2017, 3:30 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-20861
> https://issues.apache.org/jira/browse/AMBARI-20861
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update the Ambari REST API to allow for GET, POST, PUT, and DELETE operations 
> on the authentication sources related to an Ambari user account.
> 
> Add the following entry points:
> 
> - `/api/v1/users/:USER_ID`
> -- Get a user
> -- Modify a user
> 
> - `/api/v1/users/:USERNAME/sources`
> -- List a user’s authentication sources
> -- Add a new authentication source for a user
> 
> - `/api/v1/users/:USERNAME/sources/:SOURCE_ID`
> -- Get details on a specific authentication source for a user
> -- Modify details for a specific authentication source for a user
> 
> - `/api/v1/users/:USER_ID/sources`
> -- List a user’s authentication sources
> -- Add a new authentication source for a user
> 
> - `/api/v1/users/:USER_ID/sources/:SOURCE_ID`
> -- Get details on a specific authentication source for a user
> -- Modify details for a specific authentication source for a user
> 
> Update the following entry points, ensuring backwards compatibility where 
> possible:
> 
> - `/api/v1/users`
> -- List all users
> -- Add a new user
> -- Backward compatibility: Set password should create or update the 
> appropriate user_authentication record.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/api/v1/authentication-source-create.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-delete.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-get.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-list.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-resources.md PRE-CREATION 
>   ambari-server/docs/api/v1/authentication-source-update.md PRE-CREATION 
>   ambari-server/docs/api/v1/index.md da05d75bc3 
>   ambari-server/docs/api/v1/user-create.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-delete.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-get.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-list.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-resources.md PRE-CREATION 
>   ambari-server/docs/api/v1/user-update.md PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  96e288fee0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/UserResourceDefinition.java
>  b228c82aa9 
>   
> ambari-server/src/main/java/org/apa