date:20150921


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38567/
---

(Updated Sept. 21, 2015, 7:50 p.m.)


Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-13171
https://issues.apache.org/jira/browse/AMBARI-13171


Repository: ambari


Description
---

During an upgrade, AMBARI-12924 and AMBARI-12889 allow for a host to be placed 
into Maintenance Mode in order for it to be excluded from the upgrade. Although 
there is work planned to be able to finalize an upgrade with hosts that are in 
MM and out-of-sync, in the short term there also needs to be the ability to 
remove these hosts so that finalization can complete.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/events/HostRemovedEvent.java
 e005754 
  
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java
 0824760 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterEntity.java
 3fe541f 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
 6ede46b 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
 62613ff 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 eda2534 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 972ceec 
  
ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java
 b056991 
  ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 
53f7432 

Diff: https://reviews.apache.org/r/38567/diff/


Testing
---

mvn clean test


Thanks,

Jonathan Hurley

[jira] [Updated] (AMBARI-13094) Add Spark Thrift Ambari Service

2015-09-21 Thread Sumit Mohanty (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-13094:
---
Assignee: Judy Nash

> Add Spark Thrift Ambari Service
> ---
>
> Key: AMBARI-13094
> URL: https://issues.apache.org/jira/browse/AMBARI-13094
> Project: Ambari
>  Issue Type: New Feature
>Affects Versions: trunk
>Reporter: Judy Nash
>Assignee: Judy Nash
>  Labels: patch
> Fix For: trunk
>
> Attachments: AMBARI-13094.patch, Ambari Service for Spark Thrift 
> Design Specification.docx
>
>
> New feature to add spark thrift server support on Ambari. 
> Design specification attached. 
> Instruction to add thrift server to an existing cluster:
> 1) If running on HDP distro, Update metainfo.xml @ 
> /var/lib/ambari-server/resources/stacks/HDP/2.3/services/SPARK/metainfo.xml. 
> Change true to false. 
> 2) Use Ambari UI if spark has not been installed before. It will show up as a 
> installable service alongside with spark history server. 
> 3) If spark component has been installed already, use API to add 
> thrift-server as a new service. Service name: SPARK_THRIFTSERVER



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13164) RU: Knox to use versioned data dir starting in HDP 2.3.2.0


[ 
https://issues.apache.org/jira/browse/AMBARI-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901644#comment-14901644
 ] 

Hudson commented on AMBARI-13164:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3480 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3480/])
AMBARI-13164. RU: Knox to use versioned data dir starting in HDP 2.3.2.0 
(alejandro) (afernandez: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=f9271abc25614adbcfa00599db643a24ac20f0f2)
* 
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
* 
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
* ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py
* 
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
* ambari-common/src/main/python/resource_management/libraries/script/script.py


> RU: Knox to use versioned data dir starting in HDP 2.3.2.0
> --
>
> Key: AMBARI-13164
> URL: https://issues.apache.org/jira/browse/AMBARI-13164
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
> Fix For: 2.2.0, 2.1.2
>
> Attachments: AMBARI-13164.branch-2.1.2.patch, 
> AMBARI-13164.branch-2.1.patch, AMBARI-13164.trunk.patch
>
>
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> There are 3 scenarios:
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38586: Ranger Database related property values should be recommended by stack advisor

2015-09-21 Thread Yusaku Sako


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38586/#review99897
---

Ship it!


Ship It!

- Yusaku Sako


On Sept. 22, 2015, 1:17 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38586/
> ---
> 
> (Updated Sept. 22, 2015, 1:17 a.m.)
> 
> 
> Review request for Ambari, Srimanth Gunturi, Sumit Gupta, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-13175
> https://issues.apache.org/jira/browse/AMBARI-13175
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
> changed.
> 
> Also changing of ${admin-properties/db_host} and ${admin-properties/db_name} 
> should change ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.
> Please see https://issues.apache.org/jira/browse/AMBARI-13175 for further 
> description details
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/admin-properties.xml
>  5fdb1b9 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 5568e60 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/configuration/ranger-admin-site.xml
>  43be765 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> d7ea10f 
>   ambari-web/app/data/db_properties_info.js d48dc98 
>   ambari-web/app/views/common/controls_view.js a7bf5a0 
> 
> Diff: https://reviews.apache.org/r/38586/diff/
> 
> 
> Testing
> ---
> 
> Tested the patch on a cluster.
> 
> Verified that all ambari-web unit tests passes:
> 9602 tests complete (12 seconds)
> 94 tests pending
> 
> Also verified that ambari-server python unit tests are passing successfully:
> --
> Ran 231 tests in 6.860s
> OK
> --
> Total run:795
> Total errors:0
> Total failures:0
> OK
> Process finished with exit code 0
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

[jira] [Commented] (AMBARI-13055) [Hive View Visualization] Add more options for max number of rows for reports/charts


[ 
https://issues.apache.org/jira/browse/AMBARI-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901764#comment-14901764
 ] 

Hudson commented on AMBARI-13055:
-

SUCCESS: Integrated in Ambari-branch-2.1 #571 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/571/])
AMBARI-13055. [Hive View Visualization] Add more options for max number of rows 
for reports/charts. (Pallav Kulshreshtha via yusaku) (yusaku: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=1e6707e1a40d06434279a93977136d334a747a8e)
* 
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/visualization-ui.hbs
* 
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/visualization-ui.js
* contrib/views/hive/src/main/resources/ui/hive-web/app/utils/constants.js
* 
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/components/visualization-tabs-widget.hbs
* contrib/views/hive/src/main/resources/ui/hive-web/app/styles/app.scss


> [Hive View Visualization] Add more options for max number of rows for 
> reports/charts
> 
>
> Key: AMBARI-13055
> URL: https://issues.apache.org/jira/browse/AMBARI-13055
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-views
>Affects Versions: 2.1.2
>Reporter: Pallav Kulshreshtha
>Assignee: Pallav Kulshreshtha
> Fix For: 2.1.3
>
> Attachments: AMBARI-13055_branch-2.1.patch
>
>
> Go to hive view and run a query. Click "Visualization" tab after getting the 
> query results.
> Problem: All the charts (under data explorer and advance visualization) have 
> plotted for maximum 3 rows (hardcoded)
> Solution: Uses should be given more options for analyzing the data in a 
> better way. 
> These maximum number of rows options can be 1, 3, 5, 8, 
> 10, 13, 15, 18 and 20



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13175) Ranger Database related property values should be recommended by stack advisor


[ 
https://issues.apache.org/jira/browse/AMBARI-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901868#comment-14901868
 ] 

Hudson commented on AMBARI-13175:
-

SUCCESS: Integrated in Ambari-branch-2.1 #572 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/572/])
AMBARI-13175. Ranger Database related property values should be recommended by 
stack advisor. (jaimin) (jaimin: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=6a006eee5f6f87219a04f52a199be8b2446aad7c)
* ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
* 
ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/admin-properties.xml
* ambari-web/app/views/common/controls_view.js
* ambari-web/app/data/db_properties_info.js
* 
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/configuration/ranger-admin-site.xml
* ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py


> Ranger Database related property values should be recommended by stack advisor
> --
>
> Key: AMBARI-13175
> URL: https://issues.apache.org/jira/browse/AMBARI-13175
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web, stacks
>Affects Versions: 2.1.3
>Reporter: Jaimin D Jetly
>Assignee: Jaimin D Jetly
> Fix For: 2.1.3
>
> Attachments: AMBARI-13175.patch, AMBARI-13175_2.patch
>
>
> Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
> changed.
> Following properties should be changed accordingly by stack advisor:
> ||DB_FLAVOR value||Dependent properties values||
> |MYSQL|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar 
> \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url=jdbc:mysql://$\{admin-properties/db_host}/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.mysql.jdbc.Driver (HDP-2.3 
> onwards)
> |ORACLE|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:oracle:thin:@/$\{admin-properties/db_host}:1521/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=oracle.jdbc.driver.OracleDriver 
> (HDP-2.3 onwards)
> |POSTGRES|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:postgresql://$\{admin-properties/db_host}:5432/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=org.postgresql.Driver (HDP-2.3 
> onwards)
> |MSSQL|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlserver://$\{admin-properties/db_host};databaseName=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
>  (HDP-2.3 onwards)
> |SQLA|1) 
> admin-properties/SQL_CONNECTOR_JAR=/path_to_driver/sqla-client-jdbc.tar.gz 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlanywhere:host=$\{admin-properties/db_host};database=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=sap.jdbc4.sqlanywhere.IDriver 
> (HDP-2.3 onwards)
> Also changing of $\{admin-properties/db_host} and 
> $\{admin-properties/db_name\} should change 
> ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13175) Ranger Database related property values should be recommended by stack advisor


[ 
https://issues.apache.org/jira/browse/AMBARI-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901870#comment-14901870
 ] 

Hudson commented on AMBARI-13175:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3482 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3482/])
AMBARI-13175. Ranger Database related property values should be recommended by 
stack advisor. (jaimin) (jaimin: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=f596785cdec78a1a975509d72ba305904749f2c2)
* ambari-web/app/views/common/controls_view.js
* 
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/configuration/ranger-admin-site.xml
* ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py
* ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
* 
ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/admin-properties.xml
* ambari-web/app/data/db_properties_info.js


> Ranger Database related property values should be recommended by stack advisor
> --
>
> Key: AMBARI-13175
> URL: https://issues.apache.org/jira/browse/AMBARI-13175
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web, stacks
>Affects Versions: 2.1.3
>Reporter: Jaimin D Jetly
>Assignee: Jaimin D Jetly
> Fix For: 2.1.3
>
> Attachments: AMBARI-13175.patch, AMBARI-13175_2.patch
>
>
> Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
> changed.
> Following properties should be changed accordingly by stack advisor:
> ||DB_FLAVOR value||Dependent properties values||
> |MYSQL|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar 
> \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url=jdbc:mysql://$\{admin-properties/db_host}/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.mysql.jdbc.Driver (HDP-2.3 
> onwards)
> |ORACLE|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:oracle:thin:@/$\{admin-properties/db_host}:1521/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=oracle.jdbc.driver.OracleDriver 
> (HDP-2.3 onwards)
> |POSTGRES|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:postgresql://$\{admin-properties/db_host}:5432/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=org.postgresql.Driver (HDP-2.3 
> onwards)
> |MSSQL|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlserver://$\{admin-properties/db_host};databaseName=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
>  (HDP-2.3 onwards)
> |SQLA|1) 
> admin-properties/SQL_CONNECTOR_JAR=/path_to_driver/sqla-client-jdbc.tar.gz 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlanywhere:host=$\{admin-properties/db_host};database=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=sap.jdbc4.sqlanywhere.IDriver 
> (HDP-2.3 onwards)
> Also changing of $\{admin-properties/db_host} and 
> $\{admin-properties/db_name\} should change 
> ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-9421) truncate path string after '=' in flume.conf when sync from ambari server

2015-09-21 Thread Colman Madden (JIRA)


[ 
https://issues.apache.org/jira/browse/AMBARI-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901968#comment-14901968
 ] 

Colman Madden commented on AMBARI-9421:
---

We were having the same issue so we came up with a work around to get us by 
until the fix is released
In Ambari edit the flume-env.sh file (one gets created for every agent and it 
gets run every time before it starts the agent)

1) Build the flume config with a token such as '&' where you want the '=' 
character
2) Edit the flume_env.sh file in Ambari to do the replace with the below two 
lines.

echo fixing flume.conf replacing '&' with '=' in file ${BASH_ARGV[3]}
sed -i 's/&/=/g' ${BASH_ARGV[3]}

Note: BASH_ARGV[3] gives you the full file path for the flume conf file

Works really well!!

> truncate path string after '=' in flume.conf when sync from ambari server
> -
>
> Key: AMBARI-9421
> URL: https://issues.apache.org/jira/browse/AMBARI-9421
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-agent
>Affects Versions: 1.7.0
> Environment: centos 6.5
>Reporter: xiao_cao
>Priority: Critical
> Fix For: 2.1.2
>
>
> if I change the property of flume config in ambari-admin:
> collector.sinks.sink_hdfs.hdfs.path=/user/flume/%{category}/dt=%Y%m%d/hour=%H
> after save and restart flume in ambari-admin, the flume conf that sync from 
> master in agent:
> collector.sinks.sink_hdfs.hdfs.path=/user/flume/%{category}/dt



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13164) RU: Knox to use versioned data dir starting in HDP 2.3.2.0


[ 
https://issues.apache.org/jira/browse/AMBARI-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901578#comment-14901578
 ] 

Hudson commented on AMBARI-13164:
-

SUCCESS: Integrated in Ambari-branch-2.1 #570 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/570/])
AMBARI-13164. RU: Knox to use versioned data dir starting in HDP 2.3.2.0 
(alejandro) (afernandez: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=f11802c73b9e18b664ae8107b6d00fa0aedf32be)
* ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py
* 
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
* 
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
* 
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
* ambari-common/src/main/python/resource_management/libraries/script/script.py


> RU: Knox to use versioned data dir starting in HDP 2.3.2.0
> --
>
> Key: AMBARI-13164
> URL: https://issues.apache.org/jira/browse/AMBARI-13164
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
> Fix For: 2.2.0, 2.1.2
>
> Attachments: AMBARI-13164.branch-2.1.2.patch, 
> AMBARI-13164.branch-2.1.patch, AMBARI-13164.trunk.patch
>
>
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> There are 3 scenarios:
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-13175) Ranger Database related property values should be recommended by stack advisor

Jaimin D Jetly created AMBARI-13175:
---

 Summary: Ranger Database related property values should be 
recommended by stack advisor
 Key: AMBARI-13175
 URL: https://issues.apache.org/jira/browse/AMBARI-13175
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server, ambari-web, stacks
Affects Versions: 2.1.3
Reporter: Jaimin D Jetly
Assignee: Jaimin D Jetly
 Fix For: 2.1.3






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13175) Ranger Database related property values should be recommended by stack advisor


 [ 
https://issues.apache.org/jira/browse/AMBARI-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaimin D Jetly updated AMBARI-13175:

Description: 
Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
changed.
Following properties should be changed accordingly by stack advisor:
||DB_FLAVOR value||Dependent properties values||
|MYSQL|1) 
admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar 
\\2) 
ranger-admin-site/ranger.jpa.jdbc.url=jdbc:mysql://$\{admin-properties/db_host}/$\{admin-properties/db_name\}
 (HDP-2.3 onwards)\\3) 
ranger-admin-site/ranger.jpa.jdbc.driver=com.mysql.jdbc.Driver (HDP-2.3 onwards)
|ORACLE|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar \\2) 
ranger-admin-site/ranger.jpa.jdbc.url= 
jdbc:oracle:thin:@/$\{admin-properties/db_host}:1521/$\{admin-properties/db_name\}
 (HDP-2.3 onwards)\\3) 
ranger-admin-site/ranger.jpa.jdbc.driver=oracle.jdbc.driver.OracleDriver 
(HDP-2.3 onwards)
|POSTGRES|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar 
\\2) ranger-admin-site/ranger.jpa.jdbc.url= 
jdbc:postgresql://$\{admin-properties/db_host}:5432/$\{admin-properties/db_name\}
 (HDP-2.3 onwards) \\3) 
ranger-admin-site/ranger.jpa.jdbc.driver=org.postgresql.Driver (HDP-2.3 onwards)
|MSSQL|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar \\2) 
ranger-admin-site/ranger.jpa.jdbc.url= 
jdbc:sqlserver://$\{admin-properties/db_host};databaseName=$\{admin-properties/db_name\}
 (HDP-2.3 onwards) \\3) 
ranger-admin-site/ranger.jpa.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
 (HDP-2.3 onwards)
|SQLA|1) 
admin-properties/SQL_CONNECTOR_JAR=/path_to_driver/sqla-client-jdbc.tar.gz \\2) 
ranger-admin-site/ranger.jpa.jdbc.url= 
jdbc:sqlanywhere:host=$\{admin-properties/db_host};database=$\{admin-properties/db_name\}
 (HDP-2.3 onwards) \\3) 
ranger-admin-site/ranger.jpa.jdbc.driver=sap.jdbc4.sqlanywhere.IDriver (HDP-2.3 
onwards)

Also changing of $\{admin-properties/db_host} and $\{admin-properties/db_name\} 
should change ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.

> Ranger Database related property values should be recommended by stack advisor
> --
>
> Key: AMBARI-13175
> URL: https://issues.apache.org/jira/browse/AMBARI-13175
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web, stacks
>Affects Versions: 2.1.3
>Reporter: Jaimin D Jetly
>Assignee: Jaimin D Jetly
> Fix For: 2.1.3
>
>
> Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
> changed.
> Following properties should be changed accordingly by stack advisor:
> ||DB_FLAVOR value||Dependent properties values||
> |MYSQL|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar 
> \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url=jdbc:mysql://$\{admin-properties/db_host}/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.mysql.jdbc.Driver (HDP-2.3 
> onwards)
> |ORACLE|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:oracle:thin:@/$\{admin-properties/db_host}:1521/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=oracle.jdbc.driver.OracleDriver 
> (HDP-2.3 onwards)
> |POSTGRES|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:postgresql://$\{admin-properties/db_host}:5432/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=org.postgresql.Driver (HDP-2.3 
> onwards)
> |MSSQL|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlserver://$\{admin-properties/db_host};databaseName=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
>  (HDP-2.3 onwards)
> |SQLA|1) 
> admin-properties/SQL_CONNECTOR_JAR=/path_to_driver/sqla-client-jdbc.tar.gz 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlanywhere:host=$\{admin-properties/db_host};database=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=sap.jdbc4.sqlanywhere.IDriver 
> (HDP-2.3 onwards)
> Also changing of $\{admin-properties/db_host} and 
> $\{admin-properties/db_name\} should change 
> ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38586: Ranger Database related property values should be recommended by stack advisor

2015-09-21 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38586/
---

Review request for Ambari, Srimanth Gunturi, Sumit Gupta, and Yusaku Sako.


Bugs: AMBARI-13175
https://issues.apache.org/jira/browse/AMBARI-13175


Repository: ambari


Description
---

Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
changed.

Also changing of ${admin-properties/db_host} and ${admin-properties/db_name} 
should change ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.
Please see https://issues.apache.org/jira/browse/AMBARI-13175 for further 
description details


Diffs
-

  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/admin-properties.xml
 5fdb1b9 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
5568e60 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/configuration/ranger-admin-site.xml
 43be765 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/themes/theme.json
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
d7ea10f 
  ambari-web/app/data/db_properties_info.js d48dc98 
  ambari-web/app/views/common/controls_view.js a7bf5a0 

Diff: https://reviews.apache.org/r/38586/diff/


Testing
---

Tested the patch on a cluster.

Verified that all ambari-web unit tests passes:
9602 tests complete (12 seconds)
94 tests pending

Also verified that ambari-server python unit tests are passing successfully:
--
Ran 231 tests in 6.860s
OK
--
Total run:795
Total errors:0
Total failures:0
OK
Process finished with exit code 0


Thanks,

Jaimin Jetly

[jira] [Commented] (AMBARI-13055) [Hive View Visualization] Add more options for max number of rows for reports/charts


[ 
https://issues.apache.org/jira/browse/AMBARI-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901783#comment-14901783
 ] 

Hudson commented on AMBARI-13055:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3481 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3481/])
AMBARI-13055. [Hive View Visualization] Add more options for max number of rows 
for reports/charts. (Pallav Kulshreshtha via yusaku) (yusaku: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=6d165213e25ff31ce64e0bd92f07ed986f8bac28)
* 
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/visualization-ui.hbs
* 
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/components/visualization-tabs-widget.hbs
* 
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/visualization-ui.js
* contrib/views/hive/src/main/resources/ui/hive-web/app/utils/constants.js
* contrib/views/hive/src/main/resources/ui/hive-web/app/styles/app.scss


> [Hive View Visualization] Add more options for max number of rows for 
> reports/charts
> 
>
> Key: AMBARI-13055
> URL: https://issues.apache.org/jira/browse/AMBARI-13055
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-views
>Affects Versions: 2.1.2
>Reporter: Pallav Kulshreshtha
>Assignee: Pallav Kulshreshtha
> Fix For: 2.1.3
>
> Attachments: AMBARI-13055_branch-2.1.patch
>
>
> Go to hive view and run a query. Click "Visualization" tab after getting the 
> query results.
> Problem: All the charts (under data explorer and advance visualization) have 
> plotted for maximum 3 rows (hardcoded)
> Solution: Uses should be given more options for analyzing the data in a 
> better way. 
> These maximum number of rows options can be 1, 3, 5, 8, 
> 10, 13, 15, 18 and 20



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38553: Ambari with sqla db fails to start by non root user. Problem with jdbc.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38553/
---

Review request for Ambari and Myroslav Papirkovskyy.


Bugs: AMBARI-13166
https://issues.apache.org/jira/browse/AMBARI-13166


Repository: ambari


Description
---

{code}
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
sap.jdbc4.sqlanywhere.IDriver
at 
org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:103)
at 
org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance()
at 
com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
at 
com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
at 
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
at 
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
at 
com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
at 
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
at 
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
at 
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at 
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at 
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
at 
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.Scopes$1$1.get(Scopes.java:65)
{code}


Diffs
-

  ambari-server/src/main/python/ambari_server/dbConfiguration.py 2d39c72 

Diff: https://reviews.apache.org/r/38553/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

[jira] [Updated] (AMBARI-13036) Enable JPA statement caching (internal/external)

2015-09-21 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/AMBARI-13036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-13036:
--
Attachment: AMBARI-13036.patch

> Enable JPA statement caching (internal/external)
> 
>
> Key: AMBARI-13036
> URL: https://issues.apache.org/jira/browse/AMBARI-13036
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: 2.1.1
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
> Fix For: 2.2.0
>
> Attachments: AMBARI-13036.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13036) Enable JPA statement caching (internal/external)

2015-09-21 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/AMBARI-13036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-13036:
--
Attachment: (was: AMBARI-13036.patch)

> Enable JPA statement caching (internal/external)
> 
>
> Key: AMBARI-13036
> URL: https://issues.apache.org/jira/browse/AMBARI-13036
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: 2.1.1
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
> Fix For: 2.2.0
>
> Attachments: AMBARI-13036.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13167) Ambari fails to install Ranger KMS, "config validation failed" python key traceback


 [ 
https://issues.apache.org/jira/browse/AMBARI-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sekhon updated AMBARI-13167:
-
Attachment: Screen Shot 2015-09-21 at 13.51.37.png
Screen Shot 2015-09-21 at 13.51.47.png

> Ambari fails to install Ranger KMS, "config validation failed" python key 
> traceback
> ---
>
> Key: AMBARI-13167
> URL: https://issues.apache.org/jira/browse/AMBARI-13167
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.0
> Environment: HDP 2.3 + Kerberos + Ranger installed
>Reporter: Hari Sekhon
> Attachments: Screen Shot 2015-09-21 at 13.51.37.png, Screen Shot 
> 2015-09-21 at 13.51.47.png
>
>
> When trying to install Ranger KMS via Ambari I get an error pop up that 
> config validation failed and the following error:
> {code}Error occured in stack advisor.
> Error details: 'hive.security.authorization.manager'
> 21 Sep 2015 08:52:28,873  INFO [qtp-client-2798] StackAdvisorRunner:71 - 
> advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 154, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 113, 
> in main
> result = stackAdvisor.validateConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 463, in validateConfigurations
> validationItems = self.getConfigurationsValidationItems(services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 348, in getConfigurationsValidationItems
> resultItems = method(siteProperties, siteRecommendations, configurations, 
> services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.3/services/stack_advisor.py",
>  line 231, in validateHiveServer2Configurations
> super(HDP23StackAdvisor, 
> self).validateHiveServer2Configurations(properties, recommendedDefaults, 
> configurations, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.2/services/stack_advisor.py",
>  line 906, in validateHiveServer2Configurations
> if hive_server2[prop_name] != prop_val:
> KeyError: 'hive.security.authorization.manager'
> 21 Sep 2015 08:52:28,874  WARN [qtp-client-2798] AbstractResourceProvider:90 
> - Error occurred during validation
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorException: 
> Stack Advisor reported an error: KeyError: 
> 'hive.security.authorization.manager'{code}
> I've attached a couple of screen shots and the full Ambari log.
> Ambari does actually progress to the next Configure Identities screen 
> regardless however.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-13167) Ambari fails to install Ranger KMS, "config validation failed" python key traceback

Hari Sekhon created AMBARI-13167:


 Summary: Ambari fails to install Ranger KMS, "config validation 
failed" python key traceback
 Key: AMBARI-13167
 URL: https://issues.apache.org/jira/browse/AMBARI-13167
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 2.1.0
 Environment: HDP 2.3 + Kerberos + Ranger installed
Reporter: Hari Sekhon
 Attachments: Screen Shot 2015-09-21 at 13.51.37.png, Screen Shot 
2015-09-21 at 13.51.47.png

When trying to install Ranger KMS via Ambari I get an error pop up that config 
validation failed and the following error:
{code}Error occured in stack advisor.
Error details: 'hive.security.authorization.manager'
21 Sep 2015 08:52:28,873  INFO [qtp-client-2798] StackAdvisorRunner:71 - 
advisor script stderr: Traceback (most recent call last):
  File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 154, 
in 
main(sys.argv)
  File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 113, 
in main
result = stackAdvisor.validateConfigurations(services, hosts)
  File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
line 463, in validateConfigurations
validationItems = self.getConfigurationsValidationItems(services, hosts)
  File 
"/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
 line 348, in getConfigurationsValidationItems
resultItems = method(siteProperties, siteRecommendations, configurations, 
services, hosts)
  File 
"/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.3/services/stack_advisor.py",
 line 231, in validateHiveServer2Configurations
super(HDP23StackAdvisor, 
self).validateHiveServer2Configurations(properties, recommendedDefaults, 
configurations, services, hosts)
  File 
"/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.2/services/stack_advisor.py",
 line 906, in validateHiveServer2Configurations
if hive_server2[prop_name] != prop_val:
KeyError: 'hive.security.authorization.manager'
21 Sep 2015 08:52:28,874  WARN [qtp-client-2798] AbstractResourceProvider:90 - 
Error occurred during validation
org.apache.ambari.server.api.services.stackadvisor.StackAdvisorException: Stack 
Advisor reported an error: KeyError: 'hive.security.authorization.manager'{code}
I've attached a couple of screen shots and the full Ambari log.

Ambari does actually progress to the next Configure Identities screen 
regardless however.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13162) check_fs_root should be skipped for non hdfs FS

2015-09-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/AMBARI-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900488#comment-14900488
 ] 

Hadoop QA commented on AMBARI-13162:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12761346/AMBARI-13162.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3827//console

This message is automatically generated.

> check_fs_root should be skipped for non hdfs FS
> ---
>
> Key: AMBARI-13162
> URL: https://issues.apache.org/jira/browse/AMBARI-13162
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.2
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13162.patch
>
>
> check_fs_root includes the logic to modify the FS path when it changes. The 
> current implementation only supports it for hdfs. So as it stands today it 
> should be skipped for non hdfs paths. A follow up JIRA is needed to provide 
> support for any FS path. This will require Ambari remembering the prior value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37715: Add Visibility Attributes to Services

2015-09-21 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37715/#review99740
---

Ship it!


Ship It!

- Robert Levas


On Sept. 18, 2015, 12:02 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37715/
> ---
> 
> (Updated Sept. 18, 2015, 12:02 p.m.)
> 
> 
> Review request for Ambari, John Speidel, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-7469
> https://issues.apache.org/jira/browse/AMBARI-7469
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem
> In a stack there may be one or more services that should not be installable, 
> configurable, or managed via the Ambari web-based interface. Such a service 
> may need to be installed via some Ambari API call or manually. There is no 
> way to specify these “visibility” attributes within a service’s definition; 
> thus to “hide” a service, one-off code needs to be added to the different 
> Ambari facilities.
> 
> Solution
> Add visibility attributes to service definitions to describe how Ambari 
> should expose services via front-end facilities. The following Boolean 
> attributes should be settable by services:
> - installable: indicates if Ambari can install the service - if not 
> installable, the service should be hidden from “add service” features of 
> Ambari
> - managed: indicates if Ambari can start and stop the service - if not 
> managed, the service should be hidden from all views where management 
> operations can occur
> - monitored: indicates if Ambari can monitor the service - if not monitored, 
> status information should not be displayed
> 
> 
> These attributes are assumed to be trueif not specified in the service’s 
> metainfo.xml file, else
> they are declared the a propertiesblock as follows:
> 
> …
> 
> 
> …
> 
> 
> installable
> false
> 
> 
> managed
> true
> 
> 
> …
> 
> 
> 
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml 3377961 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java
>  d16f4d6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java
>  dffc74c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> e51eb21 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> d9a8a51 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServicePropertyInfo.java
>  PRE-CREATION 
>   ambari-server/src/main/resources/properties.json 0837af2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/StackServiceResponseTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackServiceResourceProviderTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/ServiceModuleTest.java
>  2737695 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceInfoTest.java
>  75e0991 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertyInfoTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/37715/diff/
> 
> 
> Testing
> ---
> 
> Manual tested on trunk:
> - all three attributes with default value "true" are returned in case these 
> are not dedined in metainfo.xml
> - merging between stack service and common services for these new attributes
> 
> Local test results:
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:11.974s
> [INFO] Finished at: Mon Aug 24 15:18:11 CEST 2015
> [INFO] Final Memory: 59M/994M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

[jira] [Updated] (AMBARI-13147) Heatmaps page: page hangs for a while on HDFS metrics

2015-09-21 Thread Andrii Tkach (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrii Tkach updated AMBARI-13147:
--
Fix Version/s: (was: 2.1.2)
   2.1.3

> Heatmaps page: page hangs for a while on HDFS metrics
> -
>
> Key: AMBARI-13147
> URL: https://issues.apache.org/jira/browse/AMBARI-13147
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Fix For: 2.1.3
>
> Attachments: AMBARI-13147.patch
>
>
> *Steps to reproduce:*
> 1. Go to Heatmaps page
> 2. Choose DataNode Process Network I/O Utilization metric
> JS execution takes 11 seconds. (400-node cluster)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13147) Heatmaps page: page hangs for a while on HDFS metrics

2015-09-21 Thread Andrii Tkach (JIRA)


[ 
https://issues.apache.org/jira/browse/AMBARI-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900560#comment-14900560
 ] 

Andrii Tkach commented on AMBARI-13147:
---

committed to trunk and branch-2.1

> Heatmaps page: page hangs for a while on HDFS metrics
> -
>
> Key: AMBARI-13147
> URL: https://issues.apache.org/jira/browse/AMBARI-13147
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Fix For: 2.1.3
>
> Attachments: AMBARI-13147.patch
>
>
> *Steps to reproduce:*
> 1. Go to Heatmaps page
> 2. Choose DataNode Process Network I/O Utilization metric
> JS execution takes 11 seconds. (400-node cluster)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38547: Oozie service have "Database host" required config after ambari upgrade from 1.7.0 to 2.1.2


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38547/#review99739
---

Ship it!


Ship It!

- Vitalyi Brodetskyi


On Вер. 21, 2015, 9:49 до полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38547/
> ---
> 
> (Updated Вер. 21, 2015, 9:49 до полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro 
> Sen.
> 
> 
> Bugs: AMBARI-13165
> https://issues.apache.org/jira/browse/AMBARI-13165
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Oozie service have required config "Database host" after ambari upgrade from 
> 1.7.0 to 2.1.2
> STR
> install ambari 1.7.0 and deploy cluster
> upgrade ambari to 2.1.2
> navigate to oozie service configs
> ACTUAL: "Database host" is required
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  f4023b2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
>  b97a7f4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog212Test.java
>  5a947e3 
> 
> Diff: https://reviews.apache.org/r/38547/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

[jira] [Commented] (AMBARI-13142) UI configs clean up


[ 
https://issues.apache.org/jira/browse/AMBARI-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900500#comment-14900500
 ] 

Hudson commented on AMBARI-13142:
-

SUCCESS: Integrated in Ambari-branch-2.1 #564 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/564/])
AMBARI-13142 UI configs clean up. (ababiichuk) (ababiichuk: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=1f759df794d4f5a8258313e703e7481caac27f70)
* ambari-web/app/controllers/main/admin/security/add/step2.js
* ambari-web/test/controllers/wizard/step8_test.js
* ambari-web/app/mixins/common/configs/configs_saver.js
* ambari-web/test/controllers/main/admin/security_test.js
* ambari-web/app/data/HDP2/ha_properties.js
* ambari-web/app/mixins/wizard/addSecurityConfigs.js
* ambari-web/test/models/configs/objects/service_config_property_test.js
* ambari-web/test/utils/configs/config_property_helper_test.js
* ambari-web/app/controllers/main/admin/serviceAccounts_controller.js
* ambari-web/app/models/configs/objects/service_config_property.js
* ambari-web/app/controllers/main/service/info/configs.js
* ambari-web/app/controllers/wizard/step7_controller.js
* ambari-web/app/data/HDP2/config_mapping.js
* ambari-web/app/views/common/configs/service_configs_by_category_view.js
* ambari-web/test/controllers/wizard/step7_test.js
* ambari-web/test/utils/config_test.js
* ambari-web/test/mixins/wizard/addSeccurityConfigs_test.js
* ambari-web/app/utils/config.js
* ambari-web/test/mock_data_setup/configs_mock_data.js
* ambari-web/app/controllers/wizard/step8_controller.js
* ambari-web/test/controllers/main/admin/security/add/step4_test.js
* ambari-web/test/views/common/configs/service_configs_by_category_view_test.js
* ambari-web/app/data/HDP2/secure_properties.js
* ambari-web/app/controllers/main/admin/security.js
* ambari-web/app/data/BIGTOP/site_properties.js
* ambari-web/app/data/HDP2/rm_ha_properties.js
* ambari-web/test/controllers/main/admin/serviceAccounts_controller_test.js
* ambari-web/app/mixins/common/configs/configs_comparator.js
* ambari-web/app/utils/configs/config_property_helper.js
* ambari-web/test/controllers/main/service/info/config_test.js


> UI configs clean up
> ---
>
> Key: AMBARI-13142
> URL: https://issues.apache.org/jira/browse/AMBARI-13142
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-web
>Affects Versions: 2.1.3
>Reporter: Andrii Babiichuk
>Assignee: Andrii Babiichuk
> Fix For: 2.1.3
>
> Attachments: AMBARI-13142.patch
>
>
> Remove unused code from UI that was used for managing configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38183: Installing falcon with blueprint, oozie extensions are missing, hence causing misconfigured installations that are using blueprint.

2015-09-21 Thread Sandor Magyari



> On Sept. 17, 2015, 1:37 p.m., Robert Nettleton wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java,
> >  line 3687
> > 
> >
> > Could you also add a unit test to verify that a configuration override 
> > of one of these properties in the excluded configuration isn't overwritten 
> > by this change?  
> > 
> > If a Blueprint or cluster creation template sets this property 
> > explicitly, then the customer-defined value should be used, rather than the 
> > stack defaults.  
> > 
> > If you could add a test, or an additional assertion to this test to 
> > verify this, that would be great.

added a new test case, also tested with a manual blueprint install, the patch 
seem to work fine, one can ovveide these properties in bluperint both global & 
host related configurations.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38183/#review99379
---


On Sept. 18, 2015, 9:10 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38183/
> ---
> 
> (Updated Sept. 18, 2015, 9:10 p.m.)
> 
> 
> Review request for Ambari, John Speidel, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-13017
> https://issues.apache.org/jira/browse/AMBARI-13017
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> 
> Installing falcon with blueprint, oozie extensions are missing, hence causing 
> misconfigured installations that are using blueprint.
> 
> SOLUTION
> 
> This issue seems to be related not only for case of falcon but for every 
> service which has a configuration file for another service.
> In case of falcon there is an oozie-site.xml (this contains oozie extensions) 
> in configuration of falcon service, which is marked as exclued-config-type in 
> service metainfo.xml. The same is true for AMBARI_METRICS which includes the 
> storm-site.xml. In case of a bluprint install these are excluded from 
> properties, which is fine because you can not add a config-type twice. The 
> solution would to add these properties from excluded-config-type at update 
> phase of properties in 
> BluprintConfigurationProcessor.doUpdateForClusterCreate.
> We can apply this only specifically for FALCON but I dont see yet any 
> drawbacks of applying it for each service with excluded property.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  892cf32 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  a97ca74 
> 
> Diff: https://reviews.apache.org/r/38183/diff/
> 
> 
> Testing
> ---
> 
> Manually tested, creating new cluster with blueprint, unitests passed, 
> created a new testcase.
> 
> 
> Thanks,
> 
> Sandor Magyari
> 
>

[jira] [Updated] (AMBARI-13167) Ambari fails to install Ranger KMS, "config validation failed" python key traceback


 [ 
https://issues.apache.org/jira/browse/AMBARI-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sekhon updated AMBARI-13167:
-
Attachment: ambari-server.log

> Ambari fails to install Ranger KMS, "config validation failed" python key 
> traceback
> ---
>
> Key: AMBARI-13167
> URL: https://issues.apache.org/jira/browse/AMBARI-13167
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.0
> Environment: HDP 2.3 + Kerberos + Ranger installed
>Reporter: Hari Sekhon
> Attachments: Screen Shot 2015-09-21 at 13.51.37.png, Screen Shot 
> 2015-09-21 at 13.51.47.png, ambari-server.log
>
>
> When trying to install Ranger KMS via Ambari I get an error pop up that 
> config validation failed and the following error:
> {code}Error occured in stack advisor.
> Error details: 'hive.security.authorization.manager'
> 21 Sep 2015 08:52:28,873  INFO [qtp-client-2798] StackAdvisorRunner:71 - 
> advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 154, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 113, 
> in main
> result = stackAdvisor.validateConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 463, in validateConfigurations
> validationItems = self.getConfigurationsValidationItems(services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 348, in getConfigurationsValidationItems
> resultItems = method(siteProperties, siteRecommendations, configurations, 
> services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.3/services/stack_advisor.py",
>  line 231, in validateHiveServer2Configurations
> super(HDP23StackAdvisor, 
> self).validateHiveServer2Configurations(properties, recommendedDefaults, 
> configurations, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.2/services/stack_advisor.py",
>  line 906, in validateHiveServer2Configurations
> if hive_server2[prop_name] != prop_val:
> KeyError: 'hive.security.authorization.manager'
> 21 Sep 2015 08:52:28,874  WARN [qtp-client-2798] AbstractResourceProvider:90 
> - Error occurred during validation
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorException: 
> Stack Advisor reported an error: KeyError: 
> 'hive.security.authorization.manager'{code}
> I've attached a couple of screen shots and the full Ambari log.
> Ambari does actually progress to the next Configure Identities screen 
> regardless however.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13166) Ambari with sqla db fails to start by non root user. Problem with jdbc.


 [ 
https://issues.apache.org/jira/browse/AMBARI-13166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-13166:
---
Attachment: AMBARI-13166.patch

> Ambari with sqla db fails to start by non root user. Problem with jdbc.
> ---
>
> Key: AMBARI-13166
> URL: https://issues.apache.org/jira/browse/AMBARI-13166
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13166.patch
>
>
> {code}
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> sap.jdbc4.sqlanywhere.IDriver
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:103)
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance()
>   at 
> com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
>   at 
> com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
>   at 
> com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
>   at 
> com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
>   at 
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
>   at com.google.inject.Scopes$1$1.get(Scopes.java:65)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-13166) Ambari with sqla db fails to start by non root user. Problem with jdbc.

Vitaly Brodetskyi created AMBARI-13166:
--

 Summary: Ambari with sqla db fails to start by non root user. 
Problem with jdbc.
 Key: AMBARI-13166
 URL: https://issues.apache.org/jira/browse/AMBARI-13166
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 2.1.2
Reporter: Vitaly Brodetskyi
Assignee: Vitaly Brodetskyi
Priority: Critical
 Fix For: 2.1.2


{code}
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
sap.jdbc4.sqlanywhere.IDriver
at 
org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:103)
at 
org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance()
at 
com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
at 
com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
at 
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
at 
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
at 
com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
at 
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
at 
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
at 
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at 
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at 
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
at 
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.Scopes$1$1.get(Scopes.java:65)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13055) [Hive View Visualization] Add more options for max number of rows for reports/charts

2015-09-21 Thread Yusaku Sako (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yusaku Sako updated AMBARI-13055:
-
Fix Version/s: (was: 2.1.2)
   2.1.3.

> [Hive View Visualization] Add more options for max number of rows for 
> reports/charts
> 
>
> Key: AMBARI-13055
> URL: https://issues.apache.org/jira/browse/AMBARI-13055
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-views
>Affects Versions: 2.1.2
>Reporter: Pallav Kulshreshtha
>Assignee: Pallav Kulshreshtha
> Fix For: 2.1.3
>
> Attachments: AMBARI-13055_branch-2.1.patch
>
>
> Go to hive view and run a query. Click "Visualization" tab after getting the 
> query results.
> Problem: All the charts (under data explorer and advance visualization) have 
> plotted for maximum 3 rows (hardcoded)
> Solution: Uses should be given more options for analyzing the data in a 
> better way. 
> These maximum number of rows options can be 1, 3, 5, 8, 
> 10, 13, 15, 18 and 20



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-13176) Ambari agent should reject configured service ports for connections

2015-09-21 Thread Robert Geiger (JIRA)

Robert Geiger created AMBARI-13176:
--

 Summary: Ambari agent should reject configured service ports for 
connections
 Key: AMBARI-13176
 URL: https://issues.apache.org/jira/browse/AMBARI-13176
 Project: Ambari
  Issue Type: Bug
  Components: ambari-agent
Affects Versions: 2.0.0
 Environment: CentOS release 6.7 (Final)
Reporter: Robert Geiger


(may be related to AMBARI-10021 which was marked fixed for 2.0.0)

ambari-agent should reject ports obtained when connecting to the server if the 
conflict with a configured service port.

Below the agent got port 45454 when connecting to the server, but port 45454 is 
configured to be the namenode address port.

$ sudo netstat -a -n -p | grep 45454
tcp0  0 172.31.26.5:45454   172.31.26.4:8441
ESTABLISHED 6706/python2.6  

$ ps aux | grep 6706
root  6706  0.4  0.0 1547504 27260 ?   Sl   Sep21   1:12 
/usr/bin/python2.6 /usr/lib/python2.6/site-packages/ambari_agent/main.py start 
restart --expected-hostname=ip-172-31-26-5.us-west-2.compute.internal

The agent should probably check if a socket open gave a port that is a 
configured service port and try again if it is.

Ambari version is 2.0.0, installed as:

wget 
http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.0.0/ambari.repo
 -O /etc/yum.repos.d/ambari.repo


sudo yum install ambari-server





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38567: Support Host in MM Removal During Upgrade

2015-09-21 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38567/#review99887
---

Ship it!


Ship It!

- Nate Cole


On Sept. 21, 2015, 7:50 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38567/
> ---
> 
> (Updated Sept. 21, 2015, 7:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-13171
> https://issues.apache.org/jira/browse/AMBARI-13171
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, AMBARI-12924 and AMBARI-12889 allow for a host to be 
> placed into Maintenance Mode in order for it to be excluded from the upgrade. 
> Although there is work planned to be able to finalize an upgrade with hosts 
> that are in MM and out-of-sync, in the short term there also needs to be the 
> ability to remove these hosts so that finalization can complete.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/HostRemovedEvent.java
>  e005754 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java
>  0824760 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterEntity.java
>  3fe541f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
>  6ede46b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
>  62613ff 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  eda2534 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  972ceec 
>   
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java
>  b056991 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 
> 53f7432 
> 
> Diff: https://reviews.apache.org/r/38567/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 38586: Ranger Database related property values should be recommended by stack advisor

2015-09-21 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38586/
---

(Updated Sept. 22, 2015, 1:17 a.m.)


Review request for Ambari, Srimanth Gunturi, Sumit Gupta, and Yusaku Sako.


Bugs: AMBARI-13175
https://issues.apache.org/jira/browse/AMBARI-13175


Repository: ambari


Description
---

Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
changed.

Also changing of ${admin-properties/db_host} and ${admin-properties/db_name} 
should change ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.
Please see https://issues.apache.org/jira/browse/AMBARI-13175 for further 
description details


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/admin-properties.xml
 5fdb1b9 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
5568e60 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/configuration/ranger-admin-site.xml
 43be765 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
d7ea10f 
  ambari-web/app/data/db_properties_info.js d48dc98 
  ambari-web/app/views/common/controls_view.js a7bf5a0 

Diff: https://reviews.apache.org/r/38586/diff/


Testing
---

Tested the patch on a cluster.

Verified that all ambari-web unit tests passes:
9602 tests complete (12 seconds)
94 tests pending

Also verified that ambari-server python unit tests are passing successfully:
--
Ran 231 tests in 6.860s
OK
--
Total run:795
Total errors:0
Total failures:0
OK
Process finished with exit code 0


Thanks,

Jaimin Jetly

[jira] [Updated] (AMBARI-13175) Ranger Database related property values should be recommended by stack advisor


 [ 
https://issues.apache.org/jira/browse/AMBARI-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaimin D Jetly updated AMBARI-13175:

Attachment: AMBARI-13175_2.patch

> Ranger Database related property values should be recommended by stack advisor
> --
>
> Key: AMBARI-13175
> URL: https://issues.apache.org/jira/browse/AMBARI-13175
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web, stacks
>Affects Versions: 2.1.3
>Reporter: Jaimin D Jetly
>Assignee: Jaimin D Jetly
> Fix For: 2.1.3
>
> Attachments: AMBARI-13175.patch, AMBARI-13175_2.patch
>
>
> Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
> changed.
> Following properties should be changed accordingly by stack advisor:
> ||DB_FLAVOR value||Dependent properties values||
> |MYSQL|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar 
> \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url=jdbc:mysql://$\{admin-properties/db_host}/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.mysql.jdbc.Driver (HDP-2.3 
> onwards)
> |ORACLE|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:oracle:thin:@/$\{admin-properties/db_host}:1521/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=oracle.jdbc.driver.OracleDriver 
> (HDP-2.3 onwards)
> |POSTGRES|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:postgresql://$\{admin-properties/db_host}:5432/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=org.postgresql.Driver (HDP-2.3 
> onwards)
> |MSSQL|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlserver://$\{admin-properties/db_host};databaseName=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
>  (HDP-2.3 onwards)
> |SQLA|1) 
> admin-properties/SQL_CONNECTOR_JAR=/path_to_driver/sqla-client-jdbc.tar.gz 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlanywhere:host=$\{admin-properties/db_host};database=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=sap.jdbc4.sqlanywhere.IDriver 
> (HDP-2.3 onwards)
> Also changing of $\{admin-properties/db_host} and 
> $\{admin-properties/db_name\} should change 
> ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38254: [Hive View Visualization] Add more options for max number of rows for reports/charts

2015-09-21 Thread Yusaku Sako


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38254/#review99872
---

Ship it!


Ship It!

- Yusaku Sako


On Sept. 18, 2015, 6:03 a.m., Pallav Kulshreshtha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38254/
> ---
> 
> (Updated Sept. 18, 2015, 6:03 a.m.)
> 
> 
> Review request for Ambari, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-13055
> https://issues.apache.org/jira/browse/AMBARI-13055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implemented more options for maximum number of rows for visulization.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/visualization-ui.js
>  5ba22e4 
>   contrib/views/hive/src/main/resources/ui/hive-web/app/styles/app.scss 
> 398a8ef 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/templates/components/visualization-tabs-widget.hbs
>  f46b709 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/templates/visualization-ui.hbs
>  79c0f63 
>   contrib/views/hive/src/main/resources/ui/hive-web/app/utils/constants.js 
> c5d5cf0 
> 
> Diff: https://reviews.apache.org/r/38254/diff/
> 
> 
> Testing
> ---
> 
> Tested manually with different datasets of different size. 
> Tested switching between the two visualization tabs multiple times.
> Tested with Google Chrome, Safari, Firefox and Internet Explorer 10.
> Tested end to end with Ambari Web and Hive View.
> Tested with few options 1, 3, 5, 8, 10 etc rows in the 
> resultset with all possible combinations of scale and chart types.
> Tested with negetive row count
> Tested with alphanumeric row count
> 
> 
> Here is the build log:
> ---
>  T E S T S
> ---
> Running org.apache.ambari.view.hive.backgroundjobs.BackgroundJobControllerTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.364 sec
> Running org.apache.ambari.view.hive.PropertyValidatorTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.076 sec
> Running org.apache.ambari.view.hive.resources.files.FileServiceTest
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.904 sec
> Running org.apache.ambari.view.hive.resources.jobs.AggregatorTest
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.156 sec
> Running org.apache.ambari.view.hive.resources.jobs.ATSParserTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.111 sec
> Running org.apache.ambari.view.hive.resources.jobs.JobServiceTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.642 sec
> Running org.apache.ambari.view.hive.resources.jobs.LogParserTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec
> Running 
> org.apache.ambari.view.hive.resources.resources.FileResourceServiceTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.433 sec
> Running 
> org.apache.ambari.view.hive.resources.savedQueries.SavedQueryResourceManagerTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.095 sec
> Running 
> org.apache.ambari.view.hive.resources.savedQueries.SavedQueryServiceTest
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.146 sec
> Running org.apache.ambari.view.hive.resources.udfs.UDFServiceTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.439 sec
> 
> Results :
> 
> Tests run: 48, Failures: 0, Errors: 0, Skipped: 0
> 
> [INFO]
> [INFO] --- apache-rat-plugin:0.11:check (default) @ hive ---
> [INFO] 51 implicit excludes (use -debug for more details).
> [INFO] Exclude: .git/
> [INFO] Exclude: **/.gitignore
> [INFO] Exclude: **/.gitattributes
> [INFO] Exclude: .idea/
> [INFO] Exclude: pass.txt
> [INFO] Exclude: .DS_Store
> [INFO] Exclude: .iml/
> [INFO] Exclude: .classpath
> [INFO] Exclude: .project
> [INFO] Exclude: .settings
> [INFO] Exclude: **/target/**
> [INFO] Exclude: **/.gitkeep
> [INFO] Exclude: **/.bowerrc
> [INFO] Exclude: **/.editorconfig
> [INFO] Exclude: **/.jshintrc
> [INFO] Exclude: **/.tmp/**
> [INFO] Exclude: **/tmp/**
> [INFO] Exclude: **/*.json
> [INFO] Exclude: **/*.svg
> [INFO] Exclude: **/vendor/**
> [INFO] Exclude: **/bin
> [INFO] Exclude: **/node_modules/**
> [INFO] Exclude: **/vendor/**
> [INFO] Exclude: **/public/**
> [INFO] Exclude: **/dist/**
> [INFO] Exclude: **/bower_components/**
> [INFO] Exclude: **/bower/**
> [INFO] Exclude: **/node/**
> [INFO] Exclude: **/runner.js
> [INFO] Exclude: **/assets/javascripts/**
> [INFO] Exclude: **/assets/stylesheets/**
> [INFO] Exclude: **/assets/static/javascripts/**
> [INFO]

[jira] [Updated] (AMBARI-13055) [Hive View Visualization] Add more options for max number of rows for reports/charts

2015-09-21 Thread Yusaku Sako (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yusaku Sako updated AMBARI-13055:
-
Fix Version/s: (was: 2.1.3.)
   2.1.3

> [Hive View Visualization] Add more options for max number of rows for 
> reports/charts
> 
>
> Key: AMBARI-13055
> URL: https://issues.apache.org/jira/browse/AMBARI-13055
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-views
>Affects Versions: 2.1.2
>Reporter: Pallav Kulshreshtha
>Assignee: Pallav Kulshreshtha
> Fix For: 2.1.3
>
> Attachments: AMBARI-13055_branch-2.1.patch
>
>
> Go to hive view and run a query. Click "Visualization" tab after getting the 
> query results.
> Problem: All the charts (under data explorer and advance visualization) have 
> plotted for maximum 3 rows (hardcoded)
> Solution: Uses should be given more options for analyzing the data in a 
> better way. 
> These maximum number of rows options can be 1, 3, 5, 8, 
> 10, 13, 15, 18 and 20



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13175) Ranger Database related property values should be recommended by stack advisor


 [ 
https://issues.apache.org/jira/browse/AMBARI-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaimin D Jetly updated AMBARI-13175:

Attachment: AMBARI-13175.patch

> Ranger Database related property values should be recommended by stack advisor
> --
>
> Key: AMBARI-13175
> URL: https://issues.apache.org/jira/browse/AMBARI-13175
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web, stacks
>Affects Versions: 2.1.3
>Reporter: Jaimin D Jetly
>Assignee: Jaimin D Jetly
> Fix For: 2.1.3
>
> Attachments: AMBARI-13175.patch
>
>
> Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
> changed.
> Following properties should be changed accordingly by stack advisor:
> ||DB_FLAVOR value||Dependent properties values||
> |MYSQL|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar 
> \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url=jdbc:mysql://$\{admin-properties/db_host}/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.mysql.jdbc.Driver (HDP-2.3 
> onwards)
> |ORACLE|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:oracle:thin:@/$\{admin-properties/db_host}:1521/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=oracle.jdbc.driver.OracleDriver 
> (HDP-2.3 onwards)
> |POSTGRES|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:postgresql://$\{admin-properties/db_host}:5432/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=org.postgresql.Driver (HDP-2.3 
> onwards)
> |MSSQL|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlserver://$\{admin-properties/db_host};databaseName=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
>  (HDP-2.3 onwards)
> |SQLA|1) 
> admin-properties/SQL_CONNECTOR_JAR=/path_to_driver/sqla-client-jdbc.tar.gz 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlanywhere:host=$\{admin-properties/db_host};database=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=sap.jdbc4.sqlanywhere.IDriver 
> (HDP-2.3 onwards)
> Also changing of $\{admin-properties/db_host} and 
> $\{admin-properties/db_name\} should change 
> ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13175) Ranger Database related property values should be recommended by stack advisor


[ 
https://issues.apache.org/jira/browse/AMBARI-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901717#comment-14901717
 ] 

Jaimin D Jetly commented on AMBARI-13175:
-

Also verified that ambari-server python unit tests are passing successfully:
--
Ran 231 tests in 6.860s

OK
--
Total run:795
Total errors:0
Total failures:0
OK

Process finished with exit code 0


> Ranger Database related property values should be recommended by stack advisor
> --
>
> Key: AMBARI-13175
> URL: https://issues.apache.org/jira/browse/AMBARI-13175
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server, ambari-web, stacks
>Affects Versions: 2.1.3
>Reporter: Jaimin D Jetly
>Assignee: Jaimin D Jetly
> Fix For: 2.1.3
>
> Attachments: AMBARI-13175.patch
>
>
> Recommendation API should be called whenever admin-properties/DB_FLAVOR is 
> changed.
> Following properties should be changed accordingly by stack advisor:
> ||DB_FLAVOR value||Dependent properties values||
> |MYSQL|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar 
> \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url=jdbc:mysql://$\{admin-properties/db_host}/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.mysql.jdbc.Driver (HDP-2.3 
> onwards)
> |ORACLE|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:oracle:thin:@/$\{admin-properties/db_host}:1521/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards)\\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=oracle.jdbc.driver.OracleDriver 
> (HDP-2.3 onwards)
> |POSTGRES|1) 
> admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar \\2) 
> ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:postgresql://$\{admin-properties/db_host}:5432/$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=org.postgresql.Driver (HDP-2.3 
> onwards)
> |MSSQL|1) admin-properties/SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlserver://$\{admin-properties/db_host};databaseName=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
>  (HDP-2.3 onwards)
> |SQLA|1) 
> admin-properties/SQL_CONNECTOR_JAR=/path_to_driver/sqla-client-jdbc.tar.gz 
> \\2) ranger-admin-site/ranger.jpa.jdbc.url= 
> jdbc:sqlanywhere:host=$\{admin-properties/db_host};database=$\{admin-properties/db_name\}
>  (HDP-2.3 onwards) \\3) 
> ranger-admin-site/ranger.jpa.jdbc.driver=sap.jdbc4.sqlanywhere.IDriver 
> (HDP-2.3 onwards)
> Also changing of $\{admin-properties/db_host} and 
> $\{admin-properties/db_name\} should change 
> ranger-admin-site/ranger.jpa.jdbc.url for HDP-2.3 stack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38571: Provide Kafka metrics filter to reduce write volume

2015-09-21 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38571/
---

Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Sriharsha 
Chintalapani.


Bugs: AMBARI-13173
https://issues.apache.org/jira/browse/AMBARI-13173


Repository: ambari


Description
---

- Kafka writes 1100 metrics for a topic producing a very high load on the 
metrics system.
- Ganglia metrics reporter provided a regex to filter out metrics that are not 
needed.
- This patch provides configuration knobs to filter unnecessary metrics:
- Exclude list:
{noformat}
kafka.network.RequestMetrics.*
kafka.server.DelayedOperationPurgatory.*
kafka.server.BrokerTopicMetrics.BytesRejectedPerSec.*
{noformat}
- Include list:
{noformat}
kafka.network.RequestMetrics.ResponseQueueTimeMs.request.OffsetCommit.98percentile
kafka.network.RequestMetrics.ResponseQueueTimeMs.request.Offsets.95percentile
kafka.network.RequestMetrics.ResponseSendTimeMs.request.Fetch.95percentile
kafka.network.RequestMetrics.RequestsPerSec.request.*
{noformat}


Diffs
-

  
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
 a259864 
  
ambari-metrics/ambari-metrics-kafka-sink/src/test/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporterTest.java
 67c61e1 
  ambari-metrics/ambari-metrics-storm-sink/pom.xml c666de0 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 36339c5 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
 b97a7f4 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/KAFKA.txt
 228a5bc 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/configuration/kafka-broker.xml
 e621ddf 

Diff: https://reviews.apache.org/r/38571/diff/


Testing
---

Unit tests pass: org.apache.hadoop.metrics2.sink.kafka.*, 
org.apache.ambari.server.upgrade.UpgradeCatalog212
Manual verification in progress


Thanks,

Sid Wagle

[jira] [Created] (AMBARI-13174) METRICS_COLLECTOR is not live messages are flooding the log files

2015-09-21 Thread Istvan Szukacs (JIRA)

Istvan Szukacs created AMBARI-13174:
---

 Summary: METRICS_COLLECTOR is not live messages are flooding the 
log files
 Key: AMBARI-13174
 URL: https://issues.apache.org/jira/browse/AMBARI-13174
 Project: Ambari
  Issue Type: Bug
  Components: ambari-web
Affects Versions: 2.1.1
Reporter: Istvan Szukacs
Priority: Critical


It seems that the any time MetricsPropertyProvider floods the log files for no 
good reason. This should be a DEBUG log instead of an INFO or it should be 
logged once when the METRICS_COLLECTOR gets enabled or disabled.

{code}
21 Sep 2015 21:15:34,697  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,697  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,697  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,698  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,698  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,699  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,699  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,699  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,700  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,700  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,700  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,701  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,701  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,702  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,702  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,702  INFO [qtp-client-551] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
21 Sep 2015 21:15:34,735  INFO [qtp-client-559] MetricsPropertyProvider:525 - 
METRICS_COLLECTOR is not live. Skip populating resources with metrics.
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38571: Provide Kafka metrics filter to reduce write volume

2015-09-21 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38571/
---

(Updated Sept. 21, 2015, 10:14 p.m.)


Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Sriharsha 
Chintalapani.


Bugs: AMBARI-13173
https://issues.apache.org/jira/browse/AMBARI-13173


Repository: ambari


Description
---

- Kafka writes 1100 metrics for a topic producing a very high load on the 
metrics system.
- Ganglia metrics reporter provided a regex to filter out metrics that are not 
needed.
- This patch provides configuration knobs to filter unnecessary metrics:
- Exclude list:
{noformat}
kafka.network.RequestMetrics.*
kafka.server.DelayedOperationPurgatory.*
kafka.server.BrokerTopicMetrics.BytesRejectedPerSec.*
{noformat}
- Include list:
{noformat}
kafka.network.RequestMetrics.ResponseQueueTimeMs.request.OffsetCommit.98percentile
kafka.network.RequestMetrics.ResponseQueueTimeMs.request.Offsets.95percentile
kafka.network.RequestMetrics.ResponseSendTimeMs.request.Fetch.95percentile
kafka.network.RequestMetrics.RequestsPerSec.request.*
{noformat}


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
 a259864 
  
ambari-metrics/ambari-metrics-kafka-sink/src/test/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporterTest.java
 67c61e1 
  
ambari-metrics/ambari-metrics-kafka-sink/src/test/java/org/apache/hadoop/metrics2/sink/kafka/ScheduledReporterTest.java
 41f9126 
  ambari-metrics/ambari-metrics-storm-sink/pom.xml c666de0 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 36339c5 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
 b97a7f4 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/KAFKA.txt
 228a5bc 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/configuration/kafka-broker.xml
 e621ddf 

Diff: https://reviews.apache.org/r/38571/diff/


Testing
---

Unit tests pass: org.apache.hadoop.metrics2.sink.kafka.*, 
org.apache.ambari.server.upgrade.UpgradeCatalog212
Manual verification in progress


Thanks,

Sid Wagle

[jira] [Commented] (AMBARI-13155) Compare config versions doesn't work for Oozie


[ 
https://issues.apache.org/jira/browse/AMBARI-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901511#comment-14901511
 ] 

Hudson commented on AMBARI-13155:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3479 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3479/])
AMBARI-13155. Compare config versions doesn't work for Oozie (rzang) (rzang: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=26da70b3f06b1b2729572a4ac29588a4a17111b5)
* ambari-web/test/views/common/controls_view_test.js
* ambari-web/app/views/common/controls_view.js


> Compare config versions doesn't work for Oozie
> --
>
> Key: AMBARI-13155
> URL: https://issues.apache.org/jira/browse/AMBARI-13155
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Richard Zang
>Assignee: Richard Zang
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13155.patch
>
>
> STR :
> Go to Oozie config tab
> Add/change some property
> Save config
> Compare new configuration with old one
> AR : comparison is not loaded



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13172) Kafka Service check fails with umask 027 on secure cluster


[ 
https://issues.apache.org/jira/browse/AMBARI-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901510#comment-14901510
 ] 

Hudson commented on AMBARI-13172:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3479 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3479/])
AMBARI-13172. Kafka Service check fails with umask 027 on secure cluster. 
(mpapirkovskyy) (mpapyrkovskyy: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=2d12f1edec147f317f55345531b200e5d4f7d9fe)
* 
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/service_check.py


> Kafka Service check fails with umask 027 on secure cluster
> --
>
> Key: AMBARI-13172
> URL: https://issues.apache.org/jira/browse/AMBARI-13172
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Myroslav Papirkovskyy
>Assignee: Myroslav Papirkovskyy
>Priority: Blocker
> Fix For: 2.1.2
>
>
> Kafka service check failed with exception:
> {noformat}
> [2015-09-21 18:16:34,906] WARN SASL configuration failed: 
> javax.security.auth.login.LoginException: Zookeeper client cannot 
> authenticate using the 'Client' section of the supplied JAAS configuration: 
> '/usr/hdp/current/kafka-broker/config/kafka_jaas.conf' because of a 
> RuntimeException: java.lang.SecurityException: Configuration Error:
>   No such file or directory Will continue connection to Zookeeper server 
> without SASL authentication, if Zookeeper server allows it. 
> (org.apache.zookeeper.ClientCnxn)
> Error while executing topic command null
> java.lang.ExceptionInInitializerError
>   at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:231)
>   at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:92)
>   at kafka.admin.TopicCommand$.main(TopicCommand.scala:54)
>   at kafka.admin.TopicCommand.main(TopicCommand.scala)
> Caused by: java.lang.RuntimeException: File 
> /usr/hdp/current/kafka-broker/config/kafka_jaas.conf cannot be read.
>   at kafka.utils.ZkUtils$.(ZkUtils.scala:65)
>   at kafka.utils.ZkUtils$.(ZkUtils.scala)
>   ... 4 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13094) Add Spark Thrift Ambari Service

2015-09-21 Thread Sumit Mohanty (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-13094:
---
Attachment: AMBARI-13094-2.1.2.patch

> Add Spark Thrift Ambari Service
> ---
>
> Key: AMBARI-13094
> URL: https://issues.apache.org/jira/browse/AMBARI-13094
> Project: Ambari
>  Issue Type: New Feature
>Affects Versions: trunk
>Reporter: Judy Nash
>Assignee: Judy Nash
>  Labels: patch
> Fix For: trunk
>
> Attachments: AMBARI-13094-2.1.2.patch, AMBARI-13094.patch, Ambari 
> Service for Spark Thrift Design Specification.docx
>
>
> New feature to add spark thrift server support on Ambari. 
> Design specification attached. 
> Instruction to add thrift server to an existing cluster:
> 1) If running on HDP distro, Update metainfo.xml @ 
> /var/lib/ambari-server/resources/stacks/HDP/2.3/services/SPARK/metainfo.xml. 
> Change true to false. 
> 2) Use Ambari UI if spark has not been installed before. It will show up as a 
> installable service alongside with spark history server. 
> 3) If spark component has been installed already, use API to add 
> thrift-server as a new service. Service name: SPARK_THRIFTSERVER



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38557: One of kafka brokers failed to start

2015-09-21 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38557/#review99751
---

Ship it!


Ship It!

- Dmytro Sen


On Сен. 21, 2015, 3:17 п.п., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38557/
> ---
> 
> (Updated Сен. 21, 2015, 3:17 п.п.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-13169
> https://issues.apache.org/jira/browse/AMBARI-13169
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After second try, it started successfully.
> {code}
> [2015-09-21 06:22:20,916] FATAL Fatal error during KafkaServerStartable 
> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to 
> zookeeper server within timeout: 15000
>   at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:143)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:121)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:88)
>   at 
> kafka.security.auth.SimpleAclAuthorizer.initialize(SimpleAclAuthorizer.scala:70)
>   at kafka.server.KafkaServer.startup(KafkaServer.scala:159)
>   at 
> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
>   at kafka.Kafka$.main(Kafka.scala:72)
>   at kafka.Kafka.main(Kafka.scala)
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml
>  255707a 
> 
> Diff: https://reviews.apache.org/r/38557/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

[jira] [Commented] (AMBARI-13147) Heatmaps page: page hangs for a while on HDFS metrics


[ 
https://issues.apache.org/jira/browse/AMBARI-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900808#comment-14900808
 ] 

Hudson commented on AMBARI-13147:
-

ABORTED: Integrated in Ambari-branch-2.1 #566 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/566/])
AMBARI-13147 Heatmaps page: page hangs for a while on HDFS metrics. (atkach) 
(atkach: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=3ad59b7ee29ac563c4b3966c1b7dba01be2127c0)
* ambari-web/app/mixins/common/widgets/widget_mixin.js
* ambari-web/test/mixins/common/widget_mixin_test.js
* ambari-web/app/views/common/widget/heatmap_widget_view.js


> Heatmaps page: page hangs for a while on HDFS metrics
> -
>
> Key: AMBARI-13147
> URL: https://issues.apache.org/jira/browse/AMBARI-13147
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Fix For: 2.1.3
>
> Attachments: AMBARI-13147.patch
>
>
> *Steps to reproduce:*
> 1. Go to Heatmaps page
> 2. Choose DataNode Process Network I/O Utilization metric
> JS execution takes 11 seconds. (400-node cluster)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13169) One of kafka brokers failed to start


 [ 
https://issues.apache.org/jira/browse/AMBARI-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-13169:
---
Attachment: AMBARI-13169.patch

> One of kafka brokers failed to start
> 
>
> Key: AMBARI-13169
> URL: https://issues.apache.org/jira/browse/AMBARI-13169
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13169.patch
>
>
> After second try, it started successfully.
> {code}
> [2015-09-21 06:22:20,916] FATAL Fatal error during KafkaServerStartable 
> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to 
> zookeeper server within timeout: 15000
>   at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:143)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:121)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:88)
>   at 
> kafka.security.auth.SimpleAclAuthorizer.initialize(SimpleAclAuthorizer.scala:70)
>   at kafka.server.KafkaServer.startup(KafkaServer.scala:159)
>   at 
> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
>   at kafka.Kafka$.main(Kafka.scala:72)
>   at kafka.Kafka.main(Kafka.scala)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13147) Heatmaps page: page hangs for a while on HDFS metrics


[ 
https://issues.apache.org/jira/browse/AMBARI-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900747#comment-14900747
 ] 

Hudson commented on AMBARI-13147:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3475 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3475/])
AMBARI-13147 Heatmaps page: page hangs for a while on HDFS metrics. (atkach) 
(atkach: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=1d292fbbacd79e42ade101f957c1ee680ac680ed)
* ambari-web/app/mixins/common/widgets/widget_mixin.js
* ambari-web/test/mixins/common/widget_mixin_test.js
* ambari-web/app/views/common/widget/heatmap_widget_view.js


> Heatmaps page: page hangs for a while on HDFS metrics
> -
>
> Key: AMBARI-13147
> URL: https://issues.apache.org/jira/browse/AMBARI-13147
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Fix For: 2.1.3
>
> Attachments: AMBARI-13147.patch
>
>
> *Steps to reproduce:*
> 1. Go to Heatmaps page
> 2. Choose DataNode Process Network I/O Utilization metric
> JS execution takes 11 seconds. (400-node cluster)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38495: [Upgrade] ambari server schema upgrade failed for Path 1.6.1 -> 2.1.2 and 2.0.2->2.1.2 and 2.1.1->2.1.2

2015-09-21 Thread Myroslav Papirkovskyy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38495/#review99749
---

Ship it!


Ship It!

- Myroslav Papirkovskyy


On Вер. 21, 2015, 5:37 після полудня, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38495/
> ---
> 
> (Updated Вер. 21, 2015, 5:37 після полудня)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and 
> Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13122
> https://issues.apache.org/jira/browse/AMBARI-13122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> Upgrade Path was 1.6.1 to 2.1.2.
> root@os-r6-ambari-upgrade-oracle-5 ~]# ambari-server upgrade
> Using python /usr/bin/python2.6
> Upgrading ambari-server
> Updating properties in ambari.properties ...
> WARNING: Can not find ambari.properties.rpmsave file from previous version, 
> skipping import of settings
> WARNING: Can not find ambari-env.sh.rpmsave file from previous version, 
> skipping restore of environment settings
> Fixing database objects owner
> Ambari Server configured for Oracle. Confirm you have made a backup of the 
> Ambari Server database y/n ? y
> Upgrading database schema
> ERROR: Error executing schema upgrade, please check the server logs.
> ERROR: Ambari server upgrade failed. Please look at 
> /var/log/ambari-server/ambari-server.log, for more details.
> ERROR: Exiting with exit code 11.
> REASON: Schema upgrade failed.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/dbConfiguration_linux.py 
> 328e53e 
>   ambari-server/src/main/python/ambari_server/serverUpgrade.py 73961d8 
>   ambari-server/src/test/python/TestAmbariServer.py fe6380f 
> 
> Diff: https://reviews.apache.org/r/38495/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 38559: [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38559/
---

Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, 
Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-13152
https://issues.apache.org/jira/browse/AMBARI-13152


Repository: ambari


Description
---

Current issue is caused by commit AMBARI-13107 ("After successful install of 
Oozie via blueprint, oozie has an invalid configuration") . Mentioned commit 
removes few "value" sections from service metainfo. That's why some properties 
in stack metainfo may have null values.

For now, we are going to revert an offending patch from 2.1.2, so probably 
current patch if going to go only to branch-2.1 or to trunk


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java
 9a2c604 

Diff: https://reviews.apache.org/r/38559/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Views .. SUCCESS [3.262s]
[INFO] Ambari Metrics Common . SUCCESS [1.659s]
[INFO] Ambari Server . SUCCESS [53:43.587s]
[INFO] Ambari Agent .. SUCCESS [9.281s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 53:58.376s
[INFO] Finished at: Mon Sep 21 18:54:27 EEST 2015
[INFO] Final Memory: 73M/1692M


Thanks,

Dmitro Lisnichenko

Re: Review Request 38559: [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38559/
---

(Updated Sept. 21, 2015, 4:41 p.m.)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, 
Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-13152
https://issues.apache.org/jira/browse/AMBARI-13152


Repository: ambari


Description (updated)
---

Current issue is caused by commit AMBARI-13107 ("After successful install of 
Oozie via blueprint, oozie has an invalid configuration") . Mentioned commit 
removes few "value" sections from service metainfo. That's why some properties 
in stack metainfo may have null values.

For now, we are going to revert an offending patch from 2.1.2, so probably 
current patch if going to go only to branch-2.1 and to trunk


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java
 9a2c604 

Diff: https://reviews.apache.org/r/38559/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Views .. SUCCESS [3.262s]
[INFO] Ambari Metrics Common . SUCCESS [1.659s]
[INFO] Ambari Server . SUCCESS [53:43.587s]
[INFO] Ambari Agent .. SUCCESS [9.281s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 53:58.376s
[INFO] Finished at: Mon Sep 21 18:54:27 EEST 2015
[INFO] Final Memory: 73M/1692M


Thanks,

Dmitro Lisnichenko

Re: Review Request 38559: [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38559/
---

(Updated Sept. 21, 2015, 5:19 p.m.)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, 
Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-13152
https://issues.apache.org/jira/browse/AMBARI-13152


Repository: ambari


Description (updated)
---

Current issue is caused by commit AMBARI-13107 ("After successful install of 
Oozie via blueprint, oozie has an invalid configuration") . Mentioned commit 
removes few "value" sections from service metainfo. That's why some properties 
in stack metainfo may have null values. 

A litle more details about the case I'm fixing:
Both previous stack value and next stack value are null (not defined in 
metainfo to avoid parsing by blueprint). But runtime property value after 
upgrade is not-null (added by UpgradeCatalog). I'm fixing exactly this case, 
other cases (only new or old stack property value is null and other is not) are 
a way different because of bug nature.

UPD we decided to commit the fix to 2.1.2 as well since mentioned patch is hard 
to revert (other patches are based on it)


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java
 9a2c604 

Diff: https://reviews.apache.org/r/38559/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Views .. SUCCESS [3.262s]
[INFO] Ambari Metrics Common . SUCCESS [1.659s]
[INFO] Ambari Server . SUCCESS [53:43.587s]
[INFO] Ambari Agent .. SUCCESS [9.281s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 53:58.376s
[INFO] Finished at: Mon Sep 21 18:54:27 EEST 2015
[INFO] Final Memory: 73M/1692M


Thanks,

Dmitro Lisnichenko

Re: Review Request 38559: [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log

2015-09-21 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38559/#review99774
---

Ship it!


Ship It!

- Nate Cole


On Sept. 21, 2015, 1:19 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38559/
> ---
> 
> (Updated Sept. 21, 2015, 1:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin 
> Jetly, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-13152
> https://issues.apache.org/jira/browse/AMBARI-13152
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Current issue is caused by commit AMBARI-13107 ("After successful install of 
> Oozie via blueprint, oozie has an invalid configuration") . Mentioned commit 
> removes few "value" sections from service metainfo. That's why some 
> properties in stack metainfo may have null values. 
> 
> A litle more details about the case I'm fixing:
> Both previous stack value and next stack value are null (not defined in 
> metainfo to avoid parsing by blueprint). But runtime property value after 
> upgrade is not-null (added by UpgradeCatalog). I'm fixing exactly this case, 
> other cases (only new or old stack property value is null and other is not) 
> are a way different because of bug nature.
> 
> UPD we decided to commit the fix to 2.1.2 as well since mentioned patch is 
> hard to revert (other patches are based on it)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java
>  9a2c604 
> 
> Diff: https://reviews.apache.org/r/38559/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Ambari Views .. SUCCESS [3.262s]
> [INFO] Ambari Metrics Common . SUCCESS [1.659s]
> [INFO] Ambari Server . SUCCESS 
> [53:43.587s]
> [INFO] Ambari Agent .. SUCCESS [9.281s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 53:58.376s
> [INFO] Finished at: Mon Sep 21 18:54:27 EEST 2015
> [INFO] Final Memory: 73M/1692M
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

[jira] [Commented] (AMBARI-13166) Ambari with sqla db fails to start by non root user. Problem with jdbc.


[ 
https://issues.apache.org/jira/browse/AMBARI-13166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901069#comment-14901069
 ] 

Hudson commented on AMBARI-13166:
-

ABORTED: Integrated in Ambari-branch-2.1 #567 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/567/])
AMBARI-13166. Ambari with sqla db fails to start by non root user. Problem with 
jdbc.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=eb9188575f81e331d20315b2fa6b5d29cfc25c62)
* ambari-server/src/main/python/ambari_server/dbConfiguration.py


> Ambari with sqla db fails to start by non root user. Problem with jdbc.
> ---
>
> Key: AMBARI-13166
> URL: https://issues.apache.org/jira/browse/AMBARI-13166
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13166.patch
>
>
> {code}
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> sap.jdbc4.sqlanywhere.IDriver
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:103)
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance()
>   at 
> com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
>   at 
> com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
>   at 
> com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
>   at 
> com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
>   at 
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
>   at com.google.inject.Scopes$1$1.get(Scopes.java:65)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (AMBARI-13152) [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log

2015-09-21 Thread Dmitry Lysnichenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko resolved AMBARI-13152.
-
Resolution: Fixed

Committed

{code}
   c933395..6a17c33  branch-2.1 -> branch-2.1
   e3b12d2..a0ea100  branch-2.1.2 -> branch-2.1.2
   4e39212..d278ed1  trunk -> trunk
{code}

> [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in 
> ambari-server log
> ---
>
> Key: AMBARI-13152
> URL: https://issues.apache.org/jira/browse/AMBARI-13152
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-upgrade
>Affects Versions: 2.1.2
> Environment: Os - Centos6
> Ambari database - Oracle 12
> Hive/Oozie database - Oracle 12
> Ambari Upgrade Path - 2.0.2 -> 2.1.2
>Reporter: Pankaj Singh
>Priority: Critical
>
> Completed ambari server and ambari database upgrade.
> Traversed to versions tab of stack and versions through ambari 
> Sample url - http://:8080/#/main/admin/stack/versions
> Through manage version added HDP-2.3.2.2874 to the stack.
> At same time saw that "18 Sep 2015 18:19:01,981 ERROR qtp-client-24 
> CheckHelper:72 - Check CONFIG_MERGE failed" message started appearing in 
> ambari-server.log
> The complete stack trace
> 18 Sep 2015 18:18:58,385 INFO qtp-client-22 PersistKeyValueService:82 - 
> Looking for keyName CLUSTER_CURRENT_STATUS
> 18 Sep 2015 18:19:01,981 ERROR qtp-client-24 CheckHelper:72 - Check 
> CONFIG_MERGE failed
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.state.ConfigMergeHelper.getConflicts(ConfigMergeHelper.java:131)
> at 
> org.apache.ambari.server.checks.ConfigurationMergeCheck.perform(ConfigurationMergeCheck.java:94)
> at 
> org.apache.ambari.server.state.CheckHelper.performChecks(CheckHelper.java:61)
> at 
> org.apache.ambari.server.controller.internal.PreUpgradeCheckResourceProvider.getResources(PreUpgradeCheckResourceProvider.java:140)
> at 
> org.apache.ambari.server.controller.internal.ClusterControllerImpl$ExtendedResourceProviderWrapper.queryForResources(ClusterControllerImpl.java:945)
> at 
> org.apache.ambari.server.controller.internal.ClusterControllerImpl.getResources(ClusterControllerImpl.java:132)
> at org.apache.ambari.server.api.query.QueryImpl.doQuery(QueryImpl.java:508)
> at 
> org.apache.ambari.server.api.query.QueryImpl.queryForResources(QueryImpl.java:380)
> at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:216)
> at 
> org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:68)
> at 
> org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:135)
> at 
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:105)
> at 
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:74)
> at 
> org.apache.ambari.server.api.services.PreUpgradeCheckService.getPreUpgradeChecks(PreUpgradeCheckService.java:62)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at 
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at 
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at 
>

Re: Review Request 38559: [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log

2015-09-21 Thread Mahadev Konar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38559/#review99769
---

Ship it!


Ship It!

- Mahadev Konar


On Sept. 21, 2015, 5:19 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38559/
> ---
> 
> (Updated Sept. 21, 2015, 5:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin 
> Jetly, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-13152
> https://issues.apache.org/jira/browse/AMBARI-13152
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Current issue is caused by commit AMBARI-13107 ("After successful install of 
> Oozie via blueprint, oozie has an invalid configuration") . Mentioned commit 
> removes few "value" sections from service metainfo. That's why some 
> properties in stack metainfo may have null values. 
> 
> A litle more details about the case I'm fixing:
> Both previous stack value and next stack value are null (not defined in 
> metainfo to avoid parsing by blueprint). But runtime property value after 
> upgrade is not-null (added by UpgradeCatalog). I'm fixing exactly this case, 
> other cases (only new or old stack property value is null and other is not) 
> are a way different because of bug nature.
> 
> UPD we decided to commit the fix to 2.1.2 as well since mentioned patch is 
> hard to revert (other patches are based on it)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java
>  9a2c604 
> 
> Diff: https://reviews.apache.org/r/38559/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Ambari Views .. SUCCESS [3.262s]
> [INFO] Ambari Metrics Common . SUCCESS [1.659s]
> [INFO] Ambari Server . SUCCESS 
> [53:43.587s]
> [INFO] Ambari Agent .. SUCCESS [9.281s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 53:58.376s
> [INFO] Finished at: Mon Sep 21 18:54:27 EEST 2015
> [INFO] Final Memory: 73M/1692M
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

[jira] [Commented] (AMBARI-13036) Enable JPA statement caching (internal/external)

2015-09-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/AMBARI-13036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901033#comment-14901033
 ] 

Hadoop QA commented on AMBARI-13036:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12761409/AMBARI-13036.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
ambari-server.

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3829//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3829//console

This message is automatically generated.

> Enable JPA statement caching (internal/external)
> 
>
> Key: AMBARI-13036
> URL: https://issues.apache.org/jira/browse/AMBARI-13036
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: 2.1.1
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
> Fix For: 2.2.0
>
> Attachments: AMBARI-13036.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13166) Ambari with sqla db fails to start by non root user. Problem with jdbc.


[ 
https://issues.apache.org/jira/browse/AMBARI-13166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900935#comment-14900935
 ] 

Hudson commented on AMBARI-13166:
-

FAILURE: Integrated in Ambari-trunk-Commit #3476 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3476/])
AMBARI-13166. Ambari with sqla db fails to start by non root user. Problem with 
jdbc.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=cbd8ec16475b8d6263b238026d48a4112578b581)
* ambari-server/src/main/python/ambari_server/dbConfiguration.py


> Ambari with sqla db fails to start by non root user. Problem with jdbc.
> ---
>
> Key: AMBARI-13166
> URL: https://issues.apache.org/jira/browse/AMBARI-13166
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13166.patch
>
>
> {code}
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> sap.jdbc4.sqlanywhere.IDriver
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:103)
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance()
>   at 
> com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
>   at 
> com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
>   at 
> com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
>   at 
> com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
>   at 
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
>   at com.google.inject.Scopes$1$1.get(Scopes.java:65)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-13170) One of the nodemanagers failed to start on Ubuntu14+secure+ranger cluster

2015-09-21 Thread Andrew Onischuk (JIRA)

Andrew Onischuk created AMBARI-13170:


 Summary: One of the nodemanagers failed to start on 
Ubuntu14+secure+ranger cluster
 Key: AMBARI-13170
 URL: https://issues.apache.org/jira/browse/AMBARI-13170
 Project: Ambari
  Issue Type: Bug
Reporter: Andrew Onischuk
Assignee: Andrew Onischuk
 Fix For: 2.1.2


In Ubuntu14+secure+ranger cluster, one of the nodemanager (72.22.74.87) failed
with below error.  
The node manager was dead before test could start.

The nodemanager pid is present on the machine




root@os-u14-qubazs-tez-2-3:~# ps aux | grep nodemanager
yarn  4029  0.0  0.9 1639500 162764 ?  Sl   Sep17   0:31 
/usr/lib/jvm/java-7-openjdk-amd64/bin/java -Dproc_nodemanager -Xmx1024m 
-Dhdp.version=2.3.2.0-2862 -Dhadoop.log.dir=/grid/0/log/hadoop/yarn 
-Dyarn.log.dir=/grid/0/log/hadoop/yarn 
-Dhadoop.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.home.dir= -Dyarn.id.str=yarn -Dhadoop.root.logger=INFO,EWMA,RFA 
-Dyarn.root.logger=INFO,EWMA,RFA 
-Djava.library.path=:/usr/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
 -Dyarn.policy.file=hadoop-policy.xml 
-Djava.io.tmpdir=/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir -server 
-Dhadoop.log.dir=/grid/0/log/hadoop/yarn -Dyarn.log.dir=/grid/0/log/hadoop/yarn 
-Dhadoop.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.home.dir=/usr/hdp/current/hadoop-yarn-nodemanager 
-Dhadoop.home.dir=/grid/0/hdp/2.3.2.0-2862/hadoop 
-Dhadoop.root.logger=INFO,EWMA,RFA -Dyarn.root.logger=INFO,EWMA,RFA 
-Djava.library.path=:/usr/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
 -classpath 
/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/conf:/usr/hdp/2.3.2.0-2862/hadoop/lib/*:/usr/hdp/2.3.2.0-2862/hadoop/.//*:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/./:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/.//*:/usr/hdp/2.3.2.0-2862/hadoop-yarn/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-yarn/.//*:/usr/hdp/2.3.2.0-2862/hadoop-mapreduce/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-mapreduce/.//*:::/usr/share/java/jdbc-mysql.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/java/mysql-jdbc-qe.jar:/usr/hdp/2.3.2.0-2862/tez/*:/usr/hdp/2.3.2.0-2862/tez/lib/*:/usr/hdp/2.3.2.0-2862/tez/conf::/usr/share/java/jdbc-mysql.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/java/mysql-jdbc-qe.jar:/usr/hdp/2.3.2.0-2862/tez/*:/usr/hdp/2.3.2.0-2862/tez/lib/*:/usr/hdp/2.3.2.0-2862/tez/conf:/usr/hdp/current/hadoop-yarn-nodemanager/.//*:/usr/hdp/current/hadoop-yarn-nodemanager/lib/*:/usr/hdp/current/hadoop-client/conf/nm-config/log4j.properties
 org.apache.hadoop.yarn.server.nodemanager.NodeManager


**nodemanager log**



STARTUP_MSG:   build = g...@github.com:hortonworks/hadoop.git -r 
b3f6bbba22d986059b77f46a96a523d6db679d50; compiled by 'jenkins' on 
2015-09-17T11:58Z
STARTUP_MSG:   java = 1.7.0_79
/
2015-09-17 23:09:23,120 INFO  nodemanager.NodeManager 
(LogAdapter.java:info(45)) - registered UNIX signal handlers for [TERM, HUP, 
INT]
2015-09-17 23:09:25,804 FATAL conf.Configuration 
(Configuration.java:loadResource(2644)) - error parsing conf yarn-site.xml
org.xml.sax.SAXParseException; Premature end of file.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at 
org.apache.hadoop.conf.Configuration.parse(Configuration.java:2480)
at 
org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
at 
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2539)
at 
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2492)
at 
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1143)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1115)
at 
org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1451)
at

Review Request 38560: One of the nodemanagers failed to start on Ubuntu14+secure+ranger cluster

2015-09-21 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38560/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-13170
https://issues.apache.org/jira/browse/AMBARI-13170


Repository: ambari


Description
---

In Ubuntu14+secure+ranger cluster, one of the nodemanager (72.22.74.87) failed
with below error.  
The node manager was dead before test could start.

The nodemanager pid is present on the machine




root@os-u14-qubazs-tez-2-3:~# ps aux | grep nodemanager
yarn  4029  0.0  0.9 1639500 162764 ?  Sl   Sep17   0:31 
/usr/lib/jvm/java-7-openjdk-amd64/bin/java -Dproc_nodemanager -Xmx1024m 
-Dhdp.version=2.3.2.0-2862 -Dhadoop.log.dir=/grid/0/log/hadoop/yarn 
-Dyarn.log.dir=/grid/0/log/hadoop/yarn 
-Dhadoop.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.home.dir= -Dyarn.id.str=yarn -Dhadoop.root.logger=INFO,EWMA,RFA 
-Dyarn.root.logger=INFO,EWMA,RFA 
-Djava.library.path=:/usr/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
 -Dyarn.policy.file=hadoop-policy.xml 
-Djava.io.tmpdir=/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir -server 
-Dhadoop.log.dir=/grid/0/log/hadoop/yarn -Dyarn.log.dir=/grid/0/log/hadoop/yarn 
-Dhadoop.l
 og.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.home.dir=/usr/hdp/current/hadoop-yarn-nodemanager 
-Dhadoop.home.dir=/grid/0/hdp/2.3.2.0-2862/hadoop 
-Dhadoop.root.logger=INFO,EWMA,RFA -Dyarn.root.logger=INFO,EWMA,RFA 
-Djava.library.path=:/usr/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
 -classpath 
/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/conf:/usr/hdp/2.3.2.0-2862/hadoop/lib/*:/usr/hdp/2.3.2.0-2862/hadoop/.//*:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/./:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/.//*:/usr/hdp/2.3.2.0-2862/hadoop-yarn/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-yarn/.//*:/us
 
r/hdp/2.3.2.0-2862/hadoop-mapreduce/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-mapreduce/.//*:::/usr/share/java/jdbc-mysql.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/java/mysql-jdbc-qe.jar:/usr/hdp/2.3.2.0-2862/tez/*:/usr/hdp/2.3.2.0-2862/tez/lib/*:/usr/hdp/2.3.2.0-2862/tez/conf::/usr/share/java/jdbc-mysql.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/java/mysql-jdbc-qe.jar:/usr/hdp/2.3.2.0-2862/tez/*:/usr/hdp/2.3.2.0-2862/tez/lib/*:/usr/hdp/2.3.2.0-2862/tez/conf:/usr/hdp/current/hadoop-yarn-nodemanager/.//*:/usr/hdp/current/hadoop-yarn-nodemanager/lib/*:/usr/hdp/current/hadoop-client/conf/nm-config/log4j.properties
 org.apache.hadoop.yarn.server.nodemanager.NodeManager


**nodemanager log**



STARTUP_MSG:   build = g...@github.com:hortonworks/hadoop.git -r 
b3f6bbba22d986059b77f46a96a523d6db679d50; compiled by 'jenkins' on 
2015-09-17T11:58Z
STARTUP_MSG:   java = 1.7.0_79
/
2015-09-17 23:09:23,120 INFO  nodemanager.NodeManager 
(LogAdapter.java:info(45)) - registered UNIX signal handlers for [TERM, HUP, 
INT]
2015-09-17 23:09:25,804 FATAL conf.Configuration 
(Configuration.java:loadResource(2644)) - error parsing conf yarn-site.xml
org.xml.sax.SAXParseException; Premature end of file.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at 
org.apache.hadoop.conf.Configuration.parse(Configuration.java:2480)
at 
org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
at 
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2539)
at 
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2492)
at 
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1143)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1115)
at 
org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1451)
at

Re: Review Request 38557: One of kafka brokers failed to start


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38557/#review99763
---

Ship it!


Ship It!

- Dmitro Lisnichenko


On Sept. 21, 2015, 3:17 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38557/
> ---
> 
> (Updated Sept. 21, 2015, 3:17 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-13169
> https://issues.apache.org/jira/browse/AMBARI-13169
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After second try, it started successfully.
> {code}
> [2015-09-21 06:22:20,916] FATAL Fatal error during KafkaServerStartable 
> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to 
> zookeeper server within timeout: 15000
>   at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:143)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:121)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:88)
>   at 
> kafka.security.auth.SimpleAclAuthorizer.initialize(SimpleAclAuthorizer.scala:70)
>   at kafka.server.KafkaServer.startup(KafkaServer.scala:159)
>   at 
> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
>   at kafka.Kafka$.main(Kafka.scala:72)
>   at kafka.Kafka.main(Kafka.scala)
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml
>  255707a 
> 
> Diff: https://reviews.apache.org/r/38557/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 38561: AMBARI-13154: Label of required fields in Details section of Create View Instance UI should include an asterisk

2015-09-21 Thread Sangeeta Ravindran


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38561/
---

Review request for Ambari and Alexandr Antonenko.


Bugs: AMBARI-13154
https://issues.apache.org/jira/browse/AMBARI-13154


Repository: ambari


Description
---

When creating a view instance of any type of view, the instance name, display 
name and description
need to be entered. These fields are mandatory, but are not marked with an 
asterisk.

Since the fields under Cluster Configuration are marked with an asterisk if 
they are mandatory,
for consistency, the mandatory fields under Details section also be marked with 
an *.


Diffs
-

  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html 
dffd66e 

Diff: https://reviews.apache.org/r/38561/diff/


Testing
---

Fix involves minor change in the string for labels. Manually tested the fix on 
trunk.


Thanks,

Sangeeta Ravindran

Re: Review Request 38559: [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38559/
---

(Updated Sept. 21, 2015, 5:11 p.m.)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, 
Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-13152
https://issues.apache.org/jira/browse/AMBARI-13152


Repository: ambari


Description (updated)
---

Current issue is caused by commit AMBARI-13107 ("After successful install of 
Oozie via blueprint, oozie has an invalid configuration") . Mentioned commit 
removes few "value" sections from service metainfo. That's why some properties 
in stack metainfo may have null values.

UPD we decided to commit the fix to 2.1.2 as well since mentioned patch is hard 
to revert (other patches are based on it)


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java
 9a2c604 

Diff: https://reviews.apache.org/r/38559/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Views .. SUCCESS [3.262s]
[INFO] Ambari Metrics Common . SUCCESS [1.659s]
[INFO] Ambari Server . SUCCESS [53:43.587s]
[INFO] Ambari Agent .. SUCCESS [9.281s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 53:58.376s
[INFO] Finished at: Mon Sep 21 18:54:27 EEST 2015
[INFO] Final Memory: 73M/1692M


Thanks,

Dmitro Lisnichenko

Re: Review Request 38560: One of the nodemanagers failed to start on Ubuntu14+secure+ranger cluster


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38560/#review99765
---

Ship it!


Ship It!

- Vitalyi Brodetskyi


On Вер. 21, 2015, 4:59 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38560/
> ---
> 
> (Updated Вер. 21, 2015, 4:59 після полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13170
> https://issues.apache.org/jira/browse/AMBARI-13170
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ubuntu14+secure+ranger cluster, one of the nodemanager (72.22.74.87) failed
> with below error.  
> The node manager was dead before test could start.
> 
> The nodemanager pid is present on the machine
> 
> 
> 
> 
> root@os-u14-qubazs-tez-2-3:~# ps aux | grep nodemanager
> yarn  4029  0.0  0.9 1639500 162764 ?  Sl   Sep17   0:31 
> /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Dproc_nodemanager -Xmx1024m 
> -Dhdp.version=2.3.2.0-2862 -Dhadoop.log.dir=/grid/0/log/hadoop/yarn 
> -Dyarn.log.dir=/grid/0/log/hadoop/yarn 
> -Dhadoop.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
> -Dyarn.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
> -Dyarn.home.dir= -Dyarn.id.str=yarn -Dhadoop.root.logger=INFO,EWMA,RFA 
> -Dyarn.root.logger=INFO,EWMA,RFA 
> -Djava.library.path=:/usr/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
>  -Dyarn.policy.file=hadoop-policy.xml 
> -Djava.io.tmpdir=/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir -server 
> -Dhadoop.log.dir=/grid/0/log/hadoop/yarn 
> -Dyarn.log.dir=/grid/0/log/hadoop/yarn -Dhadoop
 .log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.log.file=yarn-yarn-nodemanager-os-u14-qubazs-tez-2-3.log 
-Dyarn.home.dir=/usr/hdp/current/hadoop-yarn-nodemanager 
-Dhadoop.home.dir=/grid/0/hdp/2.3.2.0-2862/hadoop 
-Dhadoop.root.logger=INFO,EWMA,RFA -Dyarn.root.logger=INFO,EWMA,RFA 
-Djava.library.path=:/usr/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.2.0-2862/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir
 -classpath 
/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/conf:/usr/hdp/2.3.2.0-2862/hadoop/lib/*:/usr/hdp/2.3.2.0-2862/hadoop/.//*:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/./:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-hdfs/.//*:/usr/hdp/2.3.2.0-2862/hadoop-yarn/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-yarn/.//*:/
 
usr/hdp/2.3.2.0-2862/hadoop-mapreduce/lib/*:/usr/hdp/2.3.2.0-2862/hadoop-mapreduce/.//*:::/usr/share/java/jdbc-mysql.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/java/mysql-jdbc-qe.jar:/usr/hdp/2.3.2.0-2862/tez/*:/usr/hdp/2.3.2.0-2862/tez/lib/*:/usr/hdp/2.3.2.0-2862/tez/conf::/usr/share/java/jdbc-mysql.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/java/mysql-jdbc-qe.jar:/usr/hdp/2.3.2.0-2862/tez/*:/usr/hdp/2.3.2.0-2862/tez/lib/*:/usr/hdp/2.3.2.0-2862/tez/conf:/usr/hdp/current/hadoop-yarn-nodemanager/.//*:/usr/hdp/current/hadoop-yarn-nodemanager/lib/*:/usr/hdp/current/hadoop-client/conf/nm-config/log4j.properties
 org.apache.hadoop.yarn.server.nodemanager.NodeManager
> 
> 
> **nodemanager log**
> 
> 
> 
> STARTUP_MSG:   build = g...@github.com:hortonworks/hadoop.git -r 
> b3f6bbba22d986059b77f46a96a523d6db679d50; compiled by 'jenkins' on 
> 2015-09-17T11:58Z
> STARTUP_MSG:   java = 1.7.0_79
> /
> 2015-09-17 23:09:23,120 INFO  nodemanager.NodeManager 
> (LogAdapter.java:info(45)) - registered UNIX signal handlers for [TERM, HUP, 
> INT]
> 2015-09-17 23:09:25,804 FATAL conf.Configuration 
> (Configuration.java:loadResource(2644)) - error parsing conf yarn-site.xml
> org.xml.sax.SAXParseException; Premature end of file.
> at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
> at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown 
> Source)
> at 
> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
> at 
> org.apache.hadoop.conf.Configuration.parse(Configuration.java:2480)
> at 
> org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
> at 
>

Re: Review Request 38559: [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log

2015-09-21 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38559/#review99766
---

Ship it!


Ship It!

- Alejandro Fernandez


On Sept. 21, 2015, 5:19 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38559/
> ---
> 
> (Updated Sept. 21, 2015, 5:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin 
> Jetly, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-13152
> https://issues.apache.org/jira/browse/AMBARI-13152
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Current issue is caused by commit AMBARI-13107 ("After successful install of 
> Oozie via blueprint, oozie has an invalid configuration") . Mentioned commit 
> removes few "value" sections from service metainfo. That's why some 
> properties in stack metainfo may have null values. 
> 
> A litle more details about the case I'm fixing:
> Both previous stack value and next stack value are null (not defined in 
> metainfo to avoid parsing by blueprint). But runtime property value after 
> upgrade is not-null (added by UpgradeCatalog). I'm fixing exactly this case, 
> other cases (only new or old stack property value is null and other is not) 
> are a way different because of bug nature.
> 
> UPD we decided to commit the fix to 2.1.2 as well since mentioned patch is 
> hard to revert (other patches are based on it)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java
>  9a2c604 
> 
> Diff: https://reviews.apache.org/r/38559/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Ambari Views .. SUCCESS [3.262s]
> [INFO] Ambari Metrics Common . SUCCESS [1.659s]
> [INFO] Ambari Server . SUCCESS 
> [53:43.587s]
> [INFO] Ambari Agent .. SUCCESS [9.281s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 53:58.376s
> [INFO] Finished at: Mon Sep 21 18:54:27 EEST 2015
> [INFO] Final Memory: 73M/1692M
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 38547: Oozie service have "Database host" required config after ambari upgrade from 1.7.0 to 2.1.2


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38547/#review99737
---

Ship it!


Ship It!

- Dmitro Lisnichenko


On Sept. 21, 2015, 9:49 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38547/
> ---
> 
> (Updated Sept. 21, 2015, 9:49 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro 
> Sen.
> 
> 
> Bugs: AMBARI-13165
> https://issues.apache.org/jira/browse/AMBARI-13165
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Oozie service have required config "Database host" after ambari upgrade from 
> 1.7.0 to 2.1.2
> STR
> install ambari 1.7.0 and deploy cluster
> upgrade ambari to 2.1.2
> navigate to oozie service configs
> ACTUAL: "Database host" is required
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  f4023b2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
>  b97a7f4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog212Test.java
>  5a947e3 
> 
> Diff: https://reviews.apache.org/r/38547/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 38553: Ambari with sqla db fails to start by non root user. Problem with jdbc.

2015-09-21 Thread Myroslav Papirkovskyy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38553/#review99744
---

Ship it!


Ship It!

- Myroslav Papirkovskyy


On Вер. 21, 2015, 2:39 після полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38553/
> ---
> 
> (Updated Вер. 21, 2015, 2:39 після полудня)
> 
> 
> Review request for Ambari and Myroslav Papirkovskyy.
> 
> 
> Bugs: AMBARI-13166
> https://issues.apache.org/jira/browse/AMBARI-13166
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> sap.jdbc4.sqlanywhere.IDriver
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:103)
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance()
>   at 
> com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
>   at 
> com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
>   at 
> com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
>   at 
> com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
>   at 
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
>   at com.google.inject.Scopes$1$1.get(Scopes.java:65)
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/dbConfiguration.py 2d39c72 
> 
> Diff: https://reviews.apache.org/r/38553/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

[jira] [Commented] (AMBARI-13165) Oozie service have "Database host" required config after ambari upgrade from 1.7.0 to 2.1.2


[ 
https://issues.apache.org/jira/browse/AMBARI-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900578#comment-14900578
 ] 

Hudson commented on AMBARI-13165:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3474 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3474/])
AMBARI-13165. Oozie service have "Database host" required config after ambari 
upgrade from 1.7.0 to 2.1.2.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=61edc13bc7c7510d7682c8897d9b9662a4867095)
* 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
* 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
* 
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog212Test.java


> Oozie service have "Database host" required config after ambari upgrade from 
> 1.7.0 to 2.1.2
> ---
>
> Key: AMBARI-13165
> URL: https://issues.apache.org/jira/browse/AMBARI-13165
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13165.patch
>
>
> Oozie service have required config "Database host" after ambari upgrade from 
> 1.7.0 to 2.1.2
> STR
> install ambari 1.7.0 and deploy cluster
> upgrade ambari to 2.1.2
> navigate to oozie service configs
> ACTUAL: "Database host" is required



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13168) Ambari unable to start Yarn after Ranger integration


[ 
https://issues.apache.org/jira/browse/AMBARI-13168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900904#comment-14900904
 ] 

Hari Sekhon commented on AMBARI-13168:
--

Reverting the Ranger integration allows RM to start.

> Ambari unable to start Yarn after Ranger integration
> 
>
> Key: AMBARI-13168
> URL: https://issues.apache.org/jira/browse/AMBARI-13168
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 1.2.0
> Environment: HDP 2.3 + Kerberos + Ranger
>Reporter: Hari Sekhon
>Priority: Critical
>
> After deploying Ranger via Ambari and enabling the Yarn Ranger plugin it 
> seems Ambari is no longer able to start the Yarn Resource Manager:
> {code}Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
>  line 221, in 
> Resourcemanager().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 459, in restart
> self.start(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
>  line 115, in start
> setup_ranger_yarn() #Ranger Yarn Plugin related calls
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/setup_ranger_yarn.py",
>  line 41, in setup_ranger_yarn
> api_version = 'v2'
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/setup_ranger_plugin_xml.py",
>  line 82, in setup_ranger_plugin
> policy_user)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/ranger_functions_v2.py",
>  line 95, in create_ranger_repository
> ambari_ranger_admin, ambari_ranger_password = 
> self.create_ambari_admin_user(ambari_ranger_admin, ambari_ranger_password, 
> format("{admin_uname}:{admin_password}"))
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/ranger_functions_v2.py",
>  line 189, in create_ambari_admin_user
> response = json.loads(result.read())
>   File "/usr/lib/python2.6/site-packages/ambari_simplejson/__init__.py", line 
> 307, in loads
> return _default_decoder.decode(s)
>   File "/usr/lib/python2.6/site-packages/ambari_simplejson/decoder.py", line 
> 335, in decode
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>   File "/usr/lib/python2.6/site-packages/ambari_simplejson/decoder.py", line 
> 353, in raw_decode
> raise ValueError("No JSON object could be decoded")
> ValueError: No JSON object could be decoded{code}
> The Ambari server log shows the following:
> {code}21 Sep 2015 14:54:40,130  INFO [qtp-ambari-agent-9264] 
> HeartBeatHandler:567 - Updating applied config on service YARN, component 
> NODEMANAGER, host 
> 21 Sep 2015 14:54:44,623  WARN [qtp-ambari-agent-9264] HeartBeatHandler:603 - 
> Operation failed - may be retried. Service component host: RESOURCEMANAGER, 
> host:  Action id113-0
> 21 Sep 2015 14:54:44,623 ERROR [qtp-ambari-agent-9264] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=RESOURCEMANAGER, hostName=, 
> currentState=INSTALLED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,623  WARN [qtp-ambari-agent-9264] HeartBeatHandler:621 - 
> State machine exception. Invalid event: HOST_SVCCOMP_OP_FAILED at INSTALLED
> 21 Sep 2015 14:54:44,811  WARN [ambari-action-scheduler] ActionScheduler:311 
> - Operation completely failed, aborting request id:113
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=APP_TIMELINE_SERVER, hostName=, 
> currentState=STARTED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=NODEMANAGER, hostName=, 
> currentState=STARTED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=RESOURCEMANAGER, hostName=, 
> currentState=INSTALLED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent

[jira] [Updated] (AMBARI-13168) Ambari unable to start Yarn after Ranger integration


 [ 
https://issues.apache.org/jira/browse/AMBARI-13168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sekhon updated AMBARI-13168:
-
Priority: Critical  (was: Blocker)

> Ambari unable to start Yarn after Ranger integration
> 
>
> Key: AMBARI-13168
> URL: https://issues.apache.org/jira/browse/AMBARI-13168
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 1.2.0
> Environment: HDP 2.3 + Kerberos + Ranger
>Reporter: Hari Sekhon
>Priority: Critical
>
> After deploying Ranger via Ambari and enabling the Yarn Ranger plugin it 
> seems Ambari is no longer able to start the Yarn Resource Manager:
> {code}Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
>  line 221, in 
> Resourcemanager().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 459, in restart
> self.start(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
>  line 115, in start
> setup_ranger_yarn() #Ranger Yarn Plugin related calls
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/setup_ranger_yarn.py",
>  line 41, in setup_ranger_yarn
> api_version = 'v2'
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/setup_ranger_plugin_xml.py",
>  line 82, in setup_ranger_plugin
> policy_user)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/ranger_functions_v2.py",
>  line 95, in create_ranger_repository
> ambari_ranger_admin, ambari_ranger_password = 
> self.create_ambari_admin_user(ambari_ranger_admin, ambari_ranger_password, 
> format("{admin_uname}:{admin_password}"))
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/ranger_functions_v2.py",
>  line 189, in create_ambari_admin_user
> response = json.loads(result.read())
>   File "/usr/lib/python2.6/site-packages/ambari_simplejson/__init__.py", line 
> 307, in loads
> return _default_decoder.decode(s)
>   File "/usr/lib/python2.6/site-packages/ambari_simplejson/decoder.py", line 
> 335, in decode
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>   File "/usr/lib/python2.6/site-packages/ambari_simplejson/decoder.py", line 
> 353, in raw_decode
> raise ValueError("No JSON object could be decoded")
> ValueError: No JSON object could be decoded{code}
> The Ambari server log shows the following:
> {code}21 Sep 2015 14:54:40,130  INFO [qtp-ambari-agent-9264] 
> HeartBeatHandler:567 - Updating applied config on service YARN, component 
> NODEMANAGER, host 
> 21 Sep 2015 14:54:44,623  WARN [qtp-ambari-agent-9264] HeartBeatHandler:603 - 
> Operation failed - may be retried. Service component host: RESOURCEMANAGER, 
> host:  Action id113-0
> 21 Sep 2015 14:54:44,623 ERROR [qtp-ambari-agent-9264] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=RESOURCEMANAGER, hostName=, 
> currentState=INSTALLED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,623  WARN [qtp-ambari-agent-9264] HeartBeatHandler:621 - 
> State machine exception. Invalid event: HOST_SVCCOMP_OP_FAILED at INSTALLED
> 21 Sep 2015 14:54:44,811  WARN [ambari-action-scheduler] ActionScheduler:311 
> - Operation completely failed, aborting request id:113
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=APP_TIMELINE_SERVER, hostName=, 
> currentState=STARTED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=NODEMANAGER, hostName=, 
> currentState=STARTED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state, serviceComponentName=RESOURCEMANAGER, hostName=, 
> currentState=INSTALLED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
> HOST_SVCCOMP_OP_FAILED
> 21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
> ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event 
> at current state,

[jira] [Created] (AMBARI-13171) Support Host in MM Removal During Upgrade

2015-09-21 Thread Jonathan Hurley (JIRA)

Jonathan Hurley created AMBARI-13171:


 Summary: Support Host in MM Removal During Upgrade
 Key: AMBARI-13171
 URL: https://issues.apache.org/jira/browse/AMBARI-13171
 Project: Ambari
  Issue Type: Task
  Components: ambari-server
Affects Versions: 2.1.0
Reporter: Jonathan Hurley
Assignee: Jonathan Hurley
Priority: Critical
 Fix For: 2.1.2


During an upgrade, AMBARI-12924 and AMBARI-12889 allow for a host to be placed 
into Maintenance Mode in order for it to be excluded from the upgrade. Although 
there is work planned to be able to finalize an upgrade with hosts that are in 
MM and out-of-sync, in the short term there also needs to be the ability to 
remove these hosts so that finalization can complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38495: [Upgrade] ambari server schema upgrade failed for Path 1.6.1 -> 2.1.2 and 2.0.2->2.1.2 and 2.1.1->2.1.2

2015-09-21 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38495/
---

(Updated Сен. 21, 2015, 2:37 п.п.)


Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and 
Vitalyi Brodetskyi.


Bugs: AMBARI-13122
https://issues.apache.org/jira/browse/AMBARI-13122


Repository: ambari


Description
---

{code}
Upgrade Path was 1.6.1 to 2.1.2.
root@os-r6-ambari-upgrade-oracle-5 ~]# ambari-server upgrade
Using python /usr/bin/python2.6
Upgrading ambari-server
Updating properties in ambari.properties ...
WARNING: Can not find ambari.properties.rpmsave file from previous version, 
skipping import of settings
WARNING: Can not find ambari-env.sh.rpmsave file from previous version, 
skipping restore of environment settings
Fixing database objects owner
Ambari Server configured for Oracle. Confirm you have made a backup of the 
Ambari Server database y/n ? y
Upgrading database schema
ERROR: Error executing schema upgrade, please check the server logs.
ERROR: Ambari server upgrade failed. Please look at 
/var/log/ambari-server/ambari-server.log, for more details.
ERROR: Exiting with exit code 11.
REASON: Schema upgrade failed.
{code}


Diffs (updated)
-

  ambari-server/src/main/python/ambari_server/dbConfiguration_linux.py 328e53e 
  ambari-server/src/main/python/ambari_server/serverUpgrade.py 73961d8 
  ambari-server/src/test/python/TestAmbariServer.py fe6380f 

Diff: https://reviews.apache.org/r/38495/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

[jira] [Created] (AMBARI-13169) One of kafka brokers failed to start

Vitaly Brodetskyi created AMBARI-13169:
--

 Summary: One of kafka brokers failed to start
 Key: AMBARI-13169
 URL: https://issues.apache.org/jira/browse/AMBARI-13169
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 2.1.2
Reporter: Vitaly Brodetskyi
Assignee: Vitaly Brodetskyi
Priority: Critical
 Fix For: 2.1.2


After second try, it started successfully.
{code}
[2015-09-21 06:22:20,916] FATAL Fatal error during KafkaServerStartable 
startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to 
zookeeper server within timeout: 15000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
at org.I0Itec.zkclient.ZkClient.(ZkClient.java:143)
at org.I0Itec.zkclient.ZkClient.(ZkClient.java:121)
at org.I0Itec.zkclient.ZkClient.(ZkClient.java:88)
at 
kafka.security.auth.SimpleAclAuthorizer.initialize(SimpleAclAuthorizer.scala:70)
at kafka.server.KafkaServer.startup(KafkaServer.scala:159)
at 
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
at kafka.Kafka$.main(Kafka.scala:72)
at kafka.Kafka.main(Kafka.scala)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-13168) Ambari unable to start Yarn after Ranger integration

Hari Sekhon created AMBARI-13168:


 Summary: Ambari unable to start Yarn after Ranger integration
 Key: AMBARI-13168
 URL: https://issues.apache.org/jira/browse/AMBARI-13168
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 1.2.0
 Environment: HDP 2.3 + Kerberos + Ranger
Reporter: Hari Sekhon
Priority: Blocker


After deploying Ranger via Ambari and enabling the Yarn Ranger plugin it seems 
Ambari is no longer able to start the Yarn Resource Manager:
{code}Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
 line 221, in 
Resourcemanager().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 218, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 459, in restart
self.start(env)
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
 line 115, in start
setup_ranger_yarn() #Ranger Yarn Plugin related calls
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/setup_ranger_yarn.py",
 line 41, in setup_ranger_yarn
api_version = 'v2'
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/setup_ranger_plugin_xml.py",
 line 82, in setup_ranger_plugin
policy_user)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/ranger_functions_v2.py",
 line 95, in create_ranger_repository
ambari_ranger_admin, ambari_ranger_password = 
self.create_ambari_admin_user(ambari_ranger_admin, ambari_ranger_password, 
format("{admin_uname}:{admin_password}"))
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/ranger_functions_v2.py",
 line 189, in create_ambari_admin_user
response = json.loads(result.read())
  File "/usr/lib/python2.6/site-packages/ambari_simplejson/__init__.py", line 
307, in loads
return _default_decoder.decode(s)
  File "/usr/lib/python2.6/site-packages/ambari_simplejson/decoder.py", line 
335, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.6/site-packages/ambari_simplejson/decoder.py", line 
353, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded{code}
The Ambari server log shows the following:
{code}21 Sep 2015 14:54:40,130  INFO [qtp-ambari-agent-9264] 
HeartBeatHandler:567 - Updating applied config on service YARN, component 
NODEMANAGER, host 
21 Sep 2015 14:54:44,623  WARN [qtp-ambari-agent-9264] HeartBeatHandler:603 - 
Operation failed - may be retried. Service component host: RESOURCEMANAGER, 
host:  Action id113-0
21 Sep 2015 14:54:44,623 ERROR [qtp-ambari-agent-9264] 
ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event at 
current state, serviceComponentName=RESOURCEMANAGER, hostName=, 
currentState=INSTALLED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
HOST_SVCCOMP_OP_FAILED
21 Sep 2015 14:54:44,623  WARN [qtp-ambari-agent-9264] HeartBeatHandler:621 - 
State machine exception. Invalid event: HOST_SVCCOMP_OP_FAILED at INSTALLED
21 Sep 2015 14:54:44,811  WARN [ambari-action-scheduler] ActionScheduler:311 - 
Operation completely failed, aborting request id:113
21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event at 
current state, serviceComponentName=APP_TIMELINE_SERVER, hostName=, 
currentState=STARTED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
HOST_SVCCOMP_OP_FAILED
21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event at 
current state, serviceComponentName=NODEMANAGER, hostName=, 
currentState=STARTED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
HOST_SVCCOMP_OP_FAILED
21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event at 
current state, serviceComponentName=RESOURCEMANAGER, hostName=, 
currentState=INSTALLED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
HOST_SVCCOMP_OP_FAILED
21 Sep 2015 14:54:44,811 ERROR [ambari-action-scheduler] 
ServiceComponentHostImpl:906 - Can't handle ServiceComponentHostEvent event at 
current state, serviceComponentName=YARN_CLIENT, hostName=, 
currentState=INSTALLED, eventType=HOST_SVCCOMP_OP_FAILED, event=EventType: 
HOST_SVCCOMP_OP_FAILED
21 Sep 2015 14:54:44,814  INFO [ambari-action-scheduler] 
ActionDBAccessorImpl:176 - Aborting command. Hostname  role YARN_CLIENT 
requestId null taskId 792 stageId null{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38557: One of kafka brokers failed to start


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38557/
---

Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-13169
https://issues.apache.org/jira/browse/AMBARI-13169


Repository: ambari


Description
---

After second try, it started successfully.
{code}
[2015-09-21 06:22:20,916] FATAL Fatal error during KafkaServerStartable 
startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to 
zookeeper server within timeout: 15000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
at org.I0Itec.zkclient.ZkClient.(ZkClient.java:143)
at org.I0Itec.zkclient.ZkClient.(ZkClient.java:121)
at org.I0Itec.zkclient.ZkClient.(ZkClient.java:88)
at 
kafka.security.auth.SimpleAclAuthorizer.initialize(SimpleAclAuthorizer.scala:70)
at kafka.server.KafkaServer.startup(KafkaServer.scala:159)
at 
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
at kafka.Kafka$.main(Kafka.scala:72)
at kafka.Kafka.main(Kafka.scala)
{code}


Diffs
-

  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml
 255707a 

Diff: https://reviews.apache.org/r/38557/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

[jira] [Created] (AMBARI-13165) Oozie service have "Database host" required config after ambari upgrade from 1.7.0 to 2.1.2

Vitaly Brodetskyi created AMBARI-13165:
--

 Summary: Oozie service have "Database host" required config after 
ambari upgrade from 1.7.0 to 2.1.2
 Key: AMBARI-13165
 URL: https://issues.apache.org/jira/browse/AMBARI-13165
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 2.1.2
Reporter: Vitaly Brodetskyi
Assignee: Vitaly Brodetskyi
Priority: Critical
 Fix For: 2.1.2


Oozie service have required config "Database host" after ambari upgrade from 
1.7.0 to 2.1.2
STR
install ambari 1.7.0 and deploy cluster
upgrade ambari to 2.1.2
navigate to oozie service configs
ACTUAL: "Database host" is required



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38547: Oozie service have "Database host" required config after ambari upgrade from 1.7.0 to 2.1.2


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38547/
---

Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro Sen.


Bugs: AMBARI-13165
https://issues.apache.org/jira/browse/AMBARI-13165


Repository: ambari


Description
---

Oozie service have required config "Database host" after ambari upgrade from 
1.7.0 to 2.1.2
STR
install ambari 1.7.0 and deploy cluster
upgrade ambari to 2.1.2
navigate to oozie service configs
ACTUAL: "Database host" is required


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 f4023b2 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
 b97a7f4 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog212Test.java
 5a947e3 

Diff: https://reviews.apache.org/r/38547/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

[jira] [Updated] (AMBARI-13165) Oozie service have "Database host" required config after ambari upgrade from 1.7.0 to 2.1.2


 [ 
https://issues.apache.org/jira/browse/AMBARI-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-13165:
---
Attachment: AMBARI-13165.patch

> Oozie service have "Database host" required config after ambari upgrade from 
> 1.7.0 to 2.1.2
> ---
>
> Key: AMBARI-13165
> URL: https://issues.apache.org/jira/browse/AMBARI-13165
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13165.patch
>
>
> Oozie service have required config "Database host" after ambari upgrade from 
> 1.7.0 to 2.1.2
> STR
> install ambari 1.7.0 and deploy cluster
> upgrade ambari to 2.1.2
> navigate to oozie service configs
> ACTUAL: "Database host" is required



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13169) One of kafka brokers failed to start


[ 
https://issues.apache.org/jira/browse/AMBARI-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901315#comment-14901315
 ] 

Hudson commented on AMBARI-13169:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3478 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3478/])
AMBARI-13169. One of kafka brokers failed to start.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=4e39212135ae6e4ce998c432b392a0dd2b77f86a)
* 
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml


> One of kafka brokers failed to start
> 
>
> Key: AMBARI-13169
> URL: https://issues.apache.org/jira/browse/AMBARI-13169
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13169.patch
>
>
> After second try, it started successfully.
> {code}
> [2015-09-21 06:22:20,916] FATAL Fatal error during KafkaServerStartable 
> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to 
> zookeeper server within timeout: 15000
>   at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:143)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:121)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:88)
>   at 
> kafka.security.auth.SimpleAclAuthorizer.initialize(SimpleAclAuthorizer.scala:70)
>   at kafka.server.KafkaServer.startup(KafkaServer.scala:159)
>   at 
> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
>   at kafka.Kafka$.main(Kafka.scala:72)
>   at kafka.Kafka.main(Kafka.scala)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13152) [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in ambari-server log


[ 
https://issues.apache.org/jira/browse/AMBARI-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901313#comment-14901313
 ] 

Hudson commented on AMBARI-13152:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3478 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3478/])
AMBARI-13152. [Upgrade] Check CONFIG_MERGE failed error message is consistently 
reported in ambari-server log (dlysnichenko) (dlysnichenko: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=d278ed1a2c61653aab10ea43e4d13bf37a0b8ec2)
* 
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java


> [Upgrade] Check CONFIG_MERGE failed error message is consistently reported in 
> ambari-server log
> ---
>
> Key: AMBARI-13152
> URL: https://issues.apache.org/jira/browse/AMBARI-13152
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-upgrade
>Affects Versions: 2.1.2
> Environment: Os - Centos6
> Ambari database - Oracle 12
> Hive/Oozie database - Oracle 12
> Ambari Upgrade Path - 2.0.2 -> 2.1.2
>Reporter: Pankaj Singh
>Priority: Critical
>
> Completed ambari server and ambari database upgrade.
> Traversed to versions tab of stack and versions through ambari 
> Sample url - http://:8080/#/main/admin/stack/versions
> Through manage version added HDP-2.3.2.2874 to the stack.
> At same time saw that "18 Sep 2015 18:19:01,981 ERROR qtp-client-24 
> CheckHelper:72 - Check CONFIG_MERGE failed" message started appearing in 
> ambari-server.log
> The complete stack trace
> 18 Sep 2015 18:18:58,385 INFO qtp-client-22 PersistKeyValueService:82 - 
> Looking for keyName CLUSTER_CURRENT_STATUS
> 18 Sep 2015 18:19:01,981 ERROR qtp-client-24 CheckHelper:72 - Check 
> CONFIG_MERGE failed
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.state.ConfigMergeHelper.getConflicts(ConfigMergeHelper.java:131)
> at 
> org.apache.ambari.server.checks.ConfigurationMergeCheck.perform(ConfigurationMergeCheck.java:94)
> at 
> org.apache.ambari.server.state.CheckHelper.performChecks(CheckHelper.java:61)
> at 
> org.apache.ambari.server.controller.internal.PreUpgradeCheckResourceProvider.getResources(PreUpgradeCheckResourceProvider.java:140)
> at 
> org.apache.ambari.server.controller.internal.ClusterControllerImpl$ExtendedResourceProviderWrapper.queryForResources(ClusterControllerImpl.java:945)
> at 
> org.apache.ambari.server.controller.internal.ClusterControllerImpl.getResources(ClusterControllerImpl.java:132)
> at org.apache.ambari.server.api.query.QueryImpl.doQuery(QueryImpl.java:508)
> at 
> org.apache.ambari.server.api.query.QueryImpl.queryForResources(QueryImpl.java:380)
> at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:216)
> at 
> org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:68)
> at 
> org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:135)
> at 
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:105)
> at 
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:74)
> at 
> org.apache.ambari.server.api.services.PreUpgradeCheckService.getPreUpgradeChecks(PreUpgradeCheckService.java:62)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at 
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at 
>

Re: Review Request 38543: RU: Knox to use versioned data dir starting in HDP 2.3.2.0

2015-09-21 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38543/
---

(Updated Sept. 21, 2015, 8:18 p.m.)

Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Srimanth
Gunturi, and Sid Wagle.

Bugs: AMBARI-13164
https://issues.apache.org/jira/browse/AMBARI-13164

Repository: ambari

Description
---

Starting in HDP 2.3.2.0, Knox will version its data directory.

HDP 2.3.2.0 is meant to support Debian & Ubuntu.
A customer installing HDP 2.3.2.0 would receive the following for knox:
/usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
{version}

Hence, all of the data directories in /var/lib/knox/ would be versioned. This
is because the Debian installer fails if it tries to create the
/var/lib/knox/data folder and it already exists (since it belongs to a previous
version); further, you would get data loss if you tried to uninstall the newer
version, since it would wipe out the folder that was used by the previous
version.

So the problem doesn't arise on the initial install, but rather on installing
an additional version for Rolling Upgrade.

This means that any customer planning to upgrade Knox must first upgrade Ambari
to 2.1.2.

There are 3 scenarios:

1. Ambari 2.1.2 with HDP 2.3.0.0:
Ambari must continue to use the non-versioned data directory.

2. Ambari 2.1.2 with HDP 2.3.2.0:
A brand new install will use the versioned data directory out-of-the-box.

3. Ambari 2.1.0 with HDP 2.3.0.0:
Technically, this allows registering bits for HDP 2.3.2.0 and performing an RU
to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means that
any customer with Knox that plans to perform an RU must first upgrade Ambari to
2.1.2, which is equivalent to starting with case #1.

Because RU allows the following paths,
2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is
available)

It means that it must always copy any contents from
previous_data_dir to /usr/hdp/{new-version}/knox/data

In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
In the case of 2.3.+ -> 2.3.+, previous_data_dir is
/usr/hdp/{prev-version}/knox/data

Diffs (updated)
-

ambari-common/src/main/python/resource_management/libraries/script/script.py
e0a7877

ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
990fc87

ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
2c805fa

ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
9ac2d63
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 3ae42a0

Diff: https://reviews.apache.org/r/38543/diff/

Testing
---

Ran python unit tests, which passed.
Verified during RU in several scenarios.
1. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Before Finalizing,
Downgraded.
2. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Finalized.
3. Steps from #2, and then perform an RU to a higher build in HDP 2.3.2.0

Thanks,

Alejandro Fernandez

[jira] [Updated] (AMBARI-13164) RU: Knox to use versioned data dir starting in HDP 2.3.2.0

2015-09-21 Thread Alejandro Fernandez (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Fernandez updated AMBARI-13164:
-
Attachment: AMBARI-13164.trunk.patch

> RU: Knox to use versioned data dir starting in HDP 2.3.2.0
> --
>
> Key: AMBARI-13164
> URL: https://issues.apache.org/jira/browse/AMBARI-13164
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
> Fix For: 2.2.0, 2.1.2
>
> Attachments: AMBARI-13164.branch-2.1.2.patch, 
> AMBARI-13164.branch-2.1.patch, AMBARI-13164.trunk.patch
>
>
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> There are 3 scenarios:
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-13164) RU: Knox to use versioned data dir starting in HDP 2.3.2.0

2015-09-21 Thread Alejandro Fernandez (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Fernandez updated AMBARI-13164:
-
Attachment: (was: AMBARI-13164.trunk.patch)

> RU: Knox to use versioned data dir starting in HDP 2.3.2.0
> --
>
> Key: AMBARI-13164
> URL: https://issues.apache.org/jira/browse/AMBARI-13164
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
> Fix For: 2.2.0, 2.1.2
>
> Attachments: AMBARI-13164.branch-2.1.2.patch, 
> AMBARI-13164.branch-2.1.patch, AMBARI-13164.trunk.patch
>
>
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> There are 3 scenarios:
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13162) check_fs_root should be skipped for non hdfs FS


[ 
https://issues.apache.org/jira/browse/AMBARI-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901314#comment-14901314
 ] 

Hudson commented on AMBARI-13162:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3478 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3478/])
AMBARI-13162. check_fs_root for HIVE service should also support HCFS type 
(smohanty) (smohanty: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=78ded38f96d4d03967feddae2c96478ad578e89b)
* ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json
* 
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py
* ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py


> check_fs_root should be skipped for non hdfs FS
> ---
>
> Key: AMBARI-13162
> URL: https://issues.apache.org/jira/browse/AMBARI-13162
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.2
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13162.patch
>
>
> check_fs_root includes the logic to modify the FS path when it changes. The 
> current implementation only supports it for hdfs. So as it stands today it 
> should be skipped for non hdfs paths. A follow up JIRA is needed to provide 
> support for any FS path. This will require Ambari remembering the prior value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38543: RU: Knox to use versioned data dir starting in HDP 2.3.2.0



> On Sept. 21, 2015, 4:24 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py,
> >  line 53
> > 
> >
> > Make this different name than your username :)

Too funny.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38543/#review99818
---


On Sept. 21, 2015, 4:18 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38543/
> ---
> 
> (Updated Sept. 21, 2015, 4:18 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, 
> Srimanth Gunturi, and Sid Wagle.
> 
> 
> Bugs: AMBARI-13164
> https://issues.apache.org/jira/browse/AMBARI-13164
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> 
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> 
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> 
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> 
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> 
> There are 3 scenarios:
> 
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> 
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> 
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> 
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> e0a7877 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  990fc87 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
>  2c805fa 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
>  9ac2d63 
>   ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 3ae42a0 
> 
> Diff: https://reviews.apache.org/r/38543/diff/
> 
> 
> Testing
> ---
> 
> Ran python unit tests, which passed.
> Verified during RU in several scenarios.
> 1. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Before 
> Finalizing, Downgraded.
> 2. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Finalized.
> 3. Steps from #2, and then perform an RU to a higher build in HDP 2.3.2.0
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 38543: RU: Knox to use versioned data dir starting in HDP 2.3.2.0


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38543/#review99819
---

Ship it!


Ship It!

- Jonathan Hurley


On Sept. 21, 2015, 4:18 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38543/
> ---
> 
> (Updated Sept. 21, 2015, 4:18 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, 
> Srimanth Gunturi, and Sid Wagle.
> 
> 
> Bugs: AMBARI-13164
> https://issues.apache.org/jira/browse/AMBARI-13164
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> 
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> 
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> 
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> 
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> 
> There are 3 scenarios:
> 
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> 
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> 
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> 
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> e0a7877 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  990fc87 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
>  2c805fa 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
>  9ac2d63 
>   ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 3ae42a0 
> 
> Diff: https://reviews.apache.org/r/38543/diff/
> 
> 
> Testing
> ---
> 
> Ran python unit tests, which passed.
> Verified during RU in several scenarios.
> 1. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Before 
> Finalizing, Downgraded.
> 2. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Finalized.
> 3. Steps from #2, and then perform an RU to a higher build in HDP 2.3.2.0
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 38567: Support Host in MM Removal During Upgrade


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38567/
---

Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-13171
https://issues.apache.org/jira/browse/AMBARI-13171


Repository: ambari


Description
---

During an upgrade, AMBARI-12924 and AMBARI-12889 allow for a host to be placed 
into Maintenance Mode in order for it to be excluded from the upgrade. Although 
there is work planned to be able to finalize an upgrade with hosts that are in 
MM and out-of-sync, in the short term there also needs to be the ability to 
remove these hosts so that finalization can complete.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/events/HostRemovedEvent.java
 e005754 
  
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java
 0824760 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
 6ede46b 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
 62613ff 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 eda2534 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 972ceec 
  
ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java
 b056991 

Diff: https://reviews.apache.org/r/38567/diff/


Testing
---

mvn clean test


Thanks,

Jonathan Hurley

Re: Review Request 38543: RU: Knox to use versioned data dir starting in HDP 2.3.2.0

2015-09-21 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38543/#review99818
---

Ship it!



ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
 (line 53)


Make this different name than your username :)


- Nate Cole


On Sept. 21, 2015, 4:18 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38543/
> ---
> 
> (Updated Sept. 21, 2015, 4:18 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, 
> Srimanth Gunturi, and Sid Wagle.
> 
> 
> Bugs: AMBARI-13164
> https://issues.apache.org/jira/browse/AMBARI-13164
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> 
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> 
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> 
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> 
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> 
> There are 3 scenarios:
> 
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> 
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> 
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> 
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> e0a7877 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  990fc87 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
>  2c805fa 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
>  9ac2d63 
>   ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 3ae42a0 
> 
> Diff: https://reviews.apache.org/r/38543/diff/
> 
> 
> Testing
> ---
> 
> Ran python unit tests, which passed.
> Verified during RU in several scenarios.
> 1. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Before 
> Finalizing, Downgraded.
> 2. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Finalized.
> 3. Steps from #2, and then perform an RU to a higher build in HDP 2.3.2.0
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 38543: RU: Knox to use versioned data dir starting in HDP 2.3.2.0

2015-09-21 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38543/
---

(Updated Sept. 21, 2015, 8:39 p.m.)

Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Srimanth
Gunturi, and Sid Wagle.

Changes
---

Addressed comments, unit tests passed.

Bugs: AMBARI-13164
https://issues.apache.org/jira/browse/AMBARI-13164

Repository: ambari

Description
---

Starting in HDP 2.3.2.0, Knox will version its data directory.

HDP 2.3.2.0 is meant to support Debian & Ubuntu.
A customer installing HDP 2.3.2.0 would receive the following for knox:
/usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
{version}

So the problem doesn't arise on the initial install, but rather on installing
an additional version for Rolling Upgrade.

This means that any customer planning to upgrade Knox must first upgrade Ambari
to 2.1.2.

There are 3 scenarios:

1. Ambari 2.1.2 with HDP 2.3.0.0:
Ambari must continue to use the non-versioned data directory.

2. Ambari 2.1.2 with HDP 2.3.2.0:
A brand new install will use the versioned data directory out-of-the-box.

Because RU allows the following paths,
2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is
available)

It means that it must always copy any contents from
previous_data_dir to /usr/hdp/{new-version}/knox/data

In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
In the case of 2.3.+ -> 2.3.+, previous_data_dir is
/usr/hdp/{prev-version}/knox/data

Diffs (updated)
-

ambari-common/src/main/python/resource_management/libraries/script/script.py
e0a7877

ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
990fc87

ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
2c805fa

ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
9ac2d63
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 3ae42a0

Diff: https://reviews.apache.org/r/38543/diff/

Testing
---

Thanks,

Alejandro Fernandez

[jira] [Commented] (AMBARI-13164) RU: Knox to use versioned data dir starting in HDP 2.3.2.0

2015-09-21 Thread Alejandro Fernandez (JIRA)


[ 
https://issues.apache.org/jira/browse/AMBARI-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901373#comment-14901373
 ] 

Alejandro Fernandez commented on AMBARI-13164:
--

Pushed to trunk,
commit f9271abc25614adbcfa00599db643a24ac20f0f2

branch-2.1,
commit f11802c73b9e18b664ae8107b6d00fa0aedf32be

branch-2.1.2,
commit e16b6bbd385a61aa81e86e1a8a5b9d47e0c4e5f6



> RU: Knox to use versioned data dir starting in HDP 2.3.2.0
> --
>
> Key: AMBARI-13164
> URL: https://issues.apache.org/jira/browse/AMBARI-13164
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
> Fix For: 2.2.0, 2.1.2
>
> Attachments: AMBARI-13164.branch-2.1.2.patch, 
> AMBARI-13164.branch-2.1.patch, AMBARI-13164.trunk.patch
>
>
> Starting in HDP 2.3.2.0, Knox will version its data directory.
> HDP 2.3.2.0 is meant to support Debian & Ubuntu.
> A customer installing HDP 2.3.2.0 would receive the following for knox:
> /usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
> {version}
> Hence, all of the data directories in /var/lib/knox/ would be versioned. This 
> is because the Debian installer fails if it tries to create the 
> /var/lib/knox/data folder and it already exists (since it belongs to a 
> previous version); further, you would get data loss if you tried to uninstall 
> the newer version, since it would wipe out the folder that was used by the 
> previous version.
> So the problem doesn't arise on the initial install, but rather on installing 
> an additional version for Rolling Upgrade.
> This means that any customer planning to upgrade Knox must first upgrade 
> Ambari to 2.1.2.
> There are 3 scenarios:
> 1. Ambari 2.1.2 with HDP 2.3.0.0:
> Ambari must continue to use the non-versioned data directory.
> 2. Ambari 2.1.2 with HDP 2.3.2.0:
> A brand new install will use the versioned data directory out-of-the-box.
> 3. Ambari 2.1.0 with HDP 2.3.0.0:
> Technically, this allows registering bits for HDP 2.3.2.0 and performing an 
> RU to it. However, because HDP 2.3.2.0 uses a versioned data dir, it means 
> that any customer with Knox that plans to perform an RU must first upgrade 
> Ambari to 2.1.2, which is equivalent to starting with case #1.
> Because RU allows the following paths, 
> 2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
> 2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is 
> available)
> It means that it must always copy any contents from 
> previous_data_dir to  /usr/hdp/{new-version}/knox/data
> In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
> In the case of 2.3.+ -> 2.3.+, previous_data_dir is 
> /usr/hdp/{prev-version}/knox/data



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13162) check_fs_root should be skipped for non hdfs FS


[ 
https://issues.apache.org/jira/browse/AMBARI-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901252#comment-14901252
 ] 

Hudson commented on AMBARI-13162:
-

SUCCESS: Integrated in Ambari-branch-2.1 #568 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/568/])
AMBARI-13162. check_fs_root for HIVE service should also support HCFS type 
(smohanty) (smohanty: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=49e8b31e7cd2e1fd61c617701bd8e2b5a15463b0)
* ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py
* ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json
* 
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py


> check_fs_root should be skipped for non hdfs FS
> ---
>
> Key: AMBARI-13162
> URL: https://issues.apache.org/jira/browse/AMBARI-13162
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.2
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13162.patch
>
>
> check_fs_root includes the logic to modify the FS path when it changes. The 
> current implementation only supports it for hdfs. So as it stands today it 
> should be skipped for non hdfs paths. A follow up JIRA is needed to provide 
> support for any FS path. This will require Ambari remembering the prior value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13122) [Upgrade] ambari server schema upgrade failed for Path 1.6.1 -> 2.1.2 and 2.0.2->2.1.2 and 2.1.1->2.1.2


[ 
https://issues.apache.org/jira/browse/AMBARI-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901251#comment-14901251
 ] 

Hudson commented on AMBARI-13122:
-

SUCCESS: Integrated in Ambari-branch-2.1 #568 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/568/])
AMBARI-13122 [Upgrade] ambari server schema upgrade failed for Path 1.6.1 -> 
2.1.2 and 2.0.2->2.1.2 and 2.1.1->2.1.2 (dsen) (dsen: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=7e0965f557500a5951c638aa3f6ed5632464acc3)
* ambari-server/src/main/python/ambari_server/dbConfiguration_linux.py
* ambari-server/src/main/python/ambari_server/serverUpgrade.py
* ambari-server/src/test/python/TestAmbariServer.py


> [Upgrade] ambari server schema upgrade failed for Path 1.6.1 -> 2.1.2 and 
> 2.0.2->2.1.2 and 2.1.1->2.1.2
> ---
>
> Key: AMBARI-13122
> URL: https://issues.apache.org/jira/browse/AMBARI-13122
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Fix For: 2.1.2
>
> Attachments: AMBARI-13122.patch
>
>
> {code}
> Upgrade Path was 1.6.1 to 2.1.2.
> root@os-r6-ambari-upgrade-oracle-5 ~]# ambari-server upgrade
> Using python /usr/bin/python2.6
> Upgrading ambari-server
> Updating properties in ambari.properties ...
> WARNING: Can not find ambari.properties.rpmsave file from previous version, 
> skipping import of settings
> WARNING: Can not find ambari-env.sh.rpmsave file from previous version, 
> skipping restore of environment settings
> Fixing database objects owner
> Ambari Server configured for Oracle. Confirm you have made a backup of the 
> Ambari Server database y/n ? y
> Upgrading database schema
> ERROR: Error executing schema upgrade, please check the server logs.
> ERROR: Ambari server upgrade failed. Please look at 
> /var/log/ambari-server/ambari-server.log, for more details.
> ERROR: Exiting with exit code 11.
> REASON: Schema upgrade failed.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13169) One of kafka brokers failed to start


[ 
https://issues.apache.org/jira/browse/AMBARI-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901253#comment-14901253
 ] 

Hudson commented on AMBARI-13169:
-

SUCCESS: Integrated in Ambari-branch-2.1 #568 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/568/])
AMBARI-13169. One of kafka brokers failed to start.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=c933395f91ac8d24660b7bf80b305a47176c30a8)
* 
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml


> One of kafka brokers failed to start
> 
>
> Key: AMBARI-13169
> URL: https://issues.apache.org/jira/browse/AMBARI-13169
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13169.patch
>
>
> After second try, it started successfully.
> {code}
> [2015-09-21 06:22:20,916] FATAL Fatal error during KafkaServerStartable 
> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to 
> zookeeper server within timeout: 15000
>   at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:143)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:121)
>   at org.I0Itec.zkclient.ZkClient.(ZkClient.java:88)
>   at 
> kafka.security.auth.SimpleAclAuthorizer.initialize(SimpleAclAuthorizer.scala:70)
>   at kafka.server.KafkaServer.startup(KafkaServer.scala:159)
>   at 
> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
>   at kafka.Kafka$.main(Kafka.scala:72)
>   at kafka.Kafka.main(Kafka.scala)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13083) Use recommendedValue instead of defaultDirectory attribute for site properties


[ 
https://issues.apache.org/jira/browse/AMBARI-13083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900598#comment-14900598
 ] 

Hudson commented on AMBARI-13083:
-

SUCCESS: Integrated in Ambari-branch-2.1 #565 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/565/])
AMBARI-13083 Use recommendedValue instead of defaultDirectory attribute for 
site properties. (ababiichuk) (ababiichuk: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=2b31859c14be95442aedb3ef7bffe6fafbe3ae1d)
* ambari-web/test/utils/configs/config_property_helper_test.js
* ambari-web/app/utils/configs/config_property_helper.js
* ambari-web/app/models/configs/objects/service_config_property.js
* ambari-web/app/data/HDP2/site_properties.js
* ambari-web/app/data/BIGTOP/site_properties.js
* ambari-web/app/data/HDP2/ha_properties.js
* ambari-web/test/utils/config_test.js
* ambari-web/app/data/HDP2.2/site_properties.js
* ambari-web/app/utils/config.js


> Use recommendedValue instead of defaultDirectory attribute for site properties
> --
>
> Key: AMBARI-13083
> URL: https://issues.apache.org/jira/browse/AMBARI-13083
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.2.0
>Reporter: Andrii Babiichuk
>Assignee: Andrii Babiichuk
> Fix For: 2.2.0
>
> Attachments: AMBARI-13083.patch
>
>
> Lets remove 'defaultDirectory' attribute and instead use the recommendedValue 
> that we get from stack or recommendation endpoint API. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-13165) Oozie service have "Database host" required config after ambari upgrade from 1.7.0 to 2.1.2


[ 
https://issues.apache.org/jira/browse/AMBARI-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900599#comment-14900599
 ] 

Hudson commented on AMBARI-13165:
-

SUCCESS: Integrated in Ambari-branch-2.1 #565 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/565/])
AMBARI-13165. Oozie service have "Database host" required config after ambari 
upgrade from 1.7.0 to 2.1.2.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=db89de8f213a0af84e61cdfe1583932e35de01b6)
* 
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog212Test.java
* 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
* 
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java


> Oozie service have "Database host" required config after ambari upgrade from 
> 1.7.0 to 2.1.2
> ---
>
> Key: AMBARI-13165
> URL: https://issues.apache.org/jira/browse/AMBARI-13165
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-13165.patch
>
>
> Oozie service have required config "Database host" after ambari upgrade from 
> 1.7.0 to 2.1.2
> STR
> install ambari 1.7.0 and deploy cluster
> upgrade ambari to 2.1.2
> navigate to oozie service configs
> ACTUAL: "Database host" is required



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (AMBARI-13172) Kafka Service check fails with umask 027 on secure cluster

2015-09-21 Thread Myroslav Papirkovskyy (JIRA)


 [ 
https://issues.apache.org/jira/browse/AMBARI-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Myroslav Papirkovskyy resolved AMBARI-13172.

Resolution: Fixed

Pushed to trunk, branch-2.1, branch-2.1.2

> Kafka Service check fails with umask 027 on secure cluster
> --
>
> Key: AMBARI-13172
> URL: https://issues.apache.org/jira/browse/AMBARI-13172
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Myroslav Papirkovskyy
>Assignee: Myroslav Papirkovskyy
>Priority: Blocker
> Fix For: 2.1.2
>
>
> Kafka service check failed with exception:
> {noformat}
> [2015-09-21 18:16:34,906] WARN SASL configuration failed: 
> javax.security.auth.login.LoginException: Zookeeper client cannot 
> authenticate using the 'Client' section of the supplied JAAS configuration: 
> '/usr/hdp/current/kafka-broker/config/kafka_jaas.conf' because of a 
> RuntimeException: java.lang.SecurityException: Configuration Error:
>   No such file or directory Will continue connection to Zookeeper server 
> without SASL authentication, if Zookeeper server allows it. 
> (org.apache.zookeeper.ClientCnxn)
> Error while executing topic command null
> java.lang.ExceptionInInitializerError
>   at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:231)
>   at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:92)
>   at kafka.admin.TopicCommand$.main(TopicCommand.scala:54)
>   at kafka.admin.TopicCommand.main(TopicCommand.scala)
> Caused by: java.lang.RuntimeException: File 
> /usr/hdp/current/kafka-broker/config/kafka_jaas.conf cannot be read.
>   at kafka.utils.ZkUtils$.(ZkUtils.scala:65)
>   at kafka.utils.ZkUtils$.(ZkUtils.scala)
>   ... 4 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38515: Compare config versions doesn't work for Oozie

2015-09-21 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38515/#review99803
---

Ship it!


Ship It!

- Jaimin Jetly


On Sept. 19, 2015, 12:50 a.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38515/
> ---
> 
> (Updated Sept. 19, 2015, 12:50 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Yusaku Sako.
> 
> 
> Bugs: AMBARI-13155
> https://issues.apache.org/jira/browse/AMBARI-13155
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Avoid using parentView.xxx to get data because view hierarchy keeps changing.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/views/common/controls_view.js c157c67 
>   ambari-web/test/views/common/controls_view_test.js 6e4315c 
> 
> Diff: https://reviews.apache.org/r/38515/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> All unit test passed.
> 
>   9146 tests complete (9 seconds)
>   95 tests pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

[jira] [Commented] (AMBARI-13122) [Upgrade] ambari server schema upgrade failed for Path 1.6.1 -> 2.1.2 and 2.0.2->2.1.2 and 2.1.1->2.1.2