Re: Review Request 53801: AMBARI-18901. Use 'Number of LLAP Nodes' selected as the driver for LLAP config calculations.

2016-11-17 Thread Swapan Shridhar


> On Nov. 16, 2016, 6:23 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml,
> >  line 674
> > 
> >
> > If a customer is on Ambari 2.4 and HDP 2.5, and they upgrade to Ambari 
> > 2.5, then they will pick up these new configs.
> > Is this the intended behavior?

Added the Upgrade code.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53801/#review156094
---


On Nov. 18, 2016, 2:34 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53801/
> ---
> 
> (Updated Nov. 18, 2016, 2:34 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Madhuvanthi Radhakrishnan, 
> Sumit Mohanty, and Siddharth Seth.
> 
> 
> Bugs: AMBARI-18901
> https://issues.apache.org/jira/browse/AMBARI-18901
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18901. Use 'Number of LLAP Nodes' selected as the driver for LLAP 
> config calculations.
> 
> Below is the calculation logic used:
> 
> 
> **
> **
> 
> 
> ---
> For use with default setup - Ambari managed queue
> 
> Parameteres
> numRequestedLlapNodes UserInput
> tezAmContainerSizeComputed
> memoryPerThread   Computed// Set as parameter in 
> HiveConf. user can override in Advanced
> numConcurrentQueries  Computed// user can override in Advanced
> maxConcurrentQueries  Computed// Max value for Slider
> numLlapNodes  Computed// Can be lower for 
> small clusters. user can override in Advanced
> sliderAmContainerSize Computed// user can override in Advanced
> numExecutorsPerNode   Computed// user can override in Advanced  | 
> TODO: This and memPerThread are about the same when taking daemonSize, and 
> cache into considertaion
> cacheMemoryPerNodeComputed// user can override in 
> Advanced
> amFraction1   // Set 
> to 1 for Ambari controlled queued. | TODO Factor into concurrency for user 
> provided queues
> llapQueueFraction Computed// Computed by Ambari. 
> (Avoid changing if the current value is > computed value, and the user 
> specified the current value?)
> 
> 
> numClusterNodes   ClusterParameter
> nmMemoryPerNode   ClusterParameter
> nmCpusPerNode ClusterParameter
> minContainerSize  ClusterParameter
> 
> 
> CONSTANT DEFAULT_EXECUTOR_TO_AM_RATIO = 20;
> CONSTANT MIN_EXECUTOR_TO_AM_RATIO = 10;
> CONSTANT MAX_CONCURRENT_QUERIES = 32;
> 
> nmMemoryPerNodeNormalized = normalizeDown(nmMemoryPerNode, minContainerSize);
> 
> totalClusterCapacity = numClusterNodes * nmMemoryPerNodeNormalized;
> totalllapMemory = numRequestedLlapNodes * nmMemoryPerNodeNormalized;
> amCapacityAvailable = totalLlapMemory; // For the LLAP queue - the AM 
> fraction is set to 1
> 
> sliderAmSize -> Current calculations remain unchanged. (<=, < fixes)
> 
> llapMemoryTezAMsAndDaemons = totalllapMemory - sliderAmSize;
> FAIL("Not enough capacity available on the cluster to run LLAP") if 
> (llapMemoryTezAMsAndDaemons < 2 * minContainerSize);
> 
> tezAmContainerSize = (totalClusterCapacity, minContainerSize) {
>   desiredSize = { // This part is unchanged from current calculations.
>   if (totalClusterCapacity <= 4096) {
>   return 256;
>   } else if (totalClusterCapacity <= 73728) {
>   return 512;
>   } else {
>   return 1536;
>   }
>   }
>   return normalizeUp(desiredSize, minContainerSize);
> }
> 
> memoryPerThread = (nmMemoryPerNodeNormalized, nmCpusPerNode) { // TODO: Not 
> linear. e.g. 1024 to 1025 goes from 2 executors to 1.
>   if (userSpecifiedValue) {
>   return userSpecifiedValue;
>   } else if (nmMemoryPerNodeNormalized <= 1024) {
>   return Math.min(512, nmMemoryPerNodeNormalized)
>   } else if (nmMemoryPerNodeNormalized <= 4096 ) {
>   return 1024;
>   } else if (nmMemoryPerNodeNormalized <= 10240) {
>   return 2048;
>   } els

Re: Review Request 53801: AMBARI-18901. Use 'Number of LLAP Nodes' selected as the driver for LLAP config calculations.

2016-11-17 Thread Swapan Shridhar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53801/
---

(Updated Nov. 18, 2016, 2:34 a.m.)


Review request for Ambari, Alejandro Fernandez, Madhuvanthi Radhakrishnan, 
Sumit Mohanty, and Siddharth Seth.


Changes
---

- Updated the Ambari Upgrade code for hive configs which needs to be set to 
default values.
- Added few more suggestiosn as per discussion with Sid Seth on LLAP 
calculations.


Bugs: AMBARI-18901
https://issues.apache.org/jira/browse/AMBARI-18901


Repository: ambari


Description
---

AMBARI-18901. Use 'Number of LLAP Nodes' selected as the driver for LLAP config 
calculations.

Below is the calculation logic used:


**
**


---
For use with default setup - Ambari managed queue

Parameteres
numRequestedLlapNodes   UserInput
tezAmContainerSize  Computed
memoryPerThread Computed// Set as parameter in 
HiveConf. user can override in Advanced
numConcurrentQueriesComputed// user can override in Advanced
maxConcurrentQueriesComputed// Max value for Slider
numLlapNodesComputed// Can be lower for 
small clusters. user can override in Advanced
sliderAmContainerSize   Computed// user can override in Advanced
numExecutorsPerNode Computed// user can override in Advanced  | 
TODO: This and memPerThread are about the same when taking daemonSize, and 
cache into considertaion
cacheMemoryPerNode  Computed// user can override in 
Advanced
amFraction  1   // Set 
to 1 for Ambari controlled queued. | TODO Factor into concurrency for user 
provided queues
llapQueueFraction   Computed// Computed by Ambari. 
(Avoid changing if the current value is > computed value, and the user 
specified the current value?)


numClusterNodes ClusterParameter
nmMemoryPerNode ClusterParameter
nmCpusPerNode   ClusterParameter
minContainerSizeClusterParameter


CONSTANT DEFAULT_EXECUTOR_TO_AM_RATIO = 20;
CONSTANT MIN_EXECUTOR_TO_AM_RATIO = 10;
CONSTANT MAX_CONCURRENT_QUERIES = 32;

nmMemoryPerNodeNormalized = normalizeDown(nmMemoryPerNode, minContainerSize);

totalClusterCapacity = numClusterNodes * nmMemoryPerNodeNormalized;
totalllapMemory = numRequestedLlapNodes * nmMemoryPerNodeNormalized;
amCapacityAvailable = totalLlapMemory; // For the LLAP queue - the AM fraction 
is set to 1

sliderAmSize -> Current calculations remain unchanged. (<=, < fixes)

llapMemoryTezAMsAndDaemons = totalllapMemory - sliderAmSize;
FAIL("Not enough capacity available on the cluster to run LLAP") if 
(llapMemoryTezAMsAndDaemons < 2 * minContainerSize);

tezAmContainerSize = (totalClusterCapacity, minContainerSize) {
desiredSize = { // This part is unchanged from current calculations.
if (totalClusterCapacity <= 4096) {
return 256;
} else if (totalClusterCapacity <= 73728) {
return 512;
} else {
return 1536;
}
}
return normalizeUp(desiredSize, minContainerSize);
}

memoryPerThread = (nmMemoryPerNodeNormalized, nmCpusPerNode) { // TODO: Not 
linear. e.g. 1024 to 1025 goes from 2 executors to 1.
if (userSpecifiedValue) {
return userSpecifiedValue;
} else if (nmMemoryPerNodeNormalized <= 1024) {
return Math.min(512, nmMemoryPerNodeNormalized)
} else if (nmMemoryPerNodeNormalized <= 4096 ) {
return 1024;
} else if (nmMemoryPerNodeNormalized <= 10240) {
return 2048;
} else if (nmMemoryPerNodeNormalized <= 24576) {
return 3072;
} else {
return 4096;
}
}

numConcurrentQueries, maxConcurrentQueries = (nmMemoryPerNodeNormalized, 
nmCpusPerNode, memoryPerThread, numRequestedLlapNodes, 
llapMemoryTezAMsAndDaemons, tezAmContainerSize, amCapacityAvailable) {
maxExecutorsPerNode = getMaxExecutorsPerNode(nmMemoryPerNodeNormalized, 
nmCpusPerNode, memoryPerThread);
FAIL if maxExecutorsPerNode < 1;


// Default 1 AM for every 20 executor threads.
// The second part of the min calculates based on mem required for 
DEFAULT_EXECUTOR_TO_AM_RATIO executors + 1 AM, making use of total memory. 
However, it's po

Re: Review Request 53871: Perf: Add Hadoop Core services to PERF stack

2016-11-17 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53871/
---

(Updated Nov. 18, 2016, 1:56 a.m.)


Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Myroslav 
Papirkovskyy, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-18928
https://issues.apache.org/jira/browse/AMBARI-18928


Repository: ambari


Description (updated)
---

Right now the PERF stack has dummy services, but we need to add more complexity 
with services like Hadoop Core (HDFS, YARN, MR, ZK, HBASE) that have more 
config types and attributes.

I combined all of the scripts from common services, HDP 2.0.6, 2.1, 2.2, 2.3, 
2.4, 2.5, and 2.6 with the correct overrides for HDFS, YARN/MR, ZK, and HBASE 
to make a single stack version.


Diffs
-

  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/configuration/grumpy-site.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/package/scripts/dwarf.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/themes/theme.json
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-log4j.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-logsearch-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-policy.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-site.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-security.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metrics.json 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_client.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_master.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_regionserver.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/phoenix_queryserver.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/quicklinks/quicklinks.json
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/themes/theme.json
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/widgets.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/core-site.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-metrics2.properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-policy.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-log4j.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-logsearch-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-site.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-security.

Review Request 53871: Perf: Add Hadoop Core services to PERF stack

2016-11-17 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53871/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Myroslav 
Papirkovskyy, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-18928
https://issues.apache.org/jira/browse/AMBARI-18928


Repository: ambari


Description
---

Right now the PERF stack has dummy services, but we need to add more complexity 
with services like Hadoop Core (HDFS, YARN, MR, ZK, HBASE) that have more 
config types and attributes.


Diffs
-

  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/configuration/grumpy-site.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/package/scripts/dwarf.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/themes/theme.json
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-log4j.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-logsearch-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-policy.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-site.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-security.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metrics.json 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_client.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_master.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_regionserver.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/phoenix_queryserver.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/quicklinks/quicklinks.json
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/themes/theme.json
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/widgets.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/core-site.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-metrics2.properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-policy.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-log4j.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-logsearch-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-site.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-security.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ssl-client.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ssl-server.xml
 PRE-CREATION 

Re: Review Request 53868: AMBARI-18927: Unit test failure in ServiceComponentHostTest

2016-11-17 Thread Sid Wagle

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53868/#review156273
---


Ship it!




Ship It!

- Sid Wagle


On Nov. 18, 2016, 1:27 a.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53868/
> ---
> 
> (Updated Nov. 18, 2016, 1:27 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18927
> https://issues.apache.org/jira/browse/AMBARI-18927
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18927: Unit test failure in ServiceComponentHostTest
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
>  14a8de64b24a79027af510fac33910f7fae9d3f9 
> 
> Diff: https://reviews.apache.org/r/53868/diff/
> 
> 
> Testing
> ---
> 
> ** mvn test -DskipPythonTests -Dtest=ServiceComponentHostTest **
> 
> 
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 29.724 sec - 
> in org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest
> 
> Results :
> 
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 2
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 47.930s
> [INFO] Finished at: Thu Nov 17 17:21:49 PST 2016
> [INFO] Final Memory: 64M/813M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>



Re: Review Request 53868: AMBARI-18927: Unit test failure in ServiceComponentHostTest

2016-11-17 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53868/#review156272
---


Ship it!




Ship It!

- Alejandro Fernandez


On Nov. 18, 2016, 1:27 a.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53868/
> ---
> 
> (Updated Nov. 18, 2016, 1:27 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18927
> https://issues.apache.org/jira/browse/AMBARI-18927
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18927: Unit test failure in ServiceComponentHostTest
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
>  14a8de64b24a79027af510fac33910f7fae9d3f9 
> 
> Diff: https://reviews.apache.org/r/53868/diff/
> 
> 
> Testing
> ---
> 
> ** mvn test -DskipPythonTests -Dtest=ServiceComponentHostTest **
> 
> 
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 29.724 sec - 
> in org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest
> 
> Results :
> 
> Tests run: 13, Failures: 0, Errors: 0, Skipped: 2
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 47.930s
> [INFO] Finished at: Thu Nov 17 17:21:49 PST 2016
> [INFO] Final Memory: 64M/813M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>



Review Request 53868: AMBARI-18927: Unit test failure in ServiceComponentHostTest

2016-11-17 Thread Nahappan Somasundaram

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53868/
---

Review request for Ambari, Jonathan Hurley and Sumit Mohanty.


Bugs: AMBARI-18927
https://issues.apache.org/jira/browse/AMBARI-18927


Repository: ambari


Description
---

AMBARI-18927: Unit test failure in ServiceComponentHostTest


Diffs
-

  
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
 14a8de64b24a79027af510fac33910f7fae9d3f9 

Diff: https://reviews.apache.org/r/53868/diff/


Testing
---

** mvn test -DskipPythonTests -Dtest=ServiceComponentHostTest **


 T E S T S
---
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest
Tests run: 13, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 29.724 sec - 
in org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest

Results :

Tests run: 13, Failures: 0, Errors: 0, Skipped: 2

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 47.930s
[INFO] Finished at: Thu Nov 17 17:21:49 PST 2016
[INFO] Final Memory: 64M/813M
[INFO] 


Thanks,

Nahappan Somasundaram



Re: Review Request 53722: AMBARI-18425:Support PAM as an authentication option for Ranger in Ambari

2016-11-17 Thread Shi Wang


> On Nov. 17, 2016, 1:04 p.m., Gautam Borad wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py,
> >  line 163
> > 
> >
> > Will this affect distributions where the PAM configs are stored in a 
> > single /etc/pam.conf file instead of /etc/pam.d/* files?

Hi Gautam, I checked CentOS, Oracle, RedHat, SLES and Ubuntu, they are all 
using /etc/pam.d directory, I think only very small portion of distribution and 
maybe very old version are using /etc/pam.conf, but since I could not find a 
machine to do test so probably just abandon the circumstance of /etc/pam.conf 
for now. If any issue because of this hitten in the future we can fix easily 
based on this patch.


- Shi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53722/#review156199
---


On Nov. 17, 2016, 9:05 p.m., Shi Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53722/
> ---
> 
> (Updated Nov. 17, 2016, 9:05 p.m.)
> 
> 
> Review request for Ambari and Robert Levas.
> 
> 
> Bugs: trunk
> https://issues.apache.org/jira/browse/trunk
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger-842 has added PAM support for ranger, we need to add this part to 
> ambari, to do automatic setup for ranger to use PAM authentication.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  6462495 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_admin_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_remote_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
>  c0e537b 
>   ambari-web/app/data/HDP2.3/site_properties.js 9ae68df 
> 
> Diff: https://reviews.apache.org/r/53722/diff/
> 
> 
> Testing
> ---
> 
> In this patch, the default value for ranger-admin module is
> authsufficientpam_unix.so
> authsufficientpam_sss.so
> account sufficientpam_unix.so
> account sufficientpam_sss.so
> 1. Create a Unix User shiwang, and sync unix user to Ranger, restart 
> ranger-admin as root and login to ranger using shiwang, it will success.
> 2. Change ranger-admin pam file to 
> authsufficientpam_deny.so
> account sufficientpam_deny.so
> and login using shiwang it will fall.
> 3. Change ranger-admin pam file to 
> authsufficientpam_ldap.so
> account sufficientpam_ldap.so
> and use a ldap user that already sync in ranger(it will show user not synced 
> error if not synced in ranger) login will success.
> 4. Configure sssd with ldap and using the synced user from ldap login will 
> success.
> 
> 
> Thanks,
> 
> Shi Wang
> 
>



Re: Review Request 53722: AMBARI-18425:Support PAM as an authentication option for Ranger in Ambari

2016-11-17 Thread Shi Wang


> On Nov. 17, 2016, 12:57 p.m., Mugdha Varadkar wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py,
> >  line 164
> > 
> >
> > What if the directory is not present? Should it just ignore or create 
> > the directory?

Add Logger.error("Unable to use PAM authentication, /etc/pam.d/ directory does 
not exist.") to catch this condition. Because if create a /etc/pam.d directory 
for ranger pam authentication may cause issue to the os since if /etc/pam.d 
exsits it may ignore /etc/pam.conf file.


- Shi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53722/#review156198
---


On Nov. 17, 2016, 9:05 p.m., Shi Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53722/
> ---
> 
> (Updated Nov. 17, 2016, 9:05 p.m.)
> 
> 
> Review request for Ambari and Robert Levas.
> 
> 
> Bugs: trunk
> https://issues.apache.org/jira/browse/trunk
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger-842 has added PAM support for ranger, we need to add this part to 
> ambari, to do automatic setup for ranger to use PAM authentication.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  6462495 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_admin_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_remote_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
>  c0e537b 
>   ambari-web/app/data/HDP2.3/site_properties.js 9ae68df 
> 
> Diff: https://reviews.apache.org/r/53722/diff/
> 
> 
> Testing
> ---
> 
> In this patch, the default value for ranger-admin module is
> authsufficientpam_unix.so
> authsufficientpam_sss.so
> account sufficientpam_unix.so
> account sufficientpam_sss.so
> 1. Create a Unix User shiwang, and sync unix user to Ranger, restart 
> ranger-admin as root and login to ranger using shiwang, it will success.
> 2. Change ranger-admin pam file to 
> authsufficientpam_deny.so
> account sufficientpam_deny.so
> and login using shiwang it will fall.
> 3. Change ranger-admin pam file to 
> authsufficientpam_ldap.so
> account sufficientpam_ldap.so
> and use a ldap user that already sync in ranger(it will show user not synced 
> error if not synced in ranger) login will success.
> 4. Configure sssd with ldap and using the synced user from ldap login will 
> success.
> 
> 
> Thanks,
> 
> Shi Wang
> 
>



Re: Review Request 53722: AMBARI-18425:Support PAM as an authentication option for Ranger in Ambari

2016-11-17 Thread Shi Wang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53722/
---

(Updated Nov. 17, 2016, 9:05 p.m.)


Review request for Ambari and Robert Levas.


Bugs: trunk
https://issues.apache.org/jira/browse/trunk


Repository: ambari


Description
---

Ranger-842 has added PAM support for ranger, we need to add this part to 
ambari, to do automatic setup for ranger to use PAM authentication.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 6462495 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_admin_pam.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_remote_pam.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
 c0e537b 
  ambari-web/app/data/HDP2.3/site_properties.js 9ae68df 

Diff: https://reviews.apache.org/r/53722/diff/


Testing
---

In this patch, the default value for ranger-admin module is
authsufficientpam_unix.so
authsufficientpam_sss.so
account sufficientpam_unix.so
account sufficientpam_sss.so
1. Create a Unix User shiwang, and sync unix user to Ranger, restart 
ranger-admin as root and login to ranger using shiwang, it will success.
2. Change ranger-admin pam file to 
authsufficientpam_deny.so
account sufficientpam_deny.so
and login using shiwang it will fall.
3. Change ranger-admin pam file to 
authsufficientpam_ldap.so
account sufficientpam_ldap.so
and use a ldap user that already sync in ranger(it will show user not synced 
error if not synced in ranger) login will success.
4. Configure sssd with ldap and using the synced user from ldap login will 
success.


Thanks,

Shi Wang



Re: Review Request 53861: Previous page link enabled on empty table

2016-11-17 Thread Aleksandr Kovalenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53861/#review156230
---


Ship it!




Ship It!

- Aleksandr Kovalenko


On Ноя. 17, 2016, 7:09 п.п., Andrii Tkach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53861/
> ---
> 
> (Updated Ноя. 17, 2016, 7:09 п.п.)
> 
> 
> Review request for Ambari and Aleksandr Kovalenko.
> 
> 
> Bugs: AMBARI-18925
> https://issues.apache.org/jira/browse/AMBARI-18925
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to reproduce:
> Open table (Hosts, Config Groups)
> Apply some filters in order to have the empty table
> Actual result:
> Link to previous page enabled.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/views/common/table_view.js 7d6419a 
> 
> Diff: https://reviews.apache.org/r/53861/diff/
> 
> 
> Testing
> ---
> 
> 30621 tests complete (32 seconds)
> 151 tests pending
> 
> 
> Thanks,
> 
> Andrii Tkach
> 
>



Review Request 53861: Previous page link enabled on empty table

2016-11-17 Thread Andrii Tkach

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53861/
---

Review request for Ambari and Aleksandr Kovalenko.


Bugs: AMBARI-18925
https://issues.apache.org/jira/browse/AMBARI-18925


Repository: ambari


Description
---

Steps to reproduce:
Open table (Hosts, Config Groups)
Apply some filters in order to have the empty table
Actual result:
Link to previous page enabled.


Diffs
-

  ambari-web/app/views/common/table_view.js 7d6419a 

Diff: https://reviews.apache.org/r/53861/diff/


Testing
---

30621 tests complete (32 seconds)
151 tests pending


Thanks,

Andrii Tkach



Re: Review Request 53722: AMBARI-18425:Support PAM as an authentication option for Ranger in Ambari

2016-11-17 Thread Shi Wang


> On Nov. 16, 2016, 2:11 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py,
> >  lines 164-165
> > 
> >
> > Is there a better way to _discover_ what version of PAM we are 
> > configuring. This seems a bit problematic.

Hi Robert,
I look at pam official document and ranger 842 implement again, and find the 
implement in ranger-842 only works when the pam file is under /etc/pam.d 
because the application name is fixed: ranger-admin, pam authentication will 
find this file name under /etc/pam.d. And pam.conf is a configuration file 
which could not be used in this way. I made addressing this issue in the python 
script.


- Shi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53722/#review156059
---


On Nov. 17, 2016, 7:47 a.m., Shi Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53722/
> ---
> 
> (Updated Nov. 17, 2016, 7:47 a.m.)
> 
> 
> Review request for Ambari and Robert Levas.
> 
> 
> Bugs: trunk
> https://issues.apache.org/jira/browse/trunk
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger-842 has added PAM support for ranger, we need to add this part to 
> ambari, to do automatic setup for ranger to use PAM authentication.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  6462495 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_admin_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_remote_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
>  c0e537b 
>   ambari-web/app/data/HDP2.3/site_properties.js 9ae68df 
> 
> Diff: https://reviews.apache.org/r/53722/diff/
> 
> 
> Testing
> ---
> 
> In this patch, the default value for ranger-admin module is
> authsufficientpam_unix.so
> authsufficientpam_sss.so
> account sufficientpam_unix.so
> account sufficientpam_sss.so
> 1. Create a Unix User shiwang, and sync unix user to Ranger, restart 
> ranger-admin as root and login to ranger using shiwang, it will success.
> 2. Change ranger-admin pam file to 
> authsufficientpam_deny.so
> account sufficientpam_deny.so
> and login using shiwang it will fall.
> 3. Change ranger-admin pam file to 
> authsufficientpam_ldap.so
> account sufficientpam_ldap.so
> and use a ldap user that already sync in ranger(it will show user not synced 
> error if not synced in ranger) login will success.
> 4. Configure sssd with ldap and using the synced user from ldap login will 
> success.
> 
> 
> Thanks,
> 
> Shi Wang
> 
>



Re: Review Request 53854: Agent Auto Restart Doesn't Release Ping Port

2016-11-17 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53854/
---

(Updated Ноя. 17, 2016, 5:09 п.п.)


Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-18922
https://issues.apache.org/jira/browse/AMBARI-18922


Repository: ambari


Description
---

Ambari agent auto-restart fails


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 5d3607a 
  ambari-agent/src/main/python/ambari_agent/main.py 04e0c2c 

Diff: https://reviews.apache.org/r/53854/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen



Review Request 53854: Agent Auto Restart Doesn't Release Ping Port

2016-11-17 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53854/
---

Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-18922
https://issues.apache.org/jira/browse/AMBARI-18922


Repository: ambari


Description
---

Ambari agent auto-restart fails


Diffs
-

  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 5d3607a 
  ambari-agent/src/main/python/ambari_agent/main.py 04e0c2c 

Diff: https://reviews.apache.org/r/53854/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen



Re: Review Request 53809: Remove Unnecessary Locks Inside Of Config Business Object Implementations

2016-11-17 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53809/#review156210
---


Ship it!




Ship It!

- Robert Levas


On Nov. 16, 2016, 6:17 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53809/
> ---
> 
> (Updated Nov. 16, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Nate Cole, Robert Levas, Robert Nettleton, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-18906
> https://issues.apache.org/jira/browse/AMBARI-18906
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {{ConfigImpl}} uses internal locks around state which can lead to deadlocks 
> when this in called in the context of other business objects. In most cases, 
> this state-full data does not need locks placed around it. 
> 
> {{ConfigImpl}} should be changed to:
> - No longer store entity information
> - No longer require locks around primitive state
> 
> One of the biggest changes was made to how configs are created. We used to 
> use this anti-pattern:
> - Create config in-memory from factory (tracking persistence state)
> - Set the tag on the config instance
> - Persist the config
> - Add config to cluster
> 
> That was ridiculous - the factory should handle the creation of the entity 
> with the required data. In a few spots, I had caught bugs where we had 
> duplicated a ton of cluster entity logic when adding configs which was 
> missing some steps.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b04fdd7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
>  96bb8f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
>  5459ddb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixLzoCodecPath.java
>  ffa21ab 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixOozieAdminUsers.java
>  3a06476 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/HBaseConfigCalculation.java
>  7f6d4b1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/HBaseEnvMaxDirectMemorySizeAction.java
>  b238bca 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/HiveEnvClasspathAction.java
>  0e10160 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/HiveZKQuorumConfigAction.java
>  0ade30b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/OozieConfigCalculation.java
>  4da67ca 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/RangerConfigCalculation.java
>  ff4a20e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/RangerKerberosConfigCalculation.java
>  ba0da79 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/SparkShufflePropertyConfig.java
>  299a373 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/YarnConfigCalculation.java
>  feefcaf 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Config.java 
> b35aad9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigFactory.java 
> eaf68aa 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java 
> 28bcd5f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
>  9917720 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
>  83f8470 
>   
> ambari-server/src/main/java/org/apache/ambari/server/update/HostUpdateHelper.java
>  6a8057c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java
>  ffca51d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionSchedulerThreading.java
>  90a4421 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatTestHelper.java
>  43503fa 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
>  76ab45c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
>  6533e1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java
>  e54a117 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  0fdaa46 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  96810cf 
>   
> ambari-server/src/test/java/org

Re: Review Request 53847: ambari-server.pid might contain incorrect value

2016-11-17 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53847/#review156203
---


Ship it!




Ship It!

- Sumit Mohanty


On Nov. 17, 2016, 2:24 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53847/
> ---
> 
> (Updated Nov. 17, 2016, 2:24 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-18760
> https://issues.apache.org/jira/browse/AMBARI-18760
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server status output :
> Using python  /usr/bin/python
> Ambari-server status
> WARNING: /var/run/ambari-server/ambari-server.pid is corrupt. Removing
> Ambari Server 'status' completed with warnings.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUtils.py 069e24d 
>   ambari-server/src/main/python/ambari_server/utils.py 5f0a64d 
> 
> Diff: https://reviews.apache.org/r/53847/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>



Re: Review Request 53846: Kerberos wizard: Show recommendation API failure message in a popup

2016-11-17 Thread Oleg Nechiporenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53846/#review156201
---


Ship it!




Ship It!

- Oleg Nechiporenko


On Nov. 17, 2016, 2:47 p.m., Aleksandr Kovalenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53846/
> ---
> 
> (Updated Nov. 17, 2016, 2:47 p.m.)
> 
> 
> Review request for Ambari and Oleg Nechiporenko.
> 
> 
> Bugs: AMBARI-18920
> https://issues.apache.org/jira/browse/AMBARI-18920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If recommendation API fails on configure identities page then page keeps on 
> showing spinner. Instead user should see the failure message
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/admin/kerberos/step4_controller.js eebe133 
>   ambari-web/app/controllers/main/admin/kerberos/wizard_controller.js 918742d 
>   ambari-web/test/controllers/main/admin/kerberos/step4_controller_test.js 
> 86e126e 
> 
> Diff: https://reviews.apache.org/r/53846/diff/
> 
> 
> Testing
> ---
> 
> 30621 tests complete (36 seconds)
>   151 tests pending
> 
> 
> Thanks,
> 
> Aleksandr Kovalenko
> 
>



Re: Review Request 53727: Provide SSL related configurations for Ranger-Tagsync.

2016-11-17 Thread Vishal Suvagia

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53727/
---

(Updated Nov. 17, 2016, 2:51 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jaimin Jetly, 
Jayush Luniya, Mugdha Varadkar, and Velmurugan Periasamy.


Changes
---

Updated patch to include Tagsync configs when Atlas is ssl enabled and moved 
files to common-services from stacks, and included test-cases.


Bugs: AMBARI-18874
https://issues.apache.org/jira/browse/AMBARI-18874


Repository: ambari


Description (updated)
---

Need to provide ssl related configurations for Ranger-Tagsync service.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 3431495 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 6debaf1 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_tagsync.py
 1efa7e9 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 f1b4797 
  
ambari-server/src/main/resources/common-services/RANGER/0.7.0/configuration/atlas-tagsync-policymgr-ssl.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/RANGER/0.7.0/configuration/ranger-tagsync-policymgr-ssl.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/RANGER/0.7.0/configuration/ranger-tagsync-site.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/RANGER/0.7.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 dd87b72 
  
ambari-server/src/main/resources/stacks/HDP/2.6/services/RANGER/configuration/ranger-tagsync-site.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/RANGER/metainfo.xml 
6287e00 
  ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_tagsync.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.6/configs/ranger-admin-default.json 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.6/configs/ranger-admin-secured.json 
PRE-CREATION 

Diff: https://reviews.apache.org/r/53727/diff/


Testing (updated)
---

Tested Ranger with Tagsync installation on CentOS 6.

Verified Unit-tests:

test_configure_default (test_ranger_tagsync.TestRangerTagsync) ... 2016-11-17 
20:01:42,649 - Stack Feature Version Info: stack_version=2.6, 
version=2.6.0.0-111, current_cluster_version=2.6.0.0-111 -> 2.6.0.0-111
2016-11-17 20:01:42,670 - Stack Feature Version Info: stack_version=2.6, 
version=2.6.0.0-111, current_cluster_version=2.6.0.0-111 -> 2.6.0.0-111
ok
test_configure_secured (test_ranger_tagsync.TestRangerTagsync) ... ok
test_start_default (test_ranger_tagsync.TestRangerTagsync) ... 2016-11-17 
20:01:42,691 - Stack Feature Version Info: stack_version=2.6, 
version=2.6.0.0-111, current_cluster_version=2.6.0.0-111 -> 2.6.0.0-111
ok
test_stop_default (test_ranger_tagsync.TestRangerTagsync) ... 2016-11-17 
20:01:42,712 - Stack Feature Version Info: stack_version=2.6, 
version=2.6.0.0-111, current_cluster_version=2.6.0.0-111 -> 2.6.0.0-111
ok


test_configure_default (test_ranger_tagsync.TestRangerTagsync) ... 2016-11-17 
20:02:05,297 - Stack Feature Version Info: stack_version=2.5, 
version=2.5.0.0-801, current_cluster_version=2.5.0.0-801 -> 2.5.0.0-801
ok
test_configure_secured (test_ranger_tagsync.TestRangerTagsync) ... 2016-11-17 
20:02:05,318 - Stack Feature Version Info: stack_version=2.5, 
version=2.5.0.0-801, current_cluster_version=2.5.0.0-801 -> 2.5.0.0-801
ok
test_start_default (test_ranger_tagsync.TestRangerTagsync) ... 2016-11-17 
20:02:05,336 - Stack Feature Version Info: stack_version=2.5, 
version=2.5.0.0-801, current_cluster_version=2.5.0.0-801 -> 2.5.0.0-801
ok
test_stop_default (test_ranger_tagsync.TestRangerTagsync) ... 2016-11-17 
20:02:05,355 - Stack Feature Version Info: stack_version=2.5, 
version=2.5.0.0-801, current_cluster_version=2.5.0.0-801 -> 2.5.0.0-801
ok


Thanks,

Vishal Suvagia



Review Request 53846: Kerberos wizard: Show recommendation API failure message in a popup

2016-11-17 Thread Aleksandr Kovalenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53846/
---

Review request for Ambari and Oleg Nechiporenko.


Bugs: AMBARI-18920
https://issues.apache.org/jira/browse/AMBARI-18920


Repository: ambari


Description
---

If recommendation API fails on configure identities page then page keeps on 
showing spinner. Instead user should see the failure message


Diffs
-

  ambari-web/app/controllers/main/admin/kerberos/step4_controller.js eebe133 
  ambari-web/app/controllers/main/admin/kerberos/wizard_controller.js 918742d 
  ambari-web/test/controllers/main/admin/kerberos/step4_controller_test.js 
86e126e 

Diff: https://reviews.apache.org/r/53846/diff/


Testing
---

30621 tests complete (36 seconds)
  151 tests pending


Thanks,

Aleksandr Kovalenko



Re: Review Request 53847: ambari-server.pid might contain incorrect value

2016-11-17 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53847/#review156200
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Nov. 17, 2016, 4:24 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53847/
> ---
> 
> (Updated Nov. 17, 2016, 4:24 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-18760
> https://issues.apache.org/jira/browse/AMBARI-18760
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server status output :
> Using python  /usr/bin/python
> Ambari-server status
> WARNING: /var/run/ambari-server/ambari-server.pid is corrupt. Removing
> Ambari Server 'status' completed with warnings.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverUtils.py 069e24d 
>   ambari-server/src/main/python/ambari_server/utils.py 5f0a64d 
> 
> Diff: https://reviews.apache.org/r/53847/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>



Re: Review Request 53847: ambari-server.pid might contain incorrect value

2016-11-17 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53847/
---

(Updated Ноя. 17, 2016, 2:24 п.п.)


Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-18760
https://issues.apache.org/jira/browse/AMBARI-18760


Repository: ambari


Description
---

ambari-server status output :
Using python  /usr/bin/python
Ambari-server status
WARNING: /var/run/ambari-server/ambari-server.pid is corrupt. Removing
Ambari Server 'status' completed with warnings.


Diffs (updated)
-

  ambari-server/src/main/python/ambari_server/serverUtils.py 069e24d 
  ambari-server/src/main/python/ambari_server/utils.py 5f0a64d 

Diff: https://reviews.apache.org/r/53847/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen



Review Request 53847: ambari-server.pid might contain incorrect value

2016-11-17 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53847/
---

Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-18760
https://issues.apache.org/jira/browse/AMBARI-18760


Repository: ambari


Description
---

ambari-server status output :
Using python  /usr/bin/python
Ambari-server status
WARNING: /var/run/ambari-server/ambari-server.pid is corrupt. Removing
Ambari Server 'status' completed with warnings.


Diffs
-

  ambari-server/src/main/python/ambari_server/serverUtils.py 069e24d 
  ambari-server/src/main/python/ambari_server/utils.py 5f0a64d 

Diff: https://reviews.apache.org/r/53847/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen



Re: Review Request 53722: AMBARI-18425:Support PAM as an authentication option for Ranger in Ambari

2016-11-17 Thread Gautam Borad

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53722/#review156199
---




ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 (line 163)


Will this affect distributions where the PAM configs are stored in a single 
/etc/pam.conf file instead of /etc/pam.d/* files?


- Gautam Borad


On Nov. 17, 2016, 7:47 a.m., Shi Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53722/
> ---
> 
> (Updated Nov. 17, 2016, 7:47 a.m.)
> 
> 
> Review request for Ambari and Robert Levas.
> 
> 
> Bugs: trunk
> https://issues.apache.org/jira/browse/trunk
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger-842 has added PAM support for ranger, we need to add this part to 
> ambari, to do automatic setup for ranger to use PAM authentication.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  6462495 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_admin_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_remote_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
>  c0e537b 
>   ambari-web/app/data/HDP2.3/site_properties.js 9ae68df 
> 
> Diff: https://reviews.apache.org/r/53722/diff/
> 
> 
> Testing
> ---
> 
> In this patch, the default value for ranger-admin module is
> authsufficientpam_unix.so
> authsufficientpam_sss.so
> account sufficientpam_unix.so
> account sufficientpam_sss.so
> 1. Create a Unix User shiwang, and sync unix user to Ranger, restart 
> ranger-admin as root and login to ranger using shiwang, it will success.
> 2. Change ranger-admin pam file to 
> authsufficientpam_deny.so
> account sufficientpam_deny.so
> and login using shiwang it will fall.
> 3. Change ranger-admin pam file to 
> authsufficientpam_ldap.so
> account sufficientpam_ldap.so
> and use a ldap user that already sync in ranger(it will show user not synced 
> error if not synced in ranger) login will success.
> 4. Configure sssd with ldap and using the synced user from ldap login will 
> success.
> 
> 
> Thanks,
> 
> Shi Wang
> 
>



Re: Review Request 53722: AMBARI-18425:Support PAM as an authentication option for Ranger in Ambari

2016-11-17 Thread Mugdha Varadkar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53722/#review156198
---




ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 (line 164)


What if the directory is not present? Should it just ignore or create the 
directory?


- Mugdha Varadkar


On Nov. 17, 2016, 7:47 a.m., Shi Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53722/
> ---
> 
> (Updated Nov. 17, 2016, 7:47 a.m.)
> 
> 
> Review request for Ambari and Robert Levas.
> 
> 
> Bugs: trunk
> https://issues.apache.org/jira/browse/trunk
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger-842 has added PAM support for ranger, we need to add this part to 
> ambari, to do automatic setup for ranger to use PAM authentication.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  6462495 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_admin_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/templates/ranger_remote_pam.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
>  c0e537b 
>   ambari-web/app/data/HDP2.3/site_properties.js 9ae68df 
> 
> Diff: https://reviews.apache.org/r/53722/diff/
> 
> 
> Testing
> ---
> 
> In this patch, the default value for ranger-admin module is
> authsufficientpam_unix.so
> authsufficientpam_sss.so
> account sufficientpam_unix.so
> account sufficientpam_sss.so
> 1. Create a Unix User shiwang, and sync unix user to Ranger, restart 
> ranger-admin as root and login to ranger using shiwang, it will success.
> 2. Change ranger-admin pam file to 
> authsufficientpam_deny.so
> account sufficientpam_deny.so
> and login using shiwang it will fall.
> 3. Change ranger-admin pam file to 
> authsufficientpam_ldap.so
> account sufficientpam_ldap.so
> and use a ldap user that already sync in ranger(it will show user not synced 
> error if not synced in ranger) login will success.
> 4. Configure sssd with ldap and using the synced user from ldap login will 
> success.
> 
> 
> Thanks,
> 
> Shi Wang
> 
>



Re: Review Request 53844: Oozie Server fails to start due to timeout

2016-11-17 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53844/#review156197
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Nov. 17, 2016, 1:57 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53844/
> ---
> 
> (Updated Nov. 17, 2016, 1:57 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18919
> https://issues.apache.org/jira/browse/AMBARI-18919
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Because of slow env every operation took so much time:
> 
> 
> 
> untar - 1min
> sharelib create - 7min
> start oozie cmd - 1min
> shareliblist hive - 7 min (was waitting for Oozie to actually start)
> hdfs directories copy - 3min 
> 
> 
> Looks like we have to increase timeout here, so Oozie start can succeed on
> slow envs
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml 
> 8282240 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/metainfo.xml 
> 96ac9ce 
> 
> Diff: https://reviews.apache.org/r/53844/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>



Review Request 53844: Oozie Server fails to start due to timeout

2016-11-17 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53844/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18919
https://issues.apache.org/jira/browse/AMBARI-18919


Repository: ambari


Description
---

Because of slow env every operation took so much time:



untar - 1min
sharelib create - 7min
start oozie cmd - 1min
shareliblist hive - 7 min (was waitting for Oozie to actually start)
hdfs directories copy - 3min 


Looks like we have to increase timeout here, so Oozie start can succeed on
slow envs


Diffs
-

  ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml 
8282240 
  ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/metainfo.xml 
96ac9ce 

Diff: https://reviews.apache.org/r/53844/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk



Re: Review Request 53843: Zeppelin view doesn't work with JDK 1.8_91+

2016-11-17 Thread Prabhjyot Singh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53843/
---

(Updated Nov. 17, 2016, 10:09 a.m.)


Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush 
Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and Sumit Mohanty.


Bugs: AMBARI-18918
https://issues.apache.org/jira/browse/AMBARI-18918


Repository: ambari


Description
---

At ambari main page, click on the box icon next to the Admin tab on top right 
corner. Then click Zeppelin View. It will give the following error.


{code}
HTTP ERROR 500
Problem accessing /views/ZEPPELIN/1.0.0/AUTO_ZEPPELIN_INSTANCE/. Reason: 
Server Error
Caused by:
org.apache.jasper.JasperException: PWC6033: Unable to compile class for JSP
PWC6199: Generated servlet error:
The type java.io.ObjectInputStream cannot be resolved. It is indirectly 
referenced from required .class files
at 
org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:123)
at 
org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:296)
at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:376)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:437)
at 
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:608)
at 
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:360)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:276)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:103)
at 
org.apache.ambari.view.zeppelin.ZeppelinServlet.doGet(ZeppelinServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:286)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuth

Re: Review Request 53843: Zeppelin view doesn't work with JDK 1.8_91+

2016-11-17 Thread Prabhjyot Singh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53843/
---

(Updated Nov. 17, 2016, 9:44 a.m.)


Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush 
Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and Sumit Mohanty.


Bugs: AMBARI-18918
https://issues.apache.org/jira/browse/AMBARI-18918


Repository: ambari


Description (updated)
---

At ambari main page, click on the box icon next to the Admin tab on top right 
corner. Then click Zeppelin View. It will give the following error.


{code}
HTTP ERROR 500
Problem accessing /views/ZEPPELIN/1.0.0/AUTO_ZEPPELIN_INSTANCE/. Reason: 
Server Error
Caused by:
org.apache.jasper.JasperException: PWC6033: Unable to compile class for JSP
PWC6199: Generated servlet error:
The type java.io.ObjectInputStream cannot be resolved. It is indirectly 
referenced from required .class files
at 
org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:123)
at 
org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:296)
at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:376)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:437)
at 
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:608)
at 
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:360)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:276)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:103)
at 
org.apache.ambari.view.zeppelin.ZeppelinServlet.doGet(ZeppelinServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:286)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(

Review Request 53843: Zeppelin view doesn't work with JDK 1.8_91+

2016-11-17 Thread Prabhjyot Singh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53843/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush 
Luniya, Nitiraj Rathore, Rohit Choudhary, Renjith Kamath, and Sumit Mohanty.


Bugs: AMBARI-18918
https://issues.apache.org/jira/browse/AMBARI-18918


Repository: ambari


Description
---

At ambari main page, click on the box icon next to the Admin tab on top right 
corner. Then click Zeppelin View. It will give the following error.


{code}
HTTP ERROR 500
Problem accessing /views/ZEPPELIN/1.0.0/AUTO_ZEPPELIN_INSTANCE/. Reason: 
Server Error
Caused by:
org.apache.jasper.JasperException: PWC6033: Unable to compile class for JSP
PWC6199: Generated servlet error:
The type java.io.ObjectInputStream cannot be resolved. It is indirectly 
referenced from required .class files
at 
org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:123)
at 
org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:296)
at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:376)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:437)
at 
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:608)
at 
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:360)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:276)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:103)
at 
org.apache.ambari.view.zeppelin.ZeppelinServlet.doGet(ZeppelinServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:286)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at 
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201)
at 
org.apa

Re: Review Request 53780: create separate RPM / DEB package for views during build process.

2016-11-17 Thread Ashwin Rajeev

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53780/#review156192
---


Ship it!




Ship It!

- Ashwin Rajeev


On Nov. 15, 2016, 11:54 a.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53780/
> ---
> 
> (Updated Nov. 15, 2016, 11:54 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav 
> Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin 
> Rajeev.
> 
> 
> Bugs: AMBARI-18873
> https://issues.apache.org/jira/browse/AMBARI-18873
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Earlier : view jars were shipped with the ambari-server package. 
> Now : separate rpm and deb files can be created 
> (ambari-views-package_$VERSION.deb, ambari-views-package-$VERSION.x86_64.rpm) 
> which can be shipped and installed separately.
> 
> each view (module) in contrib/views can copy its jar into the parent 
> (ambari-contrib-view project) target/views-jars folder and when mvn runs in 
> ambari-views-package, it creates an RPM and DEB file packaging all the jars.
> 
> created RPM location : 
> $AMBARI/contrib/views/ambari-views-package/target/rpm/ambari-views-package/RPMS/x86_64/ambari-views-package-2.5.0.0-0.x86_64.rpm
> create DEB location : 
> $AMBARI/contrib/views/ambari-views-package/target/ambari-views-package_2.5.0.0-0.deb
> 
> The order of module ambari-views-package in pom.xml of ambari-contrib-views 
> should be last as it will create the package.
> 
> command to create deb and rpm
> $ cd $AMBARI/contrib/views
> $ mvn clean package rpm:rpm jdeb:jdeb install
> 
> 
> Diffs
> -
> 
>   contrib/views/ambari-views-package/pom.xml PRE-CREATION 
>   contrib/views/ambari-views-package/src/main/package/deb/control/control 
> PRE-CREATION 
>   contrib/views/capacity-scheduler/pom.xml 7b7cdc9 
>   contrib/views/files/pom.xml 67beef9 
>   contrib/views/hawq/pom.xml 4fc543f 
>   contrib/views/hive-next/pom.xml 46deaf3 
>   contrib/views/hive/pom.xml 2814049 
>   contrib/views/hueambarimigration/pom.xml 21837c3 
>   contrib/views/jobs/pom.xml 8460c5b 
>   contrib/views/pig/pom.xml d2aada7 
>   contrib/views/pom.xml 94f8421 
>   contrib/views/slider/pom.xml 0ddcb84 
>   contrib/views/storm/pom.xml 38ec8ba 
>   contrib/views/tez/pom.xml 26b0c49 
>   contrib/views/wfmanager/pom.xml e03e656 
>   contrib/views/zeppelin/pom.xml 52efdce 
> 
> Diff: https://reviews.apache.org/r/53780/diff/
> 
> 
> Testing
> ---
> 
> manual testing of creation of
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>



Re: Review Request 53780: create separate RPM / DEB package for views during build process.

2016-11-17 Thread Prabhjyot Singh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53780/#review156191
---


Ship it!




Ship It!

- Prabhjyot Singh


On Nov. 15, 2016, 11:54 a.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53780/
> ---
> 
> (Updated Nov. 15, 2016, 11:54 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav 
> Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin 
> Rajeev.
> 
> 
> Bugs: AMBARI-18873
> https://issues.apache.org/jira/browse/AMBARI-18873
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Earlier : view jars were shipped with the ambari-server package. 
> Now : separate rpm and deb files can be created 
> (ambari-views-package_$VERSION.deb, ambari-views-package-$VERSION.x86_64.rpm) 
> which can be shipped and installed separately.
> 
> each view (module) in contrib/views can copy its jar into the parent 
> (ambari-contrib-view project) target/views-jars folder and when mvn runs in 
> ambari-views-package, it creates an RPM and DEB file packaging all the jars.
> 
> created RPM location : 
> $AMBARI/contrib/views/ambari-views-package/target/rpm/ambari-views-package/RPMS/x86_64/ambari-views-package-2.5.0.0-0.x86_64.rpm
> create DEB location : 
> $AMBARI/contrib/views/ambari-views-package/target/ambari-views-package_2.5.0.0-0.deb
> 
> The order of module ambari-views-package in pom.xml of ambari-contrib-views 
> should be last as it will create the package.
> 
> command to create deb and rpm
> $ cd $AMBARI/contrib/views
> $ mvn clean package rpm:rpm jdeb:jdeb install
> 
> 
> Diffs
> -
> 
>   contrib/views/ambari-views-package/pom.xml PRE-CREATION 
>   contrib/views/ambari-views-package/src/main/package/deb/control/control 
> PRE-CREATION 
>   contrib/views/capacity-scheduler/pom.xml 7b7cdc9 
>   contrib/views/files/pom.xml 67beef9 
>   contrib/views/hawq/pom.xml 4fc543f 
>   contrib/views/hive-next/pom.xml 46deaf3 
>   contrib/views/hive/pom.xml 2814049 
>   contrib/views/hueambarimigration/pom.xml 21837c3 
>   contrib/views/jobs/pom.xml 8460c5b 
>   contrib/views/pig/pom.xml d2aada7 
>   contrib/views/pom.xml 94f8421 
>   contrib/views/slider/pom.xml 0ddcb84 
>   contrib/views/storm/pom.xml 38ec8ba 
>   contrib/views/tez/pom.xml 26b0c49 
>   contrib/views/wfmanager/pom.xml e03e656 
>   contrib/views/zeppelin/pom.xml 52efdce 
> 
> Diff: https://reviews.apache.org/r/53780/diff/
> 
> 
> Testing
> ---
> 
> manual testing of creation of
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>



Re: Review Request 53815: Log Search - make logsearch rest api docs as default under /docs/ instead of petstore

2016-11-17 Thread Miklos Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53815/#review156188
---


Ship it!




Ship It!

- Miklos Gergely


On Nov. 16, 2016, 5:58 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53815/
> ---
> 
> (Updated Nov. 16, 2016, 5:58 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18909
> https://issues.apache.org/jira/browse/AMBARI-18909
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> if a user want to read the rest api docs, its not enough to go under /docs/, 
> its needed to add the url of the generaterd swagger json/yaml on the swagger 
> UI.
> instead of index.html of swagger-ui, i added an own welcome page for /docs/ 
> servlet. (as swagger.html) which uses /api/v1/swagger.yaml by default.
> 
> also contains a small fix (images should be accessible without authentication)
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  5a7cf02 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SecurityConfig.java
>  7a6c69c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/swagger/swagger.html
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/53815/diff/
> 
> 
> Testing
> ---
> 
> testing done
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>