Re: Review Request 45252: PXF service checks fails on secured cluster without YARN

2016-03-25 Thread bhuvnesh chaudhary 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125472
---


Ship it!




Ship It!

- bhuvnesh chaudhary


On March 25, 2016, 8:50 p.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> ---
> 
> (Updated March 25, 2016, 8:50 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
> chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton 
> Alex, and Oleksandr Diachenko.
> 
> 
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 
> 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 
> 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': 
> '/usr/bin/kinit', 'principal_name': 'hdfs-...@ambari.apache.org', 'user': 
> 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: 
> PXF data read failed: HTTP Error 500: Internal Server Error
> ```
> 
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path 
> [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos 
> principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
>   at 
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
>   at 
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
>   at 
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
>   at 
> org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
>   at 
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
>   at 
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
>   at 
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
>   at 
> org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py
>  eed2d18 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
>  40e6d6f 
> 
> Diff: https://reviews.apache.org/r/45252/diff/
> 
> 
> Testing
> ---
> 
> Tested manually with and without YARN, with and without security.
> 
> 
> Thanks,
> 
> Lav Jain
> 
>

Re: Review Request 45252: PXF service checks fails on secured cluster without YARN

2016-03-25 Thread Lav Jain 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/
---

(Updated March 25, 2016, 8:50 p.m.)


Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton 
Alex, and Oleksandr Diachenko.


Bugs: AMBARI-15542
https://issues.apache.org/jira/browse/AMBARI-15542


Repository: ambari


Description
---

PXF service checks fails on secured cluster
```
2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 
'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 
'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': 
'/usr/bin/kinit', 'principal_name': 'hdfs-...@ambari.apache.org', 'user': 
'hdfs', 'action': ['execute']}
2016-03-11 23:37:35,588 - Testing PXF HDFS read
2016-03-11 23:37:35,675 - Error code: 500
2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: 
PXF data read failed: HTTP Error 500: Internal Server Error
```

Pxf log says
```
SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path 
[/pxf] threw exception [java.io.IOException: Can't get Master Kerberos 
principal for use as renewer] with root cause
java.io.IOException: Can't get Master Kerberos principal for use as renewer
at 
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
at 
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at 
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at 
org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
at 
org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
at 
org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
at 
org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
at 
org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
```


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py
 eed2d18 
  
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
 40e6d6f 

Diff: https://reviews.apache.org/r/45252/diff/


Testing
---

Tested manually with and without YARN, with and without security.


Thanks,

Lav Jain

Re: Review Request 45252: PXF service checks fails on secured cluster without YARN

2016-03-24 Thread Lav Jain 


> On March 24, 2016, 1:18 a.m., Matt wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py,
> >  line 154
> > 
> >
> > Does YARN have to be functional (green status), or just installed so 
> > that libraries are available?

YARN libraries are already present. It has to be configured (to populate 
yarn-site.xml).


- Lav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125170
---


On March 23, 2016, 10:23 p.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> ---
> 
> (Updated March 23, 2016, 10:23 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
> chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton 
> Alex, and Oleksandr Diachenko.
> 
> 
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 
> 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 
> 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': 
> '/usr/bin/kinit', 'principal_name': 'hdfs-...@ambari.apache.org', 'user': 
> 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: 
> PXF data read failed: HTTP Error 500: Internal Server Error
> ```
> 
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path 
> [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos 
> principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
>   at 
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
>   at 
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
>   at 
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
>   at 
> org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
>   at 
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
>   at 
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
>   at 
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
>   at 
> org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py
>  eed2d18 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
>  40e6d6f 
> 
> Diff: https://reviews.apache.org/r/45252/diff/
> 
> 
> Testing
> ---
> 
> Tested manually with and without YARN, with and without security.
> 
> 
> Thanks,
> 
> Lav Jain
> 
>