[jira] [Updated] (LENS-1506) Kerberos authentication in lens

[Puneet Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puneet Gupta updated LENS-1506:
---
Fix Version/s: 2.8

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Fix For: 2.8
>
> Attachments: Lens-1506.1.patch, Lens-1506.2.patch, Lens-1506.3.patch, 
> Lens-1506.4.patch, Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LENS-1506) Kerberos authentication in lens

[Puneet Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426805#comment-16426805
 ] 

Puneet Gupta commented on LENS-1506:


Committed. Thanks [~ankitkailaswar]

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Fix For: 2.8
>
> Attachments: Lens-1506.1.patch, Lens-1506.2.patch, Lens-1506.3.patch, 
> Lens-1506.4.patch, Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LENS-1506) Kerberos authentication in lens

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426800#comment-16426800
 ] 

Hadoop QA commented on LENS-1506:
-

Applied patch: 
[Lens-1506.4.patch|https://issues.apache.org/jira/secure/attachment/12917678/Lens-1506.4.patch]
 and ran command: mvn clean install -fae. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1509/

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Attachments: Lens-1506.1.patch, Lens-1506.2.patch, Lens-1506.3.patch, 
> Lens-1506.4.patch, Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (LENS-1506) Kerberos authentication in lens

[Ankit Kailaswar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Kailaswar updated LENS-1506:
--
Attachment: Lens-1506.4.patch

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Attachments: Lens-1506.1.patch, Lens-1506.2.patch, Lens-1506.3.patch, 
> Lens-1506.4.patch, Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (LENS-1506) Kerberos authentication in lens

[Ankit Kailaswar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Kailaswar updated LENS-1506:
--
Attachment: Lens-1506.2.patch

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Attachments: Lens-1506.1.patch, Lens-1506.2.patch, Lens-1506_patch, 
> design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (LENS-1506) Kerberos authentication in lens

[Ankit Kailaswar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Kailaswar updated LENS-1506:
--
Attachment: Lens-1506.1.patch

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Attachments: Lens-1506.1.patch, Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (LENS-1506) Kerberos authentication in lens

[Ankit Kailaswar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Kailaswar updated LENS-1506:
--
Status: Patch Available  (was: In Progress)

attached patch

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Attachments: Lens-1506.1.patch, Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (LENS-1506) Kerberos authentication in lens

[Ankit Kailaswar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Kailaswar updated LENS-1506:
--
Attachment: (was: Lens-1506.diff)

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Attachments: Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (LENS-1506) Kerberos authentication in lens

[Ankit Kailaswar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Kailaswar updated LENS-1506:
--
Attachment: Lens-1506.diff

> Kerberos authentication in lens
> ---
>
> Key: LENS-1506
> URL: https://issues.apache.org/jira/browse/LENS-1506
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client, driver-hive, python-client, server
>Reporter: Ankit Kailaswar
>Assignee: Ankit Kailaswar
>Priority: Major
> Attachments: Lens-1506.diff, Lens-1506_patch, design3.png
>
>
> Current Lens implementation is broken when we try to enable kerberos 
> authentication in lens as mentioned at 
> [https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2] in 
> following ways,
> 1. openSession REST API fails to create new session for user. Currently it 
> supports only passwd types of authentication.
> 2. If the underlying hive driver is running with kerberos authentication then 
> driver initialization flow to obtain hive transport for hive driver in lens 
> errors out. Hive server accepts only sasl messages but lens continues using 
> PLAINSASL.
> 3. If hadoop cluster has kerberos authentication enabled then all hdfs calls 
> (persisting services, all hdfs path in conf etc) fail.
> 4. Lens as if now doesnt supports refreshing KDC token before it expires.
> Changes required in lens to fully support kerberose authentication are as 
> follows,
>  # lens's hive driver must use SASL for all communication in to kerberozied 
> hive. Current thrift client for hive doesn't support this functionality.
>  # Lens must refresh KDC ticket before it expires.
>  # All clients must be authenticated with kerberose authentication before 
> session creation.
>  # In kerberos mode all hive driver query should be executed with single 
> cluster user as "lens".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66081: Kerberos authentication in lens

[Ankit Kailaswar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66081/
---

(Updated April 5, 2018, 10:03 a.m.)


Review request for lens, Amareshwari Sriramadasu, Rajat Khandelwal, and Puneet 
Gupta.


Repository: lens


Description
---

https://issues.apache.org/jira/browse/LENS-1506

This patch contains code changes to enable kerberos authentication for 
1. lens to hive 
2. lens to metastore
3. lens to hdfs

code changes are as follows,
1. new http thrift client for hive driver to support sasl transport for 
kerberozied hive server.
2. cron to update KDC ticket before it expires.


Diffs (updated)
-

  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
2eb94aa7 
  
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RemoteThriftConnection.java
 54885f77 
  
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RetryingThriftCLIServiceClientSasl.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
d5273be8 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 
9d732c1c 
  lens-server/src/main/java/org/apache/lens/server/BaseLensService.java 
c30a2d72 
  lens-server/src/main/java/org/apache/lens/server/LensServices.java 903b19fe 
  
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
 31ac358d 
  lens-server/src/main/resources/lensserver-default.xml 28b1db29 
  lens-server/src/test/resources/lens-site.xml 7e5f522e 
  tools/scripts/lens-ctl 304b4f5a 


Diff: https://reviews.apache.org/r/66081/diff/8/

Changes: https://reviews.apache.org/r/66081/diff/7-8/


Testing
---

unit testing


Thanks,

Ankit Kailaswar

Re: Review Request 66081: Kerberos authentication in lens

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66081/#review200541
---




lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RetryingThriftCLIServiceClientSasl.java
Lines 130 (patched)


can you add a comment why this is required.



lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java
Lines 148 (patched)


Can you import SecurityUtil ?



lens-server/src/main/java/org/apache/lens/server/LensServices.java
Lines 299 (patched)


can you move the KDC code to a separate method ?



lens-server/src/main/java/org/apache/lens/server/LensServices.java
Lines 304 (patched)


Should we catch Exception instead in this case ?


- Puneet Gupta


On April 5, 2018, 5:41 a.m., Ankit Kailaswar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66081/
> ---
> 
> (Updated April 5, 2018, 5:41 a.m.)
> 
> 
> Review request for lens, Amareshwari Sriramadasu, Rajat Khandelwal, and 
> Puneet Gupta.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/LENS-1506
> 
> This patch contains code changes to enable kerberos authentication for 
> 1. lens to hive 
> 2. lens to metastore
> 3. lens to hdfs
> 
> code changes are as follows,
> 1. new http thrift client for hive driver to support sasl transport for 
> kerberozied hive server.
> 2. cron to update KDC ticket before it expires.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 2eb94aa7 
>   
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RemoteThriftConnection.java
>  54885f77 
>   
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RetryingThriftCLIServiceClientSasl.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  d5273be8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 
> 9d732c1c 
>   lens-server/src/main/java/org/apache/lens/server/BaseLensService.java 
> c30a2d72 
>   lens-server/src/main/java/org/apache/lens/server/LensServices.java 903b19fe 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  31ac358d 
>   lens-server/src/main/resources/lensserver-default.xml 28b1db29 
>   lens-server/src/test/resources/lens-site.xml 7e5f522e 
>   tools/scripts/lens-ctl 304b4f5a 
> 
> 
> Diff: https://reviews.apache.org/r/66081/diff/7/
> 
> 
> Testing
> ---
> 
> unit testing
> 
> 
> Thanks,
> 
> Ankit Kailaswar
> 
>