[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-14 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365081#comment-16365081
 ] 

Josh Elser commented on PHOENIX-4533:
-

Good enough, Lev. I lifted the content into the markdown, edited it slightly, 
and have published it. Thanks!

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-14 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364841#comment-16364841
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Josh, is this what you are looking for?

$ svn diff
Index: site/publish/server.html
===
--- site/publish/server.html (revision 1824225)
+++ site/publish/server.html (working copy)
@@ -289,10 +289,20 @@
 unset
 
 
+ phoenix.queryserver.http.keytab.file
+ The key to look for keytab file. This 
configuration MUST be specified if phoenix.queryserver.kerberos.http.principal 
is configured
+ unset
+ 
+ 
 phoenix.queryserver.kerberos.principal
- The kerberos principal to use when 
authenticating.
+ The kerberos principal to use when 
authenticating. If phoenix.queryserver.kerberos.http.principal is not 
configured, the principlaa specified will be also used to both authenticate 
SPNEGO connections and to connect to HBase. Unless 
phoenix.queryserver.http.keytab.file is also specified, this configuration will 
be ignored
 unset
 
+ 
+ phoenix.queryserver.kerberos.http.principal
+ The kerberos principal to use when 
authenticating SPNEGO connections
+ unset
+ 
 
 phoenix.queryserver.dns.nameserver
 The DNS hostname

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-13 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362972#comment-16362972
 ] 

Josh Elser commented on PHOENIX-4533:
-

bq. I am not sure what should change for building,

Nothing to change on that page -- it has the information on where to check out 
the website's source and how to build it :)

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-13 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362911#comment-16362911
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Can do the docs, I am not sure what should change for building, definitely for 
server, where are the source for the doc website?

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-13 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362890#comment-16362890
 ] 

Hudson commented on PHOENIX-4533:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1936 (See 
[https://builds.apache.org/job/Phoenix-master/1936/])
PHOENIX-4533 Modified Query Server to use two sets of Kerberos (elserj: rev 
a71c4b7e3c11f1c7d1955b51929ad65b252feb62)
* (edit) 
phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/HttpParamImpersonationQueryServerIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java
* (edit) 
phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java
* (edit) 
phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerIT.java


> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-13 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362628#comment-16362628
 ] 

Josh Elser commented on PHOENIX-4533:
-

Pushed this to the 4.x and 5.x branches. Thanks again, [~lbronshtein].

One final thing: any interest in updating the website with content for the new 
configuration properties you've added?

We'd want to add them to https://phoenix.apache.org/server.html. 
https://phoenix.apache.org/building_website.html has instructions on how to do 
this. If you can get a diff against the website, I'd happily apply that too. 
Else, I'll just throw up something today myself.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-12 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361543#comment-16361543
 ] 

Josh Elser commented on PHOENIX-4533:
-

{{mvn verify}} with the PQS ITs passes for me. I think the only thing that 
caught my eye was that you have the IT putting both keys into one keytab file. 
This doesn't mimic what most people will do in reality, but there shouldn't be 
any functional difference in doing it in one or multiple keytab files so 
_shrug_.

Will run this through tests on each branch and push it out if it's good! Thanks 
for your help, Lev!

For the future, it's preferred if each patch is standalone, rather than 
building on the previous, Lev. I'll attach a new patch file here which is the 
collection of changes you've made across all three commits.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-10 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359660#comment-16359660
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

HttpParamImpersonationQueryServerIT is now passing as well, patch attached

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, 
> PHOENIX-4533.3.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-09 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359217#comment-16359217
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Josh, you are right, anyway I fixed SecureQueryServerIT and provided a patch 
for that fix.  Though honestly I am not sure how this test would have worked in 
the first place given the nature of the error.  Hoping to have 
HttpParamImpersonationQueryServerIT done shortly as well

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-05 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353094#comment-16353094
 ] 

Josh Elser commented on PHOENIX-4533:
-

[~lbronshtein], are you sure the ITs are passing? Remember that Maven 
integration tests are executed with the {{mvn verify}} lifecycle phase instead 
of the {{mvn package}} phase (which is for unit tests).

I'm seeing the ITs failing with the following exception in the logs:

{noformat}
2018-02-05 18:21:48,053 DEBUG [pool-55-thread-1] server.QueryServer(236): 
Current user is phoenixqs/localh...@example.com (auth:KERBEROS)
2018-02-05 18:21:48,054 FATAL [pool-55-thread-1] server.QueryServer(283): 
Unrecoverable service error. Shutting down.
java.lang.IllegalArgumentException: Could not find '@' symbol in 
'HTTP/localhost' to parse the Kerberos realm from the principal
at 
org.apache.calcite.avatica.server.HttpServer$Builder.withSpnego(HttpServer.java:489)
at 
org.apache.phoenix.queryserver.server.QueryServer.run(QueryServer.java:261)
at 
org.apache.phoenix.queryserver.server.QueryServer.run(QueryServer.java:377)
at 
org.apache.phoenix.end2end.SecureQueryServerIT$2$1.run(SecureQueryServerIT.java:254)
at 
org.apache.phoenix.end2end.SecureQueryServerIT$2$1.run(SecureQueryServerIT.java:252)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1734)
at 
org.apache.phoenix.end2end.SecureQueryServerIT$2.run(SecureQueryServerIT.java:252)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}

Similarly, the {{startQueryServer()}} method in {{SecureQueryServerIT}} isn't 
catching and failing the test like it should which is why the test hung instead 
of failing outright. LMK if this isn't clear.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-02-02 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351008#comment-16351008
 ] 

Josh Elser commented on PHOENIX-4533:
-

Thanks, Lev!

Let me take a look and run through the tests locally.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-31 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347980#comment-16347980
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Fixed the tests as well.  Also it looks like I incorrectly generated the last 
patch, so I created a new one and attached it.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-31 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346936#comment-16346936
 ] 

Josh Elser commented on PHOENIX-4533:
-

bq. Actually I think I already figured it out (though not clear how this 
affects other components).  It looks like the login is done eternally.  Just 
need to make sure the avatica server will still do SPNEGO auth

Yup, you got it. That was meant to disable Avatica from trying to login while 
when we already did the login in the test setup.

As long as you have {{kerberos}} set as the value for 
{{QueryServices.QUERY_SERVER_HBASE_SECURITY_CONF_ATTRIB}}, PQS should end up 
calling {{withSpnegoAuth(..)}} which is what forces the SPNEGO authentication 
to happen.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-31 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346859#comment-16346859
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Actually I think I already figured it out (though not clear how this affects 
other components).  It looks like the login is done eternally.  Just need to 
make sure the avatica server will still do SPNEGO auth

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-31 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346840#comment-16346840
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Josh, I am having some trouble understanding why this line is being set in both 
tests
{code:java}
conf.setBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN, true);
{code}
Especially since this seems to turn off the specific parts we want to test


{code:java}
final boolean disableLogin = 
getConf().getBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN,
QueryServicesOptions.DEFAULT_QUERY_SERVER_DISABLE_KERBEROS_LOGIN);

...

if (isKerberos && !disableSpnego && !disableLogin) {
hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
getConf().get(QueryServices.QUERY_SERVER_DNS_INTERFACE_ATTRIB, "default"),
getConf().get(QueryServices.QUERY_SERVER_DNS_NAMESERVER_ATTRIB, "default")));
if (LOG.isDebugEnabled()) {
LOG.debug("Login to " + hostname + " using " + getConf().get(
QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB)
+ " and principal " + getConf().get(
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB) + ".");
}
SecurityUtil.login(getConf(), QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB,
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB, hostname);
LOG.info("Login successful.");
} else {
hostname = InetAddress.getLocalHost().getHostName();
LOG.info(" Kerberos is off and hostname is : "+hostname);
}
{code}

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-29 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343920#comment-16343920
 ] 

Josh Elser commented on PHOENIX-4533:
-

bq. First part is done, two days later kinit as my user and access PQS, still 
able to run queries.  I will look into the tests in a bit as well

Excellent! Thanks for confirming.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-29 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343844#comment-16343844
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

First par is done, two days later kinit as my user and access PQS, still able 
to run queries.  I will look into the tests in a bit as well

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-26 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341410#comment-16341410
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Also here are my Configuration changes

 
h3. *BEFORE*
h4. *hbase-site.xml*

  

    phoenix.queryserver.kerberos.principal

    HTTP/f-bcpc-vm2.bcpc.example@bcpc.example.com

  

  

    phoenix.queryserver.keytab.file

    /etc/security/keytabs/spnego.service.keytab

  

  

    phoenix.queryserver.serialization

    JSON

  

 



    hadoop.proxyuser.HTTP.hosts

    *

  

  

    hadoop.proxyuser.HTTP.users

    *

  

 
h4. core-site.xml

  

    hadoop.proxyuser.HTTP.hosts

    *

  

  

    hadoop.proxyuser.HTTP.users

    *

  

 
h3. *AFTER*
h4. *hbase-site.xml*

  

    phoenix.queryserver.kerberos.http.principal

    HTTP/f-bcpc-vm1.bcpc.example@bcpc.example.com

  

  

    phoenix.queryserver.http.keytab.file

    /etc/security/keytabs/spnego.service.keytab

  

  

    phoenix.queryserver.kerberos.principal

    phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com

  

  

    phoenix.queryserver.keytab.file

    /etc/security/keytabs/phoenixqs.service.keytab

  

 
h4. core-site.xml

  

    hadoop.proxyuser.phoenixqs.hosts

    *

  

  

    hadoop.proxyuser.phoenixqs.users

    *

  

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-26 Thread Lev Bronshtein (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341283#comment-16341283
 ] 

Lev Bronshtein commented on PHOENIX-4533:
-

Looks like it works.  I first set the max lifetime for the principal in 
question to 5 minutes using kadmin

bq

kadmin.local:  modprinc -maxlife "5 minutes" 
phoenixqs/f-bcpc-vm1.bcpc.example.com

Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified.


 kadmin.local:  getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com

Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com

Expiration date: [never]

Last password change: Fri Jan 19 20:22:31 UTC 2018

Password expiration date: [none]

Maximum ticket life: 0 days 00:05:00

Maximum renewable life: 7 days 00:00:00

Last modified: Fri Jan 26 16:27:47 UTC 2018 (root/ad...@bcpc.example.com)

Last successful authentication: [never]

Last failed authentication: [never]

Failed password attempts: 0

Number of keys: 3

Key: vno 2, arcfour-hmac, no salt

Key: vno 2, des3-cbc-sha1, no salt

Key: vno 2, des-cbc-crc, no salt

MKey: vno 1

Attributes:

Policy: [none]



2018-01-26 11:58:58,356 DEBUG org.apache.hadoop.security.UserGroupInformation: 
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via 
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) 
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
2018-01-26 11:58:58,379 DEBUG org.apache.hadoop.security.UserGroupInformation: 
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via 
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) 
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
2018-01-26 11:58:58,386 DEBUG org.apache.hadoop.security.UserGroupInformation: 
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 
(auth:KERBEROS) 
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
2018-01-26 11:58:58,390 DEBUG org.apache.hadoop.security.UserGroupInformation: 
PrivilegedActionException 
as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) 
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by 
GSSException: No valid credentials provided (Mechanism level: Failed to find 
any Kerberos tgt)]
2018-01-26 11:58:58,391 DEBUG org.apache.hadoop.security.UserGroupInformation: 
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 
(auth:KERBEROS) 
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:637)
2018-01-26 11:58:58,393 DEBUG org.apache.hadoop.security.UserGroupInformation: 
Initiating logout for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: 
hadoop logout
2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: 
Initiating re-login for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
2018-01-26 11:58:58,398 DEBUG org.apache.hadoop.security.UserGroupInformation: 
hadoop login
2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: 
hadoop login commit
2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: 
using existing subject:[phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com, 
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com]
2018-01-26 11:59:01,227 DEBUG org.apache.hadoop.security.UserGroupInformation: 
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 
(auth:KERBEROS) 
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
2018-01-26 11:59:01,299 DEBUG org.apache.hadoop.security.UserGroupInformation: 
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via 
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) 
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 

[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests

2018-01-24 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338308#comment-16338308
 ] 

Josh Elser commented on PHOENIX-4533:
-

Using separate Kerberos identities for accepting requests and talking to HBase 
sounds like a great idea (especially, given the limitations of SPNEGO with 
Kerberos and Hadoop's impersonation rules).

My biggest concern is ensuring that ticket renewal happens for both principals, 
and that the HTTP principal is not used to talk to HBase at all. I'm thinking a 
setup like the following:

* Set short ticket lifetimes for the HTTP and hbase client kerberos principals 
(e.g. 10m)
* The HTTP user is not authorized to interact with any HBase tables, nor 
impersonate any end users
* Set up a PQS client to read from a Phoenix table through PQS at a regular 
interval (e.g. every 15s). Something trivial like a {{select *}} would be fine.

Then, just let this run for a few hours. At the end of the test, PQS should 
still be operational and the client can still read the Phoenix table through 
PQS.

It's a little elaborate to try to encapsulate this in an IT, but if you could 
run a standalone test, Lev, that'd be awesome.

> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP 
> ecosystem to perform SPNEGO authentication.  Since there can only be one 
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing 
> key material for local HTTP/ principal is shared among a few applications.  
> With so many applications having access to the HTTP/ credentials, this 
> increases the chances of an attack on the proxy user capabilities of Hadoop.  
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)