[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365081#comment-16365081 ] Josh Elser commented on PHOENIX-4533: - Good enough, Lev. I lifted the content into the markdown, edited it slightly, and have published it. Thanks! > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364841#comment-16364841 ] Lev Bronshtein commented on PHOENIX-4533: - Josh, is this what you are looking for? $ svn diff Index: site/publish/server.html === --- site/publish/server.html (revision 1824225) +++ site/publish/server.html (working copy) @@ -289,10 +289,20 @@ unset + phoenix.queryserver.http.keytab.file + The key to look for keytab file. This configuration MUST be specified if phoenix.queryserver.kerberos.http.principal is configured + unset + + phoenix.queryserver.kerberos.principal - The kerberos principal to use when authenticating. + The kerberos principal to use when authenticating. If phoenix.queryserver.kerberos.http.principal is not configured, the principlaa specified will be also used to both authenticate SPNEGO connections and to connect to HBase. Unless phoenix.queryserver.http.keytab.file is also specified, this configuration will be ignored unset + + phoenix.queryserver.kerberos.http.principal + The kerberos principal to use when authenticating SPNEGO connections + unset + phoenix.queryserver.dns.nameserver The DNS hostname > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362972#comment-16362972 ] Josh Elser commented on PHOENIX-4533: - bq. I am not sure what should change for building, Nothing to change on that page -- it has the information on where to check out the website's source and how to build it :) > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362911#comment-16362911 ] Lev Bronshtein commented on PHOENIX-4533: - Can do the docs, I am not sure what should change for building, definitely for server, where are the source for the doc website? > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362890#comment-16362890 ] Hudson commented on PHOENIX-4533: - FAILURE: Integrated in Jenkins build Phoenix-master #1936 (See [https://builds.apache.org/job/Phoenix-master/1936/]) PHOENIX-4533 Modified Query Server to use two sets of Kerberos (elserj: rev a71c4b7e3c11f1c7d1955b51929ad65b252feb62) * (edit) phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/HttpParamImpersonationQueryServerIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java * (edit) phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java * (edit) phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerIT.java > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362628#comment-16362628 ] Josh Elser commented on PHOENIX-4533: - Pushed this to the 4.x and 5.x branches. Thanks again, [~lbronshtein]. One final thing: any interest in updating the website with content for the new configuration properties you've added? We'd want to add them to https://phoenix.apache.org/server.html. https://phoenix.apache.org/building_website.html has instructions on how to do this. If you can get a diff against the website, I'd happily apply that too. Else, I'll just throw up something today myself. > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361543#comment-16361543
]
Josh Elser commented on PHOENIX-4533:
-
{{mvn verify}} with the PQS ITs passes for me. I think the only thing that
caught my eye was that you have the IT putting both keys into one keytab file.
This doesn't mimic what most people will do in reality, but there shouldn't be
any functional difference in doing it in one or multiple keytab files so
_shrug_.
Will run this through tests on each branch and push it out if it's good! Thanks
for your help, Lev!
For the future, it's preferred if each patch is standalone, rather than
building on the previous, Lev. I'll attach a new patch file here which is the
collection of changes you've made across all three commits.
> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch,
> PHOENIX-4533.3.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP
> ecosystem to perform SPNEGO authentication. Since there can only be one
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing
> key material for local HTTP/ principal is shared among a few applications.
> With so many applications having access to the HTTP/ credentials, this
> increases the chances of an attack on the proxy user capabilities of Hadoop.
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359660#comment-16359660 ] Lev Bronshtein commented on PHOENIX-4533: - HttpParamImpersonationQueryServerIT is now passing as well, patch attached > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359217#comment-16359217 ] Lev Bronshtein commented on PHOENIX-4533: - Josh, you are right, anyway I fixed SecureQueryServerIT and provided a patch for that fix. Though honestly I am not sure how this test would have worked in the first place given the nature of the error. Hoping to have HttpParamImpersonationQueryServerIT done shortly as well > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353094#comment-16353094
]
Josh Elser commented on PHOENIX-4533:
-
[~lbronshtein], are you sure the ITs are passing? Remember that Maven
integration tests are executed with the {{mvn verify}} lifecycle phase instead
of the {{mvn package}} phase (which is for unit tests).
I'm seeing the ITs failing with the following exception in the logs:
{noformat}
2018-02-05 18:21:48,053 DEBUG [pool-55-thread-1] server.QueryServer(236):
Current user is phoenixqs/localh...@example.com (auth:KERBEROS)
2018-02-05 18:21:48,054 FATAL [pool-55-thread-1] server.QueryServer(283):
Unrecoverable service error. Shutting down.
java.lang.IllegalArgumentException: Could not find '@' symbol in
'HTTP/localhost' to parse the Kerberos realm from the principal
at
org.apache.calcite.avatica.server.HttpServer$Builder.withSpnego(HttpServer.java:489)
at
org.apache.phoenix.queryserver.server.QueryServer.run(QueryServer.java:261)
at
org.apache.phoenix.queryserver.server.QueryServer.run(QueryServer.java:377)
at
org.apache.phoenix.end2end.SecureQueryServerIT$2$1.run(SecureQueryServerIT.java:254)
at
org.apache.phoenix.end2end.SecureQueryServerIT$2$1.run(SecureQueryServerIT.java:252)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1734)
at
org.apache.phoenix.end2end.SecureQueryServerIT$2.run(SecureQueryServerIT.java:252)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
Similarly, the {{startQueryServer()}} method in {{SecureQueryServerIT}} isn't
catching and failing the test like it should which is why the test hung instead
of failing outright. LMK if this isn't clear.
> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP
> ecosystem to perform SPNEGO authentication. Since there can only be one
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing
> key material for local HTTP/ principal is shared among a few applications.
> With so many applications having access to the HTTP/ credentials, this
> increases the chances of an attack on the proxy user capabilities of Hadoop.
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351008#comment-16351008 ] Josh Elser commented on PHOENIX-4533: - Thanks, Lev! Let me take a look and run through the tests locally. > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347980#comment-16347980 ] Lev Bronshtein commented on PHOENIX-4533: - Fixed the tests as well. Also it looks like I incorrectly generated the last patch, so I created a new one and attached it. > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346936#comment-16346936
]
Josh Elser commented on PHOENIX-4533:
-
bq. Actually I think I already figured it out (though not clear how this
affects other components). It looks like the login is done eternally. Just
need to make sure the avatica server will still do SPNEGO auth
Yup, you got it. That was meant to disable Avatica from trying to login while
when we already did the login in the test setup.
As long as you have {{kerberos}} set as the value for
{{QueryServices.QUERY_SERVER_HBASE_SECURITY_CONF_ATTRIB}}, PQS should end up
calling {{withSpnegoAuth(..)}} which is what forces the SPNEGO authentication
to happen.
> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP
> ecosystem to perform SPNEGO authentication. Since there can only be one
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing
> key material for local HTTP/ principal is shared among a few applications.
> With so many applications having access to the HTTP/ credentials, this
> increases the chances of an attack on the proxy user capabilities of Hadoop.
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346859#comment-16346859 ] Lev Bronshtein commented on PHOENIX-4533: - Actually I think I already figured it out (though not clear how this affects other components). It looks like the login is done eternally. Just need to make sure the avatica server will still do SPNEGO auth > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346840#comment-16346840
]
Lev Bronshtein commented on PHOENIX-4533:
-
Josh, I am having some trouble understanding why this line is being set in both
tests
{code:java}
conf.setBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN, true);
{code}
Especially since this seems to turn off the specific parts we want to test
{code:java}
final boolean disableLogin =
getConf().getBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN,
QueryServicesOptions.DEFAULT_QUERY_SERVER_DISABLE_KERBEROS_LOGIN);
...
if (isKerberos && !disableSpnego && !disableLogin) {
hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
getConf().get(QueryServices.QUERY_SERVER_DNS_INTERFACE_ATTRIB, "default"),
getConf().get(QueryServices.QUERY_SERVER_DNS_NAMESERVER_ATTRIB, "default")));
if (LOG.isDebugEnabled()) {
LOG.debug("Login to " + hostname + " using " + getConf().get(
QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB)
+ " and principal " + getConf().get(
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB) + ".");
}
SecurityUtil.login(getConf(), QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB,
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB, hostname);
LOG.info("Login successful.");
} else {
hostname = InetAddress.getLocalHost().getHostName();
LOG.info(" Kerberos is off and hostname is : "+hostname);
}
{code}
> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP
> ecosystem to perform SPNEGO authentication. Since there can only be one
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing
> key material for local HTTP/ principal is shared among a few applications.
> With so many applications having access to the HTTP/ credentials, this
> increases the chances of an attack on the proxy user capabilities of Hadoop.
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343844#comment-16343844 ] Lev Bronshtein commented on PHOENIX-4533: - First par is done, two days later kinit as my user and access PQS, still able to run queries. I will look into the tests in a bit as well > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341410#comment-16341410 ] Lev Bronshtein commented on PHOENIX-4533: - Also here are my Configuration changes h3. *BEFORE* h4. *hbase-site.xml* phoenix.queryserver.kerberos.principal HTTP/f-bcpc-vm2.bcpc.example@bcpc.example.com phoenix.queryserver.keytab.file /etc/security/keytabs/spnego.service.keytab phoenix.queryserver.serialization JSON hadoop.proxyuser.HTTP.hosts * hadoop.proxyuser.HTTP.users * h4. core-site.xml hadoop.proxyuser.HTTP.hosts * hadoop.proxyuser.HTTP.users * h3. *AFTER* h4. *hbase-site.xml* phoenix.queryserver.kerberos.http.principal HTTP/f-bcpc-vm1.bcpc.example@bcpc.example.com phoenix.queryserver.http.keytab.file /etc/security/keytabs/spnego.service.keytab phoenix.queryserver.kerberos.principal phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com phoenix.queryserver.keytab.file /etc/security/keytabs/phoenixqs.service.keytab h4. core-site.xml hadoop.proxyuser.phoenixqs.hosts * hadoop.proxyuser.phoenixqs.users * > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341283#comment-16341283 ] Lev Bronshtein commented on PHOENIX-4533: - Looks like it works. I first set the max lifetime for the principal in question to 5 minutes using kadmin bq kadmin.local: modprinc -maxlife "5 minutes" phoenixqs/f-bcpc-vm1.bcpc.example.com Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified. kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com Expiration date: [never] Last password change: Fri Jan 19 20:22:31 UTC 2018 Password expiration date: [none] Maximum ticket life: 0 days 00:05:00 Maximum renewable life: 7 days 00:00:00 Last modified: Fri Jan 26 16:27:47 UTC 2018 (root/ad...@bcpc.example.com) Last successful authentication: [never] Last failed authentication: [never] Failed password attempts: 0 Number of keys: 3 Key: vno 2, arcfour-hmac, no salt Key: vno 2, des3-cbc-sha1, no salt Key: vno 2, des-cbc-crc, no salt MKey: vno 1 Attributes: Policy: [none] 2018-01-26 11:58:58,356 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) 2018-01-26 11:58:58,379 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) 2018-01-26 11:58:58,386 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734) 2018-01-26 11:58:58,390 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedActionException as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2018-01-26 11:58:58,391 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:637) 2018-01-26 11:58:58,393 DEBUG org.apache.hadoop.security.UserGroupInformation: Initiating logout for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop logout 2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: Initiating re-login for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 2018-01-26 11:58:58,398 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login 2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login commit 2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: using existing subject:[phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com, phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com] 2018-01-26 11:59:01,227 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734) 2018-01-26 11:59:01,299 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338308#comment-16338308
]
Josh Elser commented on PHOENIX-4533:
-
Using separate Kerberos identities for accepting requests and talking to HBase
sounds like a great idea (especially, given the limitations of SPNEGO with
Kerberos and Hadoop's impersonation rules).
My biggest concern is ensuring that ticket renewal happens for both principals,
and that the HTTP principal is not used to talk to HBase at all. I'm thinking a
setup like the following:
* Set short ticket lifetimes for the HTTP and hbase client kerberos principals
(e.g. 10m)
* The HTTP user is not authorized to interact with any HBase tables, nor
impersonate any end users
* Set up a PQS client to read from a Phoenix table through PQS at a regular
interval (e.g. every 15s). Something trivial like a {{select *}} would be fine.
Then, just let this run for a few hours. At the end of the test, PQS should
still be operational and the client can still read the Phoenix table through
PQS.
It's a little elaborate to try to encapsulate this in an IT, but if you could
run a standalone test, Lev, that'd be awesome.
> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP
> ecosystem to perform SPNEGO authentication. Since there can only be one
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing
> key material for local HTTP/ principal is shared among a few applications.
> With so many applications having access to the HTTP/ credentials, this
> increases the chances of an attack on the proxy user capabilities of Hadoop.
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
