[jira] [Updated] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lev Bronshtein updated PHOENIX-4688: Comment: was deleted (was: Will investigate if perhaps when I did the test, it worked with avatica 1.11.0, the expectation was that. CALCITE-1922 fixed the issue that necessitated a path to requests=kerberos or requests-gssapi. However the associated pull request was closed without a merge [https://github.com/apache/calcite-avatica/pull/15] Looking at Avatica on github [https://github.com/apache/calcite/blob/calcite-1.11.0/avatica/server/src/main/java/org/apache/calcite/avatica/server/PropertyBasedSpnegoLoginService.java,] it does not look like this was ever taken care of [~elserj] I am kinda floundering and going around in circles here, could you please check on what happened to the change you proposed?) > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550751#comment-16550751 ] Lev Bronshtein commented on PHOENIX-4688: - Not a bad idea, I have been a little busy this week, but will try to get to this soon. If you have any suggestions on how to instrument this, let me know. I am assuming all we need is a well written docker file > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4808) Phoenix Spark Do not verify zKUrl in DefaultSource.scala
[
https://issues.apache.org/jira/browse/PHOENIX-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lev Bronshtein updated PHOENIX-4808:
Description:
Currently verifyParameters will throw an exception
_[zkUrl
check|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L56]_
_if (parameters.get("zkUrl").isEmpty)_ however this is unnecessary as
ultimately new PhoenixRelation(...)
[PhoenixRelation|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L29]
is called (see PhoenixRelation.scala) which in turn calls new PhoenixRDD(...)
(See PhoenixRDD.scala). PhoenixRDD constructor declares zkUrl as optional.
PhoenixRDD will try to use zkUrl and fall back on HBase Configuration
{code:java}
// Override the Zookeeper URL if present. Throw exception if no address given.
zkUrl match {
case Some(url) => ConfigurationUtil.setZookeeperURL(config, url)
case _ => {
if(ConfigurationUtil.getZookeeperURL(config).isEmpty) {
throw new UnsupportedOperationException(
s"One of zkUrl or '${HConstants.ZOOKEEPER_QUORUM}' config
property must be provided"
)
}
}
{code}
I propose that this check is unnecessary and should be removed. As we will
still get an exception, but a little later
was:
Currently verifyParameters will throw an exception
_[zkUrl
check|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L56]_
_if (parameters.get("zkUrl").isEmpty)_ however this is unnecessary as
ultimately new PhoenixRelation(...)
[PhoenixRelation|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L29]
is called (see PhoenixRelation.scala) which in turn calls new PhoenixRDD(...)
(See PhoenixRDD.scala). PhoenixRDD constructor declares zkUrl as optional.
PhoenixRDD will try to use zkUrl and fall back on HBase Configuration
{code:java}
// Override the Zookeeper URL if present. Throw exception if no address given.
zkUrl match {
case Some(url) => ConfigurationUtil.setZookeeperURL(config, url)
case _ => {
if(ConfigurationUtil.getZookeeperURL(config).isEmpty) {
throw new UnsupportedOperationException(
s"One of zkUrl or '${HConstants.ZOOKEEPER_QUORUM}' config property must be
provided"
)
}
}
{code}
I propose that this check is unnecessary and should be removed. As we will
still get an exception, but a little later
> Phoenix Spark Do not verify zKUrl in DefaultSource.scala
>
>
> Key: PHOENIX-4808
> URL: https://issues.apache.org/jira/browse/PHOENIX-4808
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Priority: Major
>
> Currently verifyParameters will throw an exception
> _[zkUrl
> check|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L56]_
> _if (parameters.get("zkUrl").isEmpty)_ however this is unnecessary as
> ultimately new PhoenixRelation(...)
> [PhoenixRelation|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L29]
> is called (see PhoenixRelation.scala) which in turn calls new PhoenixRDD(...)
> (See PhoenixRDD.scala). PhoenixRDD constructor declares zkUrl as optional.
> PhoenixRDD will try to use zkUrl and fall back on HBase Configuration
>
> {code:java}
> // Override the Zookeeper URL if present. Throw exception if no address given.
> zkUrl match {
> case Some(url) => ConfigurationUtil.setZookeeperURL(config, url)
> case _ => {
> if(ConfigurationUtil.getZookeeperURL(config).isEmpty) {
> throw new UnsupportedOperationException(
> s"One of zkUrl or '${HConstants.ZOOKEEPER_QUORUM}' config
> property must be provided"
> )
> }
> }
> {code}
>
> I propose that this check is unnecessary and should be removed. As we will
> still get an exception, but a little later
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PHOENIX-4808) Phoenix Spark Do not verify zKUrl in DefaultSource.scala
Lev Bronshtein created PHOENIX-4808:
---
Summary: Phoenix Spark Do not verify zKUrl in DefaultSource.scala
Key: PHOENIX-4808
URL: https://issues.apache.org/jira/browse/PHOENIX-4808
Project: Phoenix
Issue Type: Improvement
Reporter: Lev Bronshtein
Currently verifyParameters will throw an exception
_[zkUrl
check|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L56]_
_if (parameters.get("zkUrl").isEmpty)_ however this is unnecessary as
ultimately new PhoenixRelation(...)
[PhoenixRelation|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/DefaultSource.scala#L29]
is called (see PhoenixRelation.scala) which in turn calls new PhoenixRDD(...)
(See PhoenixRDD.scala). PhoenixRDD constructor declares zkUrl as optional.
PhoenixRDD will try to use zkUrl and fall back on HBase Configuration
{code:java}
// Override the Zookeeper URL if present. Throw exception if no address given.
zkUrl match {
case Some(url) => ConfigurationUtil.setZookeeperURL(config, url)
case _ => {
if(ConfigurationUtil.getZookeeperURL(config).isEmpty) {
throw new UnsupportedOperationException(
s"One of zkUrl or '${HConstants.ZOOKEEPER_QUORUM}' config property must be
provided"
)
}
}
{code}
I propose that this check is unnecessary and should be removed. As we will
still get an exception, but a little later
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537266#comment-16537266 ] Lev Bronshtein edited comment on PHOENIX-4688 at 7/10/18 8:19 PM: -- Create and activate environment here [https://github.com/apache/phoenix/pull/307/files#diff-0d0a748959965a7cfdc725f33414d1c0R30] KINIT here [https://github.com/apache/phoenix/pull/307/files#diff-0d0a748959965a7cfdc725f33414d1c0R50] There are minor improvements from the first take where the python script and krb5.conf both started out as heredocs inside the shell script. I attempted to pull kinit into JAVA as well and pass the environment around but # I failed to make this work # executing various shell commands from java adds a lot of bloat # I don't even know how I would source a script and then pass the resulting shell modifications onto the next one in java Having encountered #3, I gave up on further shell script pruning/elimination Just realized that that I never transitioned from conda to virtualenv. I am going to attempt to support both, but for now there is probably enough to look at. was (Author: lbronshtein): Create and activate environment here [https://github.com/apache/phoenix/pull/307/files#diff-0d0a748959965a7cfdc725f33414d1c0R30] KINIT here [https://github.com/apache/phoenix/pull/307/files#diff-0d0a748959965a7cfdc725f33414d1c0R50] There are minor improvements from the first take where the python script and krb5.conf both started out as heredocs inside the shell script. I attempted to pull kin it into JAVA as well and pass the environment around but # I failed to make this work # executing various shell commands from java adds a lot of bloat # I don't even know how I would source a script and then pass the resulting shell modifications onto the next one in java Having encountered #3 I gave on further shell script pruning/elimination Just realized that that I never transitioned from conda to virtualenv. I am going to attempt to support both, but for now there is probably enough to look at. > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537266#comment-16537266 ] Lev Bronshtein commented on PHOENIX-4688: - Create and activate environment here https://github.com/apache/phoenix/pull/307/files#diff-0d0a748959965a7cfdc725f33414d1c0R30 KINIT here [https://github.com/apache/phoenix/pull/307/files#diff-0d0a748959965a7cfdc725f33414d1c0R50 ] There are minor improvements from the first take where the python script and krb5.conf both started out as heredocs inside the shell script. I attempted to pull kin it into JAVA as well and pass the environment around but # I failed to make this work # executing various shell commands from java adds a lot of bloat # I don't even know how I would source a script and then pass the resulting shell modifications onto the next one in java Having encountered #3 I gave on further shell script pruning/elimination Just realized that that I never transitioned from conda to virtualenv. I am going to attempt to support both, but for now there is probably enough to look at. > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/28/18 7:26 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit, also it needs to source activate script * -Do we care if it only works on Linux?- It will work on MAC OS too * -currently only works with Anaconda, would rather see it with virtualenv,- though neither one comes pre installed on stock MAC OS was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit, also it needs to source activate script * -Do we care if it only works on Linux?- It will work on MAC OS too * currently only works with Anaconda, would rather see it with virtualenv, though neither one comes pre installed on stock MAC OS > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526692#comment-16526692 ] Lev Bronshtein commented on PHOENIX-4688: - This will require a very large patch, if anyone would like to start reviewing this, I opened https://github.com/apache/phoenix/pull/307 > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/28/18 6:57 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit, also it needs to source activate script * -Do we care if it only works on Linux?- It will work on MAC OS too * currently only works with Anaconda, would rather see it with virtualenv, though neither one comes pre installed on stock MAC OS was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit, also it needs to source activate script * -Do we care if it only works on Linux?- It will work on MAC OS too * currently only works with Anaconda, would rather see it with virtualenv > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/28/18 6:47 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit, also it needs to source activate script * -Do we care if it only works on Linux?- It will work on MAC OS too * currently only works with Anaconda, would rather see it with virtualenv was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit * -Do we care if it only works on Linux?- It will work on MAC OS too * currently only works with Anaconda, would rather see it with virtualenv > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/27/18 7:54 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit * -Do we care if it only works on Linux?- It will work on MAC OS too * currently only works with Anaconda, would rather see it with virtualenv was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit * Do we care if it only works on Linux? * currently only works with Anaconda, would rather see it with virtualenv > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/27/18 7:48 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * -The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations)- Render a custom krb5.conf if MAC * Currently a shell script is needed to launch. I tried taking out as much as possible but I still need to kinit * Do we care if it only works on Linux? * currently only works with Anaconda, would rather see it with virtualenv was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? * currently only works with Anaconda, would rather see it with virtualenv > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/27/18 7:06 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? * currently only works with Anaconda, would rather see it with virtualenv was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/27/18 5:32 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- Inherited from callers shell * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/27/18 5:14 PM: -- Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * -How do I pass down proxy settings? Or should I assume no proxy?- * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? was (Author: lbronshtein): Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * How do I pass down proxy settings? Or should I assume no proxy? * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525303#comment-16525303 ] Lev Bronshtein commented on PHOENIX-4688: - Test is working 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): CREATING PQS CONNECTION 2018-06-27 12:52:15,048 INFO [main] end2end.SecureQueryServerPhoenixDBIT(310): [[1, u'admin'], [2, u'user']] now just need to clean it up. * How do I pass down proxy settings? Or should I assume no proxy? * The heimdal kerberos utilities apple ships does not support krb5.conf format miniKDC ships (minor variations) * Currently a shell script is needed to launch * Do we care if it only works on Linux? > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525167#comment-16525167 ] Lev Bronshtein commented on PHOENIX-4688: - Having to render a custom KDC entry kdc = tcp/localhost:56819 > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525150#comment-16525150 ] Lev Bronshtein commented on PHOENIX-4688: - Enabling KRB5_DEBUG it looks like only UDP is being tried 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 set-error: -1765328242: Reached end of credential caches 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 set-error: -1765328243: Principal us...@example.com not found in any credential cache 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 set-error: -1765328234: Encryption type des-cbc-md5-deprecated not supported 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 Adding PA mech: ENCRYPTED_CHALLENGE 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 Adding PA mech: ENCRYPTED_TIMESTAMP 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 krb5_get_init_creds: loop 1 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 KDC sent 0 patypes 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 fast disabled, not doing any fast wrapping 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 Trying to find service kdc for realm EXAMPLE.COM flags 0 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 configuration file for realm EXAMPLE.COM found 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 submissing new requests to new host 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 host_create: setting hostname localhost 2018-06-27 10:28:22,731 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 connecting to host: udp ::1:56481 (localhost) tid: 0001 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 host_create: setting hostname localhost 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 Queuing host in future (in 3s), its the 2 address on the same name: udp 127.0.0.1:56481 (localhost) tid: 0002 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 writing packet: udp ::1:56481 (localhost) tid: 0001 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 reading packet: udp ::1:56481 (localhost) tid: 0001 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 failed to get nbytes from socket, no bytes there?: udp ::1:56481 (localhost) tid: 0001 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 host disconnected: udp ::1:56481 (localhost) tid: 0001 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 connecting to host: udp 127.0.0.1:56481 (localhost) tid: 0002 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 writing packet: udp 127.0.0.1:56481 (localhost) tid: 0002 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 reading packet: udp 127.0.0.1:56481 (localhost) tid: 0002 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 failed to get nbytes from socket, no bytes there?: udp 127.0.0.1:56481 (localhost) tid: 0002 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 host disconnected: udp 127.0.0.1:56481 (localhost) tid: 0002 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 no more hosts to send/recv packets to/from and no more hosts -> failure 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 Configuration exists for realm EXAMPLE.COM, wont go to DNS 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 out of hosts, waiting for replies 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 no more hosts to send/recv packets to/from trying to pulling more hosts 2018-06-27 10:28:22,732 INFO [main] end2end.SecureQueryServerPhoenixDBIT(308): 2018-06-27T10:28:22 set-error: -1765328228: unable to reach any KDC in realm EXAMPLE.COM, tried 1 KDC 2018-06-27 10:28:22,733 INFO [main]
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[
https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525011#comment-16525011
]
Lev Bronshtein commented on PHOENIX-4688:
-
I am closing in on the integration test. Currently have the following issues
* Unable to connect to miniKDC and perform a KINIT even after exporting
KRB5_CONFIG to be whatever the miniKDC dumped
2018-06-27 08:51:26,641 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
[libdefaults]
2018-06-27 08:51:26,641 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
default_realm = EXAMPLE.COM
2018-06-27 08:51:26,641 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
udp_preference_limit = 1
2018-06-27 08:51:26,641 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
2018-06-27 08:51:26,641 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
[realms]
2018-06-27 08:51:26,642 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
EXAMPLE.COM = {
2018-06-27 08:51:26,642 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
kdc = localhost:54339
2018-06-27 08:51:26,642 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312): }
2018-06-27 08:51:26,642 INFO [main] end2end.SecureQueryServerPhoenixDBIT(312):
RUNNING KINIT
2018-06-27 08:51:26,650 ERROR [main] end2end.SecureQueryServerPhoenixDBIT(316):
+ kinit -kt
/Users/lbronshtein/DEV/phoenix/phoenix-queryserver/target/SecureQueryServerPhoenixDBIT/keytabs/user1.keytab
us...@example.com
2018-06-27 08:51:26,650 ERROR [main] end2end.SecureQueryServerPhoenixDBIT(316):
kinit: krb5_get_init_creds: unable to reach any KDC in realm EXAMPLE.COM, tried
1 KDC
2018-06-27 08:51:26,650 ERROR [main] end2end.SecureQueryServerPhoenixDBIT(316):
+ cleanup
* There are a few places in the IT and the script it launches that hardcode
paths so I need some pointers on figuring out paths relative to the IT test or
perhaps another work around
> Add kerberos authentication to python-phoenixdb
> ---
>
> Key: PHOENIX-4688
> URL: https://issues.apache.org/jira/browse/PHOENIX-4688
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Priority: Minor
>
> In its current state python-phoenixdv does not support support kerberos
> authentication. Using a modern python http library such as requests or
> urllib it would be simple (if not trivial) to add this support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523112#comment-16523112 ] Lev Bronshtein commented on PHOENIX-4688: - installing from source directly ensures that only extra packages are downloaded pip install file:///Users/lbronshtein/DEV/phoenix/python/requests-kerberos >>> import requests_kerberos >>> requests_kerberos.__version__ '0.13.0.dev0-phoenixdb' So we can just install requests-kerberos locally and the phoenixdb > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522874#comment-16522874 ] Lev Bronshtein commented on PHOENIX-4688: - Rather then renaming a package I decided to change the version string -__version__ = '0.13.0.dev0' +__version__ = '0.13.0.dev0-phoenixdb' > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503729#comment-16503729 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/6/18 6:40 PM: - Currently this would be a manual where a user would be forced to spin up a virtual environment go to first install requests-kerberos by going to requests-kerberos-fork-module and running setup.py install. Following that going to phoenixdb-module and running setup.py install. Using a virtual environment would ensure that our kerberos-requests for does not conflict with any previously installed kerberos-requests module (I suppose I should also bump the version number to ensure that, alternatively do a full rename). If a user is careful then a virtual environment is not necessarily needed. In addition we do a rename and push both modules into pipy, so a simple pip install would do what is needed. Is there any ASF governance regarding the latter option? was (Author: lbronshtein): Currently this would be a manual where a user would be forced to spin up a virtual environment go to first install requests-kerberos by going to requests-kerberos-fork-module and running setup.py install. Following that going to phoenixdb-module and running setup.py install. Using a virtual environment would ensure that our kerberos-requests for does not conflict with any previously installed kerberos-requests module (I suppose I should also bump the version number to ensure that, alternatively do a full rename). If a user is careful then a virtual environment is not necessarily needed. Alternatively we do a rename and push both modules into pipy, so a simple pip install would do what is needed. > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503729#comment-16503729 ] Lev Bronshtein edited comment on PHOENIX-4688 at 6/6/18 6:39 PM: - Currently this would be a manual where a user would be forced to spin up a virtual environment go to first install requests-kerberos by going to requests-kerberos-fork-module and running setup.py install. Following that going to phoenixdb-module and running setup.py install. Using a virtual environment would ensure that our kerberos-requests for does not conflict with any previously installed kerberos-requests module (I suppose I should also bump the version number to ensure that, alternatively do a full rename). If a user is careful then a virtual environment is not necessarily needed. Alternatively we do a rename and push both modules into pipy, so a simple pip install would do what is needed. was (Author: lbronshtein): Currently this would be a manual where a user would be forced to spin up a virtual environment go to first install requests-kerberos by going to requests-kerberos-fork-module and running setup.py install. Following that going to phoenixdb-module and running setup.py install. Using a virtual environment would ensure that our kerberos-requests for does not conflict with any previously installed kerberos-requests module (I suppose I should also bump the version number to ensure that, alternatively do a full rename). > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503729#comment-16503729 ] Lev Bronshtein commented on PHOENIX-4688: - Currently this would be a manual where a user would be forced to spin up a virtual environment go to first install requests-kerberos by going to requests-kerberos-fork-module and running setup.py install. Following that going to phoenixdb-module and running setup.py install. Using a virtual environment would ensure that our kerberos-requests for does not conflict with any previously installed kerberos-requests module (I suppose I should also bump the version number to ensure that, alternatively do a full rename). > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503666#comment-16503666 ] Lev Bronshtein commented on PHOENIX-4688: - I have yet to look into renaming the forked requests-kerberos modules we should see for now if it works in a python virtual environment and then make this decision. I am hoping that the maintainers will eventually get back to me on this > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503503#comment-16503503 ] Lev Bronshtein commented on PHOENIX-4688: - Added my changes on top! > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481629#comment-16481629 ] Lev Bronshtein commented on PHOENIX-4688: - One more update, someone did finally look at [https://github.com/requests/requests-kerberos/pull/89,] however deemed it too complicated to be merged and asked for someone else to step up. I asked in turn to consider my PR [https://github.com/requests/requests-kerberos/pull/89|https://github.com/requests/requests-kerberos/pull/89,], which essentially accomplishes the same thing but with less code change > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481627#comment-16481627 ] Lev Bronshtein commented on PHOENIX-4688: - I need some advise on structural changes. We can do one of the following things # in the python directory [https://github.com/apache/phoenix/tree/master/python] create a subdirectory for python-phoenixdb and one for our patched version of python requests-kerberos. # Make a private fork of requests kerberos on github as in github/pu239ppy or github/jelser and use that Once the fork is made will make sure that every instance of requests_kerberos and requests-kerberos is renamed into phoenix-requests_kerberos and phoenix-requests-kerberos. The readme file will explicitly state that this is a fork and is only here until requests kerberos get back to us about merging > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440196#comment-16440196 ] Lev Bronshtein commented on PHOENIX-4688: - Still trying to get in touch with maintainers. Got a few redirects I am following up on > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437183#comment-16437183 ] Lev Bronshtein commented on PHOENIX-4688: - Another potential issue with this approach is that some user may at some point override our version of requests-kerberos with a newer one. This can be avoided if a local phoenix install decides to have its own python environment > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434258#comment-16434258 ] Lev Bronshtein commented on PHOENIX-4688: - We can bundle a fork and run "our_requests-kerberos/setup.py install" this will work for phoenix installs. However this is not an MO we want to support as we push pythondb-phoenix to pypi.python.org > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
[ https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433964#comment-16433964 ] Lev Bronshtein commented on PHOENIX-4688: - I currently have some code in progress at [https://github.com/pu239ppy/python-phoenixdb] There are two issues with it 1. I started working on this before I became aware that apache/phoenix has assumed ownership of python-phoenixdb from Lucas 2. In its current state the code relies on the fact that [https://github.com/requests/requests-kerberos/pull/115] will be merged, as an alternative to this there is [https://github.com/requests/requests-kerberos/pull/89] a much more ambitious PR, but one that is already a year with no merge. An alternative exists as urllib, however the same person who filed #89 for requests-kerberos has also filed [https://github.com/willthames/urllib_kerberos/pull/1] around the same time and this had also gone un merged. > Add kerberos authentication to python-phoenixdb > --- > > Key: PHOENIX-4688 > URL: https://issues.apache.org/jira/browse/PHOENIX-4688 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Priority: Minor > > In its current state python-phoenixdv does not support support kerberos > authentication. Using a modern python http library such as requests or > urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb
Lev Bronshtein created PHOENIX-4688: --- Summary: Add kerberos authentication to python-phoenixdb Key: PHOENIX-4688 URL: https://issues.apache.org/jira/browse/PHOENIX-4688 Project: Phoenix Issue Type: Improvement Reporter: Lev Bronshtein In its current state python-phoenixdv does not support support kerberos authentication. Using a modern python http library such as requests or urllib it would be simple (if not trivial) to add this support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4672) Fix naming of QUERY_SERVER_KERBEROS_HTTP_PRINCIPAL_ATTRIB
[
https://issues.apache.org/jira/browse/PHOENIX-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431071#comment-16431071
]
Lev Bronshtein commented on PHOENIX-4672:
-
I think we're all ok with this chnage
> Fix naming of QUERY_SERVER_KERBEROS_HTTP_PRINCIPAL_ATTRIB
> -
>
> Key: PHOENIX-4672
> URL: https://issues.apache.org/jira/browse/PHOENIX-4672
> Project: Phoenix
> Issue Type: Task
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Trivial
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4672.diff
>
>
> The HTTP-specific kerberos credentials implemented in PHOENIX-4533 introduce
> some ambiguity: It is presently
> {{phoenix.queryserver.kerberos.http.principal}}, but it should be
> {{phoenix.queryserver.http.kerberos.principal}} to match the rest of Hadoop,
> HBase, and Phoenix configuration kerberos principal properties.
> Need to update docs too.
> FYI [~lbronshtein]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4672) Fix naming of QUERY_SERVER_KERBEROS_HTTP_PRINCIPAL_ATTRIB
[
https://issues.apache.org/jira/browse/PHOENIX-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431071#comment-16431071
]
Lev Bronshtein edited comment on PHOENIX-4672 at 4/9/18 7:05 PM:
-
I think we're all ok with this change
was (Author: lbronshtein):
I think we're all ok with this chnage
> Fix naming of QUERY_SERVER_KERBEROS_HTTP_PRINCIPAL_ATTRIB
> -
>
> Key: PHOENIX-4672
> URL: https://issues.apache.org/jira/browse/PHOENIX-4672
> Project: Phoenix
> Issue Type: Task
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Trivial
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4672.diff
>
>
> The HTTP-specific kerberos credentials implemented in PHOENIX-4533 introduce
> some ambiguity: It is presently
> {{phoenix.queryserver.kerberos.http.principal}}, but it should be
> {{phoenix.queryserver.http.kerberos.principal}} to match the rest of Hadoop,
> HBase, and Phoenix configuration kerberos principal properties.
> Need to update docs too.
> FYI [~lbronshtein]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364841#comment-16364841 ] Lev Bronshtein commented on PHOENIX-4533: - Josh, is this what you are looking for? $ svn diff Index: site/publish/server.html === --- site/publish/server.html (revision 1824225) +++ site/publish/server.html (working copy) @@ -289,10 +289,20 @@ unset + phoenix.queryserver.http.keytab.file + The key to look for keytab file. This configuration MUST be specified if phoenix.queryserver.kerberos.http.principal is configured + unset + + phoenix.queryserver.kerberos.principal - The kerberos principal to use when authenticating. + The kerberos principal to use when authenticating. If phoenix.queryserver.kerberos.http.principal is not configured, the principlaa specified will be also used to both authenticate SPNEGO connections and to connect to HBase. Unless phoenix.queryserver.http.keytab.file is also specified, this configuration will be ignored unset + + phoenix.queryserver.kerberos.http.principal + The kerberos principal to use when authenticating SPNEGO connections + unset + phoenix.queryserver.dns.nameserver The DNS hostname > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362911#comment-16362911 ] Lev Bronshtein commented on PHOENIX-4533: - Can do the docs, I am not sure what should change for building, definitely for server, where are the source for the doc website? > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Fix For: 5.0.0, 4.14.0 > > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch, PHOENIX-4533.squash.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lev Bronshtein updated PHOENIX-4533: Attachment: PHOENIX-4533.3.patch > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359660#comment-16359660 ] Lev Bronshtein commented on PHOENIX-4533: - HttpParamImpersonationQueryServerIT is now passing as well, patch attached > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch, > PHOENIX-4533.3.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359217#comment-16359217 ] Lev Bronshtein commented on PHOENIX-4533: - Josh, you are right, anyway I fixed SecureQueryServerIT and provided a patch for that fix. Though honestly I am not sure how this test would have worked in the first place given the nature of the error. Hoping to have HttpParamImpersonationQueryServerIT done shortly as well > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lev Bronshtein updated PHOENIX-4533: Attachment: PHOENIX-4533.2.patch > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch, PHOENIX-4533.2.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347980#comment-16347980 ] Lev Bronshtein commented on PHOENIX-4533: - Fixed the tests as well. Also it looks like I incorrectly generated the last patch, so I created a new one and attached it. > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lev Bronshtein updated PHOENIX-4533: Attachment: PHOENIX-4533.1.patch > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lev Bronshtein updated PHOENIX-4533: Attachment: (was: PHOENIX-4533.1.patch) > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346859#comment-16346859 ] Lev Bronshtein commented on PHOENIX-4533: - Actually I think I already figured it out (though not clear how this affects other components). It looks like the login is done eternally. Just need to make sure the avatica server will still do SPNEGO auth > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346840#comment-16346840
]
Lev Bronshtein commented on PHOENIX-4533:
-
Josh, I am having some trouble understanding why this line is being set in both
tests
{code:java}
conf.setBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN, true);
{code}
Especially since this seems to turn off the specific parts we want to test
{code:java}
final boolean disableLogin =
getConf().getBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN,
QueryServicesOptions.DEFAULT_QUERY_SERVER_DISABLE_KERBEROS_LOGIN);
...
if (isKerberos && !disableSpnego && !disableLogin) {
hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
getConf().get(QueryServices.QUERY_SERVER_DNS_INTERFACE_ATTRIB, "default"),
getConf().get(QueryServices.QUERY_SERVER_DNS_NAMESERVER_ATTRIB, "default")));
if (LOG.isDebugEnabled()) {
LOG.debug("Login to " + hostname + " using " + getConf().get(
QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB)
+ " and principal " + getConf().get(
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB) + ".");
}
SecurityUtil.login(getConf(), QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB,
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB, hostname);
LOG.info("Login successful.");
} else {
hostname = InetAddress.getLocalHost().getHostName();
LOG.info(" Kerberos is off and hostname is : "+hostname);
}
{code}
> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP
> ecosystem to perform SPNEGO authentication. Since there can only be one
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing
> key material for local HTTP/ principal is shared among a few applications.
> With so many applications having access to the HTTP/ credentials, this
> increases the chances of an attack on the proxy user capabilities of Hadoop.
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346840#comment-16346840
]
Lev Bronshtein edited comment on PHOENIX-4533 at 1/31/18 1:43 PM:
--
Josh, I am having some trouble understanding why this line is being set in both
tests
{code:java}
conf.setBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN, true);
{code}
Especially since this seems to turn off the specific parts we want to test in
*phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java*
{code:java}
final boolean disableLogin =
getConf().getBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN,
QueryServicesOptions.DEFAULT_QUERY_SERVER_DISABLE_KERBEROS_LOGIN);
...
if (isKerberos && !disableSpnego && !disableLogin) {
hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
getConf().get(QueryServices.QUERY_SERVER_DNS_INTERFACE_ATTRIB, "default"),
getConf().get(QueryServices.QUERY_SERVER_DNS_NAMESERVER_ATTRIB, "default")));
if (LOG.isDebugEnabled()) {
LOG.debug("Login to " + hostname + " using " + getConf().get(
QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB)
+ " and principal " + getConf().get(
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB) + ".");
}
SecurityUtil.login(getConf(), QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB,
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB, hostname);
LOG.info("Login successful.");
} else {
hostname = InetAddress.getLocalHost().getHostName();
LOG.info(" Kerberos is off and hostname is : "+hostname);
}
{code}
was (Author: lbronshtein):
Josh, I am having some trouble understanding why this line is being set in both
tests
{code:java}
conf.setBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN, true);
{code}
Especially since this seems to turn off the specific parts we want to test
{code:java}
final boolean disableLogin =
getConf().getBoolean(QueryServices.QUERY_SERVER_DISABLE_KERBEROS_LOGIN,
QueryServicesOptions.DEFAULT_QUERY_SERVER_DISABLE_KERBEROS_LOGIN);
...
if (isKerberos && !disableSpnego && !disableLogin) {
hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
getConf().get(QueryServices.QUERY_SERVER_DNS_INTERFACE_ATTRIB, "default"),
getConf().get(QueryServices.QUERY_SERVER_DNS_NAMESERVER_ATTRIB, "default")));
if (LOG.isDebugEnabled()) {
LOG.debug("Login to " + hostname + " using " + getConf().get(
QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB)
+ " and principal " + getConf().get(
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB) + ".");
}
SecurityUtil.login(getConf(), QueryServices.QUERY_SERVER_KEYTAB_FILENAME_ATTRIB,
QueryServices.QUERY_SERVER_KERBEROS_PRINCIPAL_ATTRIB, hostname);
LOG.info("Login successful.");
} else {
hostname = InetAddress.getLocalHost().getHostName();
LOG.info(" Kerberos is off and hostname is : "+hostname);
}
{code}
> Phoenix Query Server should not use SPNEGO principal to proxy user requests
> ---
>
> Key: PHOENIX-4533
> URL: https://issues.apache.org/jira/browse/PHOENIX-4533
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Lev Bronshtein
>Assignee: Lev Bronshtein
>Priority: Minor
> Attachments: PHOENIX-4533.1.patch
>
>
> Currently the HTTP/ principal is used by various components in the HADOOP
> ecosystem to perform SPNEGO authentication. Since there can only be one
> HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing
> key material for local HTTP/ principal is shared among a few applications.
> With so many applications having access to the HTTP/ credentials, this
> increases the chances of an attack on the proxy user capabilities of Hadoop.
> This JIRA proposes that two different key tabs can be used to
> 1. Authenticate kerberized web requests
> 2. Communicate with the phoenix back end
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343844#comment-16343844 ] Lev Bronshtein commented on PHOENIX-4533: - First par is done, two days later kinit as my user and access PQS, still able to run queries. I will look into the tests in a bit as well > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343844#comment-16343844 ] Lev Bronshtein edited comment on PHOENIX-4533 at 1/29/18 7:15 PM: -- First part is done, two days later kinit as my user and access PQS, still able to run queries. I will look into the tests in a bit as well was (Author: lbronshtein): First par is done, two days later kinit as my user and access PQS, still able to run queries. I will look into the tests in a bit as well > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341410#comment-16341410 ] Lev Bronshtein commented on PHOENIX-4533: - Also here are my Configuration changes h3. *BEFORE* h4. *hbase-site.xml* phoenix.queryserver.kerberos.principal HTTP/f-bcpc-vm2.bcpc.example@bcpc.example.com phoenix.queryserver.keytab.file /etc/security/keytabs/spnego.service.keytab phoenix.queryserver.serialization JSON hadoop.proxyuser.HTTP.hosts * hadoop.proxyuser.HTTP.users * h4. core-site.xml hadoop.proxyuser.HTTP.hosts * hadoop.proxyuser.HTTP.users * h3. *AFTER* h4. *hbase-site.xml* phoenix.queryserver.kerberos.http.principal HTTP/f-bcpc-vm1.bcpc.example@bcpc.example.com phoenix.queryserver.http.keytab.file /etc/security/keytabs/spnego.service.keytab phoenix.queryserver.kerberos.principal phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com phoenix.queryserver.keytab.file /etc/security/keytabs/phoenixqs.service.keytab h4. core-site.xml hadoop.proxyuser.phoenixqs.hosts * hadoop.proxyuser.phoenixqs.users * > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this > increases the chances of an attack on the proxy user capabilities of Hadoop. > This JIRA proposes that two different key tabs can be used to > 1. Authenticate kerberized web requests > 2. Communicate with the phoenix back end -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341283#comment-16341283
]
Lev Bronshtein edited comment on PHOENIX-4533 at 1/26/18 5:06 PM:
--
Looks like it works. I first set the max lifetime for the principal in
question to 5 minutes using kadmin and verified, see text in bold below
{quote}1. kadmin.local: modprinc -maxlife "5 minutes"
phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified.
2. kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
Expiration date: [never]
Last password change: Fri Jan 19 20:22:31 UTC 2018
Password expiration date: [none]
*Maximum ticket life: 0 days 00:05:00*
Maximum renewable life: 7 days 00:00:00
Last modified: Fri Jan 26 16:27:47 UTC 2018 (root/ad...@bcpc.example.com)
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 3
Key: vno 2, arcfour-hmac, no salt
Key: vno 2, des3-cbc-sha1, no salt
Key: vno 2, des-cbc-crc, no salt
MKey: vno 1
Attributes:
Policy: [none]
{quote}
And attempted to access PQS a few times in the span of an hour, you can see
here that PQS will realize that its TGT has expired and needs renewal.
Following, it performs a relogin (see text in RED)
2018-01-26 11:58:58,356 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
2018-01-26 11:58:58,379 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
2018-01-26 11:58:58,386 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
(auth:KERBEROS)
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
{color:#FF}2018-01-26 11:58:58,390 DEBUG
org.apache.hadoop.security.UserGroupInformation: PrivilegedActionException
as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
cause:javax.security.sasl.SaslException:{color} {color:#FF}*GSS initiate
failed [Caused by GSSException: No valid credentials provided (Mechanism level:
Failed to find any Kerberos tgt)]*{color}
2018-01-26 11:58:58,391 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
(auth:KERBEROS)
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:637)
2018-01-26 11:58:58,393 DEBUG org.apache.hadoop.security.UserGroupInformation:
Initiating logout for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
{color:#FF}2018-01-26 11:58:58,394 DEBUG
org.apache.hadoop.security.UserGroupInformation: hadoop logout{color}
{color:#FF}2018-01-26 11:58:58,394 DEBUG
org.apache.hadoop.security.UserGroupInformation: Initiating re-login for
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com{color}
{color:#FF} 2018-01-26 11:58:58,398 DEBUG
org.apache.hadoop.security.UserGroupInformation: hadoop login{color}
2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation:
hadoop login commit
2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation:
using existing subject:[phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com,
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com]
2018-01-26 11:59:01,227 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
(auth:KERBEROS)
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
2018-01-26 11:59:01,299 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
was (Author: lbronshtein):
Looks like it works. I first set the max lifetime for the principal in
question to 5 minutes using kadmin
{quote}1. kadmin.local: modprinc -maxlife "5 minutes"
phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified.
2. kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal:
[jira] [Comment Edited] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[
https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341283#comment-16341283
]
Lev Bronshtein edited comment on PHOENIX-4533 at 1/26/18 5:04 PM:
--
Looks like it works. I first set the max lifetime for the principal in
question to 5 minutes using kadmin
{quote}1. kadmin.local: modprinc -maxlife "5 minutes"
phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified.
2. kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
Expiration date: [never]
Last password change: Fri Jan 19 20:22:31 UTC 2018
Password expiration date: [none]
*Maximum ticket life: 0 days 00:05:00*
Maximum renewable life: 7 days 00:00:00
Last modified: Fri Jan 26 16:27:47 UTC 2018 (root/ad...@bcpc.example.com)
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 3
Key: vno 2, arcfour-hmac, no salt
Key: vno 2, des3-cbc-sha1, no salt
Key: vno 2, des-cbc-crc, no salt
MKey: vno 1
Attributes:
Policy: [none]
{quote}
And attempted to access PQS a few times in the span of an hour, you can see
here that PQS will realize that its TGT has expired and needs renewal
2018-01-26 11:58:58,356 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
2018-01-26 11:58:58,379 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
2018-01-26 11:58:58,386 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
(auth:KERBEROS)
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
2018-01-26 11:58:58,390 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedActionException
as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
cause:javax.security.sasl.SaslException: *GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]*
2018-01-26 11:58:58,391 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
(auth:KERBEROS)
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:637)
2018-01-26 11:58:58,393 DEBUG org.apache.hadoop.security.UserGroupInformation:
Initiating logout for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation:
hadoop logout
2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation:
Initiating re-login for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
2018-01-26 11:58:58,398 DEBUG org.apache.hadoop.security.UserGroupInformation:
hadoop login
2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation:
hadoop login commit
2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation:
using existing subject:[phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com,
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com]
2018-01-26 11:59:01,227 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
(auth:KERBEROS)
from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
2018-01-26 11:59:01,299 DEBUG org.apache.hadoop.security.UserGroupInformation:
PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via
phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS)
from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313)
was (Author: lbronshtein):
Looks like it works. I first set the max lifetime for the principal in
question to 5 minutes using kadmin
kadmin.local: modprinc -maxlife "5 minutes"
phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified.
kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com
Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com
Expiration date: [never]
Last password change: Fri Jan 19 20:22:31 UTC 2018
Password expiration date: [none]
*Maximum ticket life: 0 days 00:05:00*
Maximum renewable life: 7 days 00:00:00
Last modified:
[jira] [Comment Edited] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341283#comment-16341283 ] Lev Bronshtein edited comment on PHOENIX-4533 at 1/26/18 5:02 PM: -- Looks like it works. I first set the max lifetime for the principal in question to 5 minutes using kadmin kadmin.local: modprinc -maxlife "5 minutes" phoenixqs/f-bcpc-vm1.bcpc.example.com Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified. kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com Expiration date: [never] Last password change: Fri Jan 19 20:22:31 UTC 2018 Password expiration date: [none] *Maximum ticket life: 0 days 00:05:00* Maximum renewable life: 7 days 00:00:00 Last modified: Fri Jan 26 16:27:47 UTC 2018 (root/ad...@bcpc.example.com) Last successful authentication: [never] Last failed authentication: [never] Failed password attempts: 0 Number of keys: 3 Key: vno 2, arcfour-hmac, no salt Key: vno 2, des3-cbc-sha1, no salt Key: vno 2, des-cbc-crc, no salt MKey: vno 1 Attributes: Policy: [none] 2018-01-26 11:58:58,356 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) 2018-01-26 11:58:58,379 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) 2018-01-26 11:58:58,386 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734) 2018-01-26 11:58:58,390 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedActionException as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2018-01-26 11:58:58,391 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:637) 2018-01-26 11:58:58,393 DEBUG org.apache.hadoop.security.UserGroupInformation: Initiating logout for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop logout 2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: Initiating re-login for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 2018-01-26 11:58:58,398 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login 2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login commit 2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: using existing subject:[phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com, phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com] 2018-01-26 11:59:01,227 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734) 2018-01-26 11:59:01,299 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) was (Author: lbronshtein): Looks like it works. I first set the max lifetime for the principal in question to 5 minutes using kadmin bq kadmin.local: modprinc -maxlife "5 minutes" phoenixqs/f-bcpc-vm1.bcpc.example.com Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified. kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com Expiration date: [never] Last password change: Fri Jan 19 20:22:31 UTC 2018 Password expiration date: [none] Maximum ticket life: 0 days 00:05:00 Maximum renewable life: 7 days 00:00:00 Last modified: Fri Jan 26 16:27:47 UTC 2018 (root/ad...@bcpc.example.com) Last successful authentication: [never] Last failed authentication: [never] Failed password attempts: 0
[jira] [Commented] (PHOENIX-4533) Phoenix Query Server should not use SPNEGO principal to proxy user requests
[ https://issues.apache.org/jira/browse/PHOENIX-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341283#comment-16341283 ] Lev Bronshtein commented on PHOENIX-4533: - Looks like it works. I first set the max lifetime for the principal in question to 5 minutes using kadmin bq kadmin.local: modprinc -maxlife "5 minutes" phoenixqs/f-bcpc-vm1.bcpc.example.com Principal "phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com" modified. kadmin.local: getprinc phoenixqs/f-bcpc-vm1.bcpc.example.com Principal: phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com Expiration date: [never] Last password change: Fri Jan 19 20:22:31 UTC 2018 Password expiration date: [none] Maximum ticket life: 0 days 00:05:00 Maximum renewable life: 7 days 00:00:00 Last modified: Fri Jan 26 16:27:47 UTC 2018 (root/ad...@bcpc.example.com) Last successful authentication: [never] Last failed authentication: [never] Failed password attempts: 0 Number of keys: 3 Key: vno 2, arcfour-hmac, no salt Key: vno 2, des3-cbc-sha1, no salt Key: vno 2, des-cbc-crc, no salt MKey: vno 1 Attributes: Policy: [none] 2018-01-26 11:58:58,356 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) 2018-01-26 11:58:58,379 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) 2018-01-26 11:58:58,386 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734) 2018-01-26 11:58:58,390 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedActionException as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2018-01-26 11:58:58,391 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:637) 2018-01-26 11:58:58,393 DEBUG org.apache.hadoop.security.UserGroupInformation: Initiating logout for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop logout 2018-01-26 11:58:58,394 DEBUG org.apache.hadoop.security.UserGroupInformation: Initiating re-login for phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com 2018-01-26 11:58:58,398 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login 2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: hadoop login commit 2018-01-26 11:58:58,399 DEBUG org.apache.hadoop.security.UserGroupInformation: using existing subject:[phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com, phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com] 2018-01-26 11:59:01,227 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734) 2018-01-26 11:59:01,299 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:ubu...@bcpc.example.com (auth:PROXY) via phoenixqs/f-bcpc-vm1.bcpc.example@bcpc.example.com (auth:KERBEROS) from:org.apache.phoenix.queryserver.server.Main$PhoenixDoAsCallback.doAsRemoteUser(Main.java:313) > Phoenix Query Server should not use SPNEGO principal to proxy user requests > --- > > Key: PHOENIX-4533 > URL: https://issues.apache.org/jira/browse/PHOENIX-4533 > Project: Phoenix > Issue Type: Improvement >Reporter: Lev Bronshtein >Assignee: Lev Bronshtein >Priority: Minor > Attachments: PHOENIX-4533.1.patch > > > Currently the HTTP/ principal is used by various components in the HADOOP > ecosystem to perform SPNEGO authentication. Since there can only be one > HTTP/ per host, even outside of the Hadoop ecosystem, the keytab containing > key material for local HTTP/ principal is shared among a few applications. > With so many applications having access to the HTTP/ credentials, this
