[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096944#comment-16096944 ] Josh Elser commented on PHOENIX-3598: - bq. Does this mean we have no test coverage for this feature in 0.98? Yeah, we're presently lacking the e2e coverage for 0.98 and 1.1 branches. bq. If so, for the documentation, can you please make it clear this feature in 0.98 is of unknown quality (maybe alpha, maybe not working at all) and is use at your own risk. Let me drop a note on PHOENIX-4019 so I don't forget. bq. Maybe we should start another discuss thread on dev. Is it better to have a feature only in some branches (i.e. let the branches diverge) or is it better to have them in all even without test coverage? Either way, documentation and support becomes much harder. Yeah, this one is hard, especially with the wide breadth of versions that Phoenix tries to support. Let me put some thought into this and send a note. It gets frustrating when, after development/review you realize that the patch is actually only good on some branches :) For this case specifically, it may be possible to copy-paste the relevant code from HBase into our codebase as a short-term workaround. I'm also open to remedying this specific case in that way -- I don't think it would be too bad. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Josh Elser > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.4.x-HBase-0.98.patch, PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096631#comment-16096631 ] James Taylor commented on PHOENIX-3598: --- Thanks for the back port, [~elserj]. Does this mean we have no test coverage for this feature in 0.98? If so, for the documentation, can you please make it clear this feature in 0.98 is of unknown quality (maybe alpha, maybe not working at all) and is use at your own risk. Maybe we should start another discuss thread on dev. Is it better to have a feature only in some branches (i.e. let the branches diverge) or is it better to have them in all even without test coverage? Either way, documentation and support becomes much harder. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Josh Elser > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.4.x-HBase-0.98.patch, PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095160#comment-16095160 ] Josh Elser commented on PHOENIX-3598: - bq. against 4.x-HBase-0.98 branch at commit ca1105630dab43a8629e2efa0171d914e0140b3e. Uhh, test-patch.sh seems to be a bit confused. That commit ID is in master, not the 0.98 branch. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Josh Elser > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.4.x-HBase-0.98.patch, PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095152#comment-16095152 ] Hadoop QA commented on PHOENIX-3598: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12878224/PHOENIX-3598.002.4.x-HBase-0.98.patch against 4.x-HBase-0.98 branch at commit ca1105630dab43a8629e2efa0171d914e0140b3e. ATTACHMENT ID: 12878224 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1231//console This message is automatically generated. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Josh Elser > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.4.x-HBase-0.98.patch, PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095133#comment-16095133 ] Josh Elser commented on PHOENIX-3598: - bq. Now that I think about it some more, I have this nagging thought that I was going to remove these tests because the dependent upstream changes in HBase don't exist in 0.98. Either way, I'll rectify it. Yeah, this was it. It looks like I cherry-pick'ed from master (or something other than the HBase-1.x branch). Both 1.x and 0.98 don't have the test-code for setting up the fully-kerberized test environment. Will put up a 0.98 patch shortly to make sure I didn't screw it up again. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Josh Elser > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094848#comment-16094848 ] Josh Elser commented on PHOENIX-3598: - Thanks James/Geoffrey. I apologize, I typically do a cycle of cherry-pick+mvn-package for each branch. Obviously, I forgot to do that for each branch here. Now that I think about it some more, I have this nagging thought that I was going to remove these tests because the dependent upstream changes in HBase don't exist in 0.98. Either way, I'll rectify it. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Josh Elser > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090354#comment-16090354 ] James Taylor commented on PHOENIX-3598: --- Revert commit to 4.x-HBase-0.98 branch as it's not compiling. Minimum bar for committing anything should be a successful run of "mvn package" > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088443#comment-16088443 ] James Taylor commented on PHOENIX-3598: --- [~elserj] - would it be possible to have a pre-commit hook that rejects a commit that either doesn't compile or doesn't pass our "mvn package" unit tests? What should we do with this particular commit? Revert it completely or attempt to fix it? > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085164#comment-16085164 ] Hudson commented on PHOENIX-3598: - FAILURE: Integrated in Jenkins build Phoenix-master #1685 (See [https://builds.apache.org/job/Phoenix-master/1685/]) PHOENIX-3598 Implement HTTP parameter impersonation for PQS (elserj: rev f2eac858eab64fda3eacf7f6e1b2ab9656bf4cfa) * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java * (edit) phoenix-queryserver/src/it/resources/log4j.properties * (add) phoenix-queryserver/src/test/java/org/apache/phoenix/queryserver/server/PhoenixRemoteUserExtractorTest.java * (add) phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDatabaseMetaData.java * (add) phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/HttpParamImpersonationQueryServerIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java * (edit) phoenix-queryserver/pom.xml * (edit) phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085114#comment-16085114 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user Wancy commented on the issue: https://github.com/apache/phoenix/pull/265 Thanks @joshelser !! > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085115#comment-16085115 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user Wancy closed the pull request at: https://github.com/apache/phoenix/pull/265 > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085039#comment-16085039 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on the issue: https://github.com/apache/phoenix/pull/265 Woops. I forgot to close this via commit message. If you could close it at your convenience, @Wancy, I'd appreciate it! > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Fix For: 4.12.0 > > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084527#comment-16084527 ] Josh Elser commented on PHOENIX-3598: - Thanks Devaraj. Trying to get this one committed, running into a few things. PHOENIX-4014 is blocking the 1.2 branch. I'm going to have to remove the tests for 0.98 and 1.1 as they don't contain the necessary fixes in HBase. As long as we have those tests on the newer versions, I feel ok about it. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083553#comment-16083553 ] Devaraj Das commented on PHOENIX-3598: -- LGTM. Nice tests. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082871#comment-16082871 ] Hadoop QA commented on PHOENIX-3598: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12876432/PHOENIX-3598.002.patch against master branch at commit b0109feb92fdd9e19bb6f70412d0c476ec60d3d4. ATTACHMENT ID: 12876432 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 50 warning messages. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: +public static final TableName SYSTEM_SCHEMA_HBASE_TABLE_NAME = TableName.valueOf(SYSTEM_SCHEMA_NAME); +public static final TableName SYSTEM_STATS_HBASE_TABLE_NAME = TableName.valueOf(SYSTEM_STATS_NAME); +public static final TableName SYSTEM_SEQUENCE_HBASE_TABLE_NAME = TableName.valueOf(SYSTEM_SEQUENCE_NAME); +public static final TableName SYSTEM_FUNCTION_HBASE_TABLE_NAME = TableName.valueOf(SYSTEM_FUNCTION_NAME); +public static final String QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR_ATTRIB = "phoenix.queryserver.withRemoteUserExtractor"; +public static final String QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM = "phoenix.queryserver.remoteUserExtractor.param"; +public static final String QUERY_SERVER_DISABLE_KERBEROS_LOGIN = "phoenix.queryserver.disable.kerberos.login"; +private static final List SYSTEM_TABLE_NAMES = Arrays.asList(PhoenixDatabaseMetaData.SYSTEM_CATALOG_HBASE_TABLE_NAME, +conf.set(DFSConfigKeys.DFS_NAMENODE_KERBEROS_PRINCIPAL_KEY, SERVICE_PRINCIPAL + "@" + KDC.getRealm()); +conf.set(DFSConfigKeys.DFS_DATANODE_KERBEROS_PRINCIPAL_KEY, SERVICE_PRINCIPAL + "@" + KDC.getRealm()); {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1195//testReport/ Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1195//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1195//console This message is automatically generated. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080687#comment-16080687 ] James Taylor commented on PHOENIX-3598: --- [~gjacoby], [~churromorales], [~vincentpoon], [~rahulshrivastava], [~apurtell] - maybe one of you guys could review this? > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch, > PHOENIX-3598.002.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078522#comment-16078522 ] Hadoop QA commented on PHOENIX-3598: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12876120/PHOENIX-3598.001.patch against master branch at commit b0109feb92fdd9e19bb6f70412d0c476ec60d3d4. ATTACHMENT ID: 12876120 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1179//console This message is automatically generated. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078516#comment-16078516 ] Josh Elser commented on PHOENIX-3598: - Also, a rebase of Shi's work (with one minor log message tweak) plus this patch is at https://github.com/joshelser/phoenix/tree/3598-pqs-doAs > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch, PHOENIX-3598.001.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077189#comment-16077189 ] Josh Elser commented on PHOENIX-3598: - I finally have a working test framework locally :). Let me clean this up and get an end-to-end test working for your patch. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065615#comment-16065615 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124414777 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -273,6 +282,54 @@ public int run(String[] args) throws Exception { } } + // add remoteUserExtractor to builder if enabled + @VisibleForTesting + public void setRemoteUserExtractorIfNecessary(HttpServer.Builder builder, Configuration conf) { +if (conf.getBoolean(QueryServices.QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR_ATTRIB, + QueryServicesOptions.DEFAULT_QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR)) { + builder.withRemoteUserExtractor(new PhoenixRemoteUserExtractor(conf)); +} + } + + /** + * Use the correctly way to extract end user. + */ + + static class PhoenixRemoteUserExtractor implements RemoteUserExtractor{ +private final HttpQueryStringParameterRemoteUserExtractor paramRemoteUserExtractor; +private final HttpRequestRemoteUserExtractor requestRemoteUserExtractor; +private final String userExtractParam; + +public PhoenixRemoteUserExtractor(Configuration conf) { + this.requestRemoteUserExtractor = new HttpRequestRemoteUserExtractor(); + this.userExtractParam = conf.get(QueryServices.QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM, + QueryServicesOptions.DEFAULT_QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM); + this.paramRemoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(userExtractParam); +} + +@Override +public String extract(HttpServletRequest request) throws RemoteUserExtractionException { + if (request.getParameter(userExtractParam) != null) { +String extractedUser = paramRemoteUserExtractor.extract(request); +UserGroupInformation ugi = UserGroupInformation.createRemoteUser(request.getRemoteUser()); +UserGroupInformation proxyUser = UserGroupInformation.createProxyUser(extractedUser, ugi); --- End diff -- Agreed! I think the work you've put in would be nice to support for the non-Kerberos case, but let's not hold up this change for that. I will try to write up a test case for PQS (mini-hbase, mini-kdc, and PQS) to validate your changes here before I commit. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065599#comment-16065599 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user Wancy commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124413815 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -273,6 +282,54 @@ public int run(String[] args) throws Exception { } } + // add remoteUserExtractor to builder if enabled + @VisibleForTesting + public void setRemoteUserExtractorIfNecessary(HttpServer.Builder builder, Configuration conf) { +if (conf.getBoolean(QueryServices.QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR_ATTRIB, + QueryServicesOptions.DEFAULT_QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR)) { + builder.withRemoteUserExtractor(new PhoenixRemoteUserExtractor(conf)); +} + } + + /** + * Use the correctly way to extract end user. + */ + + static class PhoenixRemoteUserExtractor implements RemoteUserExtractor{ +private final HttpQueryStringParameterRemoteUserExtractor paramRemoteUserExtractor; +private final HttpRequestRemoteUserExtractor requestRemoteUserExtractor; +private final String userExtractParam; + +public PhoenixRemoteUserExtractor(Configuration conf) { + this.requestRemoteUserExtractor = new HttpRequestRemoteUserExtractor(); + this.userExtractParam = conf.get(QueryServices.QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM, + QueryServicesOptions.DEFAULT_QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM); + this.paramRemoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(userExtractParam); +} + +@Override +public String extract(HttpServletRequest request) throws RemoteUserExtractionException { + if (request.getParameter(userExtractParam) != null) { +String extractedUser = paramRemoteUserExtractor.extract(request); +UserGroupInformation ugi = UserGroupInformation.createRemoteUser(request.getRemoteUser()); +UserGroupInformation proxyUser = UserGroupInformation.createProxyUser(extractedUser, ugi); --- End diff -- Hi @joshelser, I think I understand your concern of the edge cases. I originally wanna add it just for kerberos cases, but I thought user extract could be extended to simple auth as well in the future, but seems a lot more work needs to be done than I thought. Also like you said most people just want point1. I think for this jira just add it for the kerberos case is more practical. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065602#comment-16065602 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user Wancy commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124413926 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -228,7 +235,9 @@ public int run(String[] args) throws Exception { builder.withSpnego(ugi.getUserName(), additionalAllowedRealms) .withAutomaticLogin(keytab) .withImpersonation(new PhoenixDoAsCallback(ugi, getConf())); + } + setRemoteUserExtractorIfNecessary(builder, getConf()); --- End diff -- Agree to put it inside the if-block for only kerberos case :) > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065567#comment-16065567 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124402049 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -273,6 +282,54 @@ public int run(String[] args) throws Exception { } } + // add remoteUserExtractor to builder if enabled + @VisibleForTesting + public void setRemoteUserExtractorIfNecessary(HttpServer.Builder builder, Configuration conf) { +if (conf.getBoolean(QueryServices.QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR_ATTRIB, + QueryServicesOptions.DEFAULT_QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR)) { + builder.withRemoteUserExtractor(new PhoenixRemoteUserExtractor(conf)); +} + } + + /** + * Use the correctly way to extract end user. + */ + + static class PhoenixRemoteUserExtractor implements RemoteUserExtractor{ +private final HttpQueryStringParameterRemoteUserExtractor paramRemoteUserExtractor; +private final HttpRequestRemoteUserExtractor requestRemoteUserExtractor; +private final String userExtractParam; + +public PhoenixRemoteUserExtractor(Configuration conf) { + this.requestRemoteUserExtractor = new HttpRequestRemoteUserExtractor(); + this.userExtractParam = conf.get(QueryServices.QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM, + QueryServicesOptions.DEFAULT_QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM); + this.paramRemoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(userExtractParam); +} + +@Override +public String extract(HttpServletRequest request) throws RemoteUserExtractionException { + if (request.getParameter(userExtractParam) != null) { --- End diff -- We should put a `requestRemoteUserExtractor.extract(request)` at the top of this method implementation. We should be using it in both branches of the conditional (replacing the `request.getRemoteUser()` call you have below) > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065566#comment-16065566 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124409282 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -228,7 +235,9 @@ public int run(String[] args) throws Exception { builder.withSpnego(ugi.getUserName(), additionalAllowedRealms) .withAutomaticLogin(keytab) .withImpersonation(new PhoenixDoAsCallback(ugi, getConf())); + } + setRemoteUserExtractorIfNecessary(builder, getConf()); --- End diff -- With respect to my long-winded comment below, if you're only looking to support Kerberos, we want to move this line into the above if-block. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065565#comment-16065565 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124409157 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -273,6 +282,54 @@ public int run(String[] args) throws Exception { } } + // add remoteUserExtractor to builder if enabled + @VisibleForTesting + public void setRemoteUserExtractorIfNecessary(HttpServer.Builder builder, Configuration conf) { +if (conf.getBoolean(QueryServices.QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR_ATTRIB, + QueryServicesOptions.DEFAULT_QUERY_SERVER_WITH_REMOTEUSEREXTRACTOR)) { + builder.withRemoteUserExtractor(new PhoenixRemoteUserExtractor(conf)); +} + } + + /** + * Use the correctly way to extract end user. + */ + + static class PhoenixRemoteUserExtractor implements RemoteUserExtractor{ +private final HttpQueryStringParameterRemoteUserExtractor paramRemoteUserExtractor; +private final HttpRequestRemoteUserExtractor requestRemoteUserExtractor; +private final String userExtractParam; + +public PhoenixRemoteUserExtractor(Configuration conf) { + this.requestRemoteUserExtractor = new HttpRequestRemoteUserExtractor(); + this.userExtractParam = conf.get(QueryServices.QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM, + QueryServicesOptions.DEFAULT_QUERY_SERVER_REMOTEUSEREXTRACTOR_PARAM); + this.paramRemoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(userExtractParam); +} + +@Override +public String extract(HttpServletRequest request) throws RemoteUserExtractionException { + if (request.getParameter(userExtractParam) != null) { +String extractedUser = paramRemoteUserExtractor.extract(request); +UserGroupInformation ugi = UserGroupInformation.createRemoteUser(request.getRemoteUser()); +UserGroupInformation proxyUser = UserGroupInformation.createProxyUser(extractedUser, ugi); --- End diff -- In re-reading the above, I'm a little worried about the edge-cases. With PQS right now, we have the following cases "supported" 1) Kerberos+SPNEGO as the Kerberos user (els...@example.com authenticates to PQS and the PQS credentials are used to query Phoenix as els...@example.com) 2) Kerberos auth with HBase but no SPNEGO for PQS clients (legacy support for how things used to work before the SPNEGO auth was built -- PQS user does everything for users) 3) Without Kerberos, all queries run as the PQS user (out of the box). Avatica supports more than what point 3 above does, but we haven't prioritized wiring that up as most people just want point 1. @Wancy, I had originally thought you were just trying to enable a PQS client with Kerberos credentials to say that they are someone else (extension of point 1 -- Credentials to PQS are for "elserj" but instead of querying Phoenix as "elserj", query as "bob"). Did you also want this to work for cases when Kerberos is not in the mix? I think that would require some additional changes as I don't think this will work as-is. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065259#comment-16065259 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user Wancy commented on the issue: https://github.com/apache/phoenix/pull/265 Hi @joshelser, I made some changes according to your comments, please review, thanks. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065008#comment-16065008 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124312271 --- Diff: phoenix-queryserver/src/test/java/org/apache/phoenix/queryserver/server/PhoenixRemoteUserExtractorTest.java --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to you under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.queryserver.server; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import org.apache.calcite.avatica.server.RemoteUserExtractionException; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.security.UserGroupInformation; +import org.apache.hadoop.security.authorize.AuthorizationException; +import org.apache.hadoop.security.authorize.ProxyUsers; +import org.apache.phoenix.queryserver.server.QueryServer.PhoenixRemoteUserExtractor; +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.http.HttpServletRequest; + +/** + * Tests for the RemoteUserExtractor Method Avatica provides for Phoenix to implement. + */ +public class PhoenixRemoteUserExtractorTest { + private static final Logger LOG = LoggerFactory.getLogger(PhoenixRemoteUserExtractorTest.class); + + @Test + public void testUseDoAsSuccess() { +HttpServletRequest request = mock(HttpServletRequest.class); +when(request.getRemoteUser()).thenReturn("proxyserver"); +when(request.getParameter("doAs")).thenReturn("enduser"); +when(request.getRemoteAddr()).thenReturn("localhost:1234"); + +Configuration conf = new Configuration(false); +conf.set("hadoop.proxyuser.proxyserver.groups", "*"); +conf.set("hadoop.proxyuser.proxyserver.hosts", "*"); +conf.set("phoenix.queryserver.doAs.enabled", "true"); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); + +PhoenixRemoteUserExtractor extractor = new PhoenixRemoteUserExtractor(conf); +try { + assertEquals("enduser", extractor.extract(request)); +} catch (RemoteUserExtractionException e) { + LOG.info(e.getMessage()); +} + } + + @Test + public void testDoNotUseDoAs() { --- End diff -- No, there is no getter on the builder to verify it's called. Instead you can use the `Mockito.verify(builder)` method. Something like: ```java Configuration conf = createTestConfiguration(); Builder b = Mockito.mock(Builder.class); Mockito.when(b.withRemoteUserExtractor(Mockito.any(PhoenixRemoteUserExtractor.class))).thenReturn(b); setRemoteUserExtractorIfNecessary(b, conf); Mockito.verify(b); ``` This should essentially verify that `withRemoteUserExtractor` was invoked by `setRemoteUserExtractorIfNecessary` > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064037#comment-16064037 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user Wancy commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124153233 --- Diff: phoenix-queryserver/src/test/java/org/apache/phoenix/queryserver/server/PhoenixRemoteUserExtractorTest.java --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to you under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.queryserver.server; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import org.apache.calcite.avatica.server.RemoteUserExtractionException; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.security.UserGroupInformation; +import org.apache.hadoop.security.authorize.AuthorizationException; +import org.apache.hadoop.security.authorize.ProxyUsers; +import org.apache.phoenix.queryserver.server.QueryServer.PhoenixRemoteUserExtractor; +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.http.HttpServletRequest; + +/** + * Tests for the RemoteUserExtractor Method Avatica provides for Phoenix to implement. + */ +public class PhoenixRemoteUserExtractorTest { + private static final Logger LOG = LoggerFactory.getLogger(PhoenixRemoteUserExtractorTest.class); + + @Test + public void testUseDoAsSuccess() { +HttpServletRequest request = mock(HttpServletRequest.class); +when(request.getRemoteUser()).thenReturn("proxyserver"); +when(request.getParameter("doAs")).thenReturn("enduser"); +when(request.getRemoteAddr()).thenReturn("localhost:1234"); + +Configuration conf = new Configuration(false); +conf.set("hadoop.proxyuser.proxyserver.groups", "*"); +conf.set("hadoop.proxyuser.proxyserver.hosts", "*"); +conf.set("phoenix.queryserver.doAs.enabled", "true"); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); + +PhoenixRemoteUserExtractor extractor = new PhoenixRemoteUserExtractor(conf); +try { + assertEquals("enduser", extractor.extract(request)); +} catch (RemoteUserExtractionException e) { + LOG.info(e.getMessage()); +} + } + + @Test + public void testDoNotUseDoAs() { --- End diff -- Hi @joshelser, Is there a way to check if builder called withRemoteUserExtractor or not? I tried used "equals" but there will always be two new builder object to compare. Also there is no getRemoteUserExtractor method for HttpBuilder. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063731#comment-16063731 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124110879 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -274,6 +282,47 @@ public int run(String[] args) throws Exception { } /** + * Use the correctly way to extract end user. + */ + + static class PhoenixRemoteUserExtractor implements RemoteUserExtractor{ +private final HttpQueryStringParameterRemoteUserExtractor paramRemoteUserExtractor; +private final HttpRequestRemoteUserExtractor requestRemoteUserExtractor; +private final boolean enableDoAs; +private final String doAsParam; + +public PhoenixRemoteUserExtractor(Configuration conf) { + this.requestRemoteUserExtractor = new HttpRequestRemoteUserExtractor(); + this.doAsParam = conf.get(QueryServices.QUERY_SERVER_DOAS_PARAM, + QueryServicesOptions.DEFAULT_QUERY_SERVER_DOAS_PARAM); + this.paramRemoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(doAsParam); + this.enableDoAs = conf.getBoolean(QueryServices.QUERY_SERVER_DOAS_ENABLED_ATTRIB, + QueryServicesOptions.DEFAULT_QUERY_SERVER_DOAS_ENABLED); +} + +@Override +public String extract(HttpServletRequest request) throws RemoteUserExtractionException { + if (request.getParameter(doAsParam) != null && enableDoAs) { --- End diff -- This can be simplified when we remove the `enableDoAs` logic. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063727#comment-16063727 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124113153 --- Diff: phoenix-queryserver/pom.xml --- @@ -147,6 +147,10 @@ commons-logging commons-logging + + org.mockito + mockito-all --- End diff -- Needs a `test` > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063730#comment-16063730 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124112286 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -274,6 +282,47 @@ public int run(String[] args) throws Exception { } /** + * Use the correctly way to extract end user. + */ + + static class PhoenixRemoteUserExtractor implements RemoteUserExtractor{ +private final HttpQueryStringParameterRemoteUserExtractor paramRemoteUserExtractor; +private final HttpRequestRemoteUserExtractor requestRemoteUserExtractor; +private final boolean enableDoAs; +private final String doAsParam; + +public PhoenixRemoteUserExtractor(Configuration conf) { + this.requestRemoteUserExtractor = new HttpRequestRemoteUserExtractor(); + this.doAsParam = conf.get(QueryServices.QUERY_SERVER_DOAS_PARAM, + QueryServicesOptions.DEFAULT_QUERY_SERVER_DOAS_PARAM); + this.paramRemoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(doAsParam); + this.enableDoAs = conf.getBoolean(QueryServices.QUERY_SERVER_DOAS_ENABLED_ATTRIB, + QueryServicesOptions.DEFAULT_QUERY_SERVER_DOAS_ENABLED); +} + +@Override +public String extract(HttpServletRequest request) throws RemoteUserExtractionException { + if (request.getParameter(doAsParam) != null && enableDoAs) { +String doAsUser = paramRemoteUserExtractor.extract(request); +UserGroupInformation ugi = UserGroupInformation.createRemoteUser(request.getRemoteUser()); +UserGroupInformation proxyUser = UserGroupInformation.createProxyUser(doAsUser, ugi); + +// Check if this user is allowed to be impersonated. +// Will throw AuthorizationException if the impersonation as this user is not allowed +try { + ProxyUsers.authorize(proxyUser, request.getRemoteAddr()); + return doAsUser; +} catch (AuthorizationException e) { + throw new RemoteUserExtractionException(e.getMessage()); --- End diff -- Can the exception be passed into the RemoteUserExtractionException instead of just the message? (to preserve the stack trace) > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063729#comment-16063729 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124110721 --- Diff: phoenix-queryserver/src/main/java/org/apache/phoenix/queryserver/server/QueryServer.java --- @@ -274,6 +282,47 @@ public int run(String[] args) throws Exception { } /** + * Use the correctly way to extract end user. + */ + + static class PhoenixRemoteUserExtractor implements RemoteUserExtractor{ +private final HttpQueryStringParameterRemoteUserExtractor paramRemoteUserExtractor; +private final HttpRequestRemoteUserExtractor requestRemoteUserExtractor; +private final boolean enableDoAs; +private final String doAsParam; + +public PhoenixRemoteUserExtractor(Configuration conf) { + this.requestRemoteUserExtractor = new HttpRequestRemoteUserExtractor(); + this.doAsParam = conf.get(QueryServices.QUERY_SERVER_DOAS_PARAM, + QueryServicesOptions.DEFAULT_QUERY_SERVER_DOAS_PARAM); + this.paramRemoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(doAsParam); + this.enableDoAs = conf.getBoolean(QueryServices.QUERY_SERVER_DOAS_ENABLED_ATTRIB, --- End diff -- Can you move this check of whether or not we enable `doAs` above to selectively call `withRemoteUserExtractor`, please? > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063728#comment-16063728 ] ASF GitHub Bot commented on PHOENIX-3598: - Github user joshelser commented on a diff in the pull request: https://github.com/apache/phoenix/pull/265#discussion_r124113023 --- Diff: phoenix-queryserver/src/test/java/org/apache/phoenix/queryserver/server/PhoenixRemoteUserExtractorTest.java --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to you under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.queryserver.server; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import org.apache.calcite.avatica.server.RemoteUserExtractionException; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.security.UserGroupInformation; +import org.apache.hadoop.security.authorize.AuthorizationException; +import org.apache.hadoop.security.authorize.ProxyUsers; +import org.apache.phoenix.queryserver.server.QueryServer.PhoenixRemoteUserExtractor; +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.http.HttpServletRequest; + +/** + * Tests for the RemoteUserExtractor Method Avatica provides for Phoenix to implement. + */ +public class PhoenixRemoteUserExtractorTest { + private static final Logger LOG = LoggerFactory.getLogger(PhoenixRemoteUserExtractorTest.class); + + @Test + public void testUseDoAsSuccess() { +HttpServletRequest request = mock(HttpServletRequest.class); +when(request.getRemoteUser()).thenReturn("proxyserver"); +when(request.getParameter("doAs")).thenReturn("enduser"); +when(request.getRemoteAddr()).thenReturn("localhost:1234"); + +Configuration conf = new Configuration(false); +conf.set("hadoop.proxyuser.proxyserver.groups", "*"); +conf.set("hadoop.proxyuser.proxyserver.hosts", "*"); +conf.set("phoenix.queryserver.doAs.enabled", "true"); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); + +PhoenixRemoteUserExtractor extractor = new PhoenixRemoteUserExtractor(conf); +try { + assertEquals("enduser", extractor.extract(request)); +} catch (RemoteUserExtractionException e) { + LOG.info(e.getMessage()); +} + } + + @Test + public void testDoNotUseDoAs() { --- End diff -- To test this code if you take my above suggestion, you could make a new method in QueryServer which does ```java Builder setRemoteUserExtractorIfNecessary(Builder b, Configuration conf) { if (conf.getBoolean(QueryServices.QUERY_SERVER_DOAS_ENABLED_ATTRIB, QueryServicesOptions.DEFAULT_QUERY_SERVER_DOAS_ENABLED)) { return builder.withRemoteUserExtractor(new PhoenixRemoteUserExtractor(getConf())); } return builder; } ``` This would let you easily mock the Builder and verify that your extractor is configured when the property is set to "true". > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062578#comment-16062578 ] ASF GitHub Bot commented on PHOENIX-3598: - GitHub user Wancy opened a pull request: https://github.com/apache/phoenix/pull/265 PHOENIX-3598 Add two params "phoenix.queryserver.doAs.enabled" and "phoenix.queryserver.doAs.param" to control whether to get enduser from request parameters and what is the parameter key name. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Wancy/phoenix master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/265.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #265 commit 60b97e1475eedcc8452ba5953d53431988ac9e45 Author: shiwangDate: 2017-06-26T06:27:31Z PHOENIX-3598 > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061311#comment-16061311 ] Shi Wang commented on PHOENIX-3598: --- [~elserj], thanks I'll put a new path soon. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061155#comment-16061155 ] Josh Elser commented on PHOENIX-3598: - [~Wancy], would you be able to put up a new patch now that we have the changes you made in Avatica downstream in Phoenix, please? > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934029#comment-15934029 ] Josh Elser commented on PHOENIX-3598: - {code} +public String extractRemoteUser(HttpServletRequest request) throws Exception { + if (request.getParameter("doAs") != null) { +String doAsUser = request.getParameter("doAs"); +UserGroupInformation proxyUser = UserGroupInformation.createProxyUser(doAsUser, serverUgi); + +// Check if this user is allowed to be impersonated. +// Will throw AuthorizationException if the impersonation as this user is not allowed +ProxyUsers.authorize(proxyUser, request.getRemoteAddr();); +this.remoteUserExtractor = new HttpQueryStringParameterRemoteUserExtractor(); {code} This needs to be done via explicit configuration. Otherwise, it's introducing a security hole. {code} + } else { +this.remoteUserExtractor = new HttpRequestRemoteUserExtractor(); + } {code} This is creating a new object unnecessarily for every request to PQS which is bad. Just create a single instance in the constructor. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933770#comment-15933770 ] Shi Wang commented on PHOENIX-3598: --- Hi [~elserj], Could you also take a look at this patch? It has dependency on CALCITE-1593 so cannot compile for now, but would like to have your opinion on the implementation of CALCITE1593, thanks! > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He >Assignee: Shi Wang > Attachments: 0001-PHOENIX-3598.patch > > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3598) Enable proxy access to Phoenix query server for third party on behalf of end users
[ https://issues.apache.org/jira/browse/PHOENIX-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15822292#comment-15822292 ] Shi Wang commented on PHOENIX-3598: --- Hi Jerry, I would like to contribute to this jira. > Enable proxy access to Phoenix query server for third party on behalf of end > users > -- > > Key: PHOENIX-3598 > URL: https://issues.apache.org/jira/browse/PHOENIX-3598 > Project: Phoenix > Issue Type: Improvement >Reporter: Jerry He > > This JIRA tracks the follow-on work of CALCITE-1539 needed on Phoenix query > server side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)