Fang-Yu Rao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22236
Change subject: IMPALA-13011: [WIP] Support authorization for Calcite in Impala ...................................................................... IMPALA-13011: [WIP] Support authorization for Calcite in Impala This patch supports the authorization of the select queries for the Calcite planner. In this patch, we focus on the following 2 types of resources. - For table-level privilege requests, we register the privilege request for a table whether it exists or not. - For column-level privilege requests, we register the privilege request for a column only if it exists. Implementation We register the table-level privilege requests in CalciteMetadataHandler#registerTablePrivReqs() when processing each table used in the select query. To register column-level privilege requests, we identified the methods in SqlValidatorImpl to override in order to register the column-level privilege requests in the select queries. Specifically, to derive the namespace of a column involved in a where clause or a select list that does not require expression expansion, we override SqlValidatorImpl#expand() to call a custom-made Expander deriving from SqlScopedShuttle so that we are able to derive the namespace of the column in visit() of the custom-made Expander. To derive the namespace of a column that is an item on the select list from the expansion of an original select list item, i.e., STAR(*), we override SqlValidatorImpl#validateIdentifier() to derive the namespace. Testing: - Briefly verified that when authorization is enabled, the instantiated AuthorizationChecker is able to authorize each registered privilege request in the Analyzer of CalciteMetadataHandler and that the respective Ranger audit events are stored in the RangerBufferAuditHandler of the instantiated AuthorizationContext. Note: - Because we call Analyzer#analyze() on tables and columns during the creation of the physical plan, more privilege requests than needed are registered. If we decide to register the privilege requests after or during the creation of the physical plan, we will need to disable the registration in this case, or need to find another place to store the registered privilege requests when Calcite is the planner. - Different column-level privilege requests would be registered if not all column names could be successfully resolved by Calcite's validator. This is because in a select query, Impala's classic frontend analyzes the select clause before the where clause, whereas it is the other way around when Calcite is the planner. For instance, in the query "SELECT id FROM functional.alltypestiny WHERE non_existent_col = false", Impala's classic frontend registers a privilege request for the column 'id' but this patch does not. To-do's: - We need to refactor the code related to authorization by creating a class corresponding to Frontend in the classic frontend (e.g., CalciteFrontend) and moving some code to CalciteFrontend. - We need to call flush() on the RangerBufferAuditHandler of the instantiated AuthorizationContext so that the Ranger audit events could be sent to the remote logging service. Change-Id: I5ad9faee847c7dce61d4db87b1fda4ff9e1aae0c --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalcitePhysPlanCreator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ExecRequestCreator.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaPrivilegeRequestsRegistrar.java M testdata/workloads/functional-query/queries/QueryTest/calcite.test 8 files changed, 431 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/22236/1 -- To view, visit http://gerrit.cloudera.org:8080/22236 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5ad9faee847c7dce61d4db87b1fda4ff9e1aae0c Gerrit-Change-Number: 22236 Gerrit-PatchSet: 1 Gerrit-Owner: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]>
