[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Reviewed-on: http://gerrit.cloudera.org:8080/17092 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java 14 files changed, 82 insertions(+), 25 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 13 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 12: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 12 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Tue, 02 Mar 2021 05:57:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 12: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 12 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Tue, 02 Mar 2021 00:11:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 12: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6929/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 12 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Tue, 02 Mar 2021 00:11:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 11 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Tue, 02 Mar 2021 00:10:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8271/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 11 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Mon, 01 Mar 2021 22:21:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Steve Carlin has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 --- M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java 14 files changed, 82 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/11 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 11 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 10: (2 comments) The change in HdfsTable may introduce an issue. Other changes look good to me. http://gerrit.cloudera.org:8080/#/c/17092/10/be/src/rpc/hs2-http-test.cc File be/src/rpc/hs2-http-test.cc: http://gerrit.cloudera.org:8080/#/c/17092/10/be/src/rpc/hs2-http-test.cc@51 PS10, Line 51: virtual void GetBackendConfig(TGetBackendConfigResp& return_val, Is this used somewhere? http://gerrit.cloudera.org:8080/#/c/17092/10/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/17092/10/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1719 PS10, Line 1719: if (isSchemaLoaded_) return; I don't think we can do this. This means the table schema will only be loaded once. However, when executiing REFRESH or some DDL commands, we need to reload these to update the schema. Here's an example: # Create a table in Impala and run any query on it to make its metadata loaded impala> create table my_tbl (id int, name string); impala> desc my_tbl; # Change its schema in Hive beeline> alter table my_tbl add columns (address string); # Refresh in Impala. Impala should reload its metadata and get the latest schema. impala> refresh my_tbl; impala> desc my_tbl; -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 10 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Mon, 01 Mar 2021 13:29:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Steve Carlin has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 --- M be/src/rpc/hs2-http-test.cc M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java 16 files changed, 85 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/10 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 10 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 7: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/8256/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 7 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Fri, 26 Feb 2021 19:49:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/17092/7/fe/src/main/java/org/apache/impala/analysis/StatementBase.java File fe/src/main/java/org/apache/impala/analysis/StatementBase.java: http://gerrit.cloudera.org:8080/#/c/17092/7/fe/src/main/java/org/apache/impala/analysis/StatementBase.java@203 PS7, Line 203: public static Expr checkTypeCompatibility(String dstTableName, Column dstCol, Expr srcExpr, line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 7 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Fri, 26 Feb 2021 19:38:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Steve Carlin has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 --- M be/src/rpc/hs2-http-test.cc M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java 11 files changed, 78 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/7 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 7 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 6: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/8254/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 6 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Fri, 26 Feb 2021 18:28:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/17092/6/fe/src/main/java/org/apache/impala/analysis/StatementBase.java File fe/src/main/java/org/apache/impala/analysis/StatementBase.java: http://gerrit.cloudera.org:8080/#/c/17092/6/fe/src/main/java/org/apache/impala/analysis/StatementBase.java@203 PS6, Line 203: public static Expr checkTypeCompatibility(String dstTableName, Column dstCol, Expr srcExpr, line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 6 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Fri, 26 Feb 2021 18:18:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Steve Carlin has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 --- M be/src/rpc/hs2-http-test.cc M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java 10 files changed, 77 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/6 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 6 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8200/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 5 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Tue, 23 Feb 2021 03:01:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Steve Carlin has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 --- M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java 6 files changed, 51 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/5 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 5 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8179/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 4 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Sat, 20 Feb 2021 17:49:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java: http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@699 PS3, Line 699: this(table, partitionIdCounter_.getAndIncrement(), prevId, partName, partitionKeyValues, > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@701 PS3, Line 701: encodedDeleteFileDescriptors, location, isMarkedCached, accessLevel, hmsParameters, > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@702 PS3, Line 702: cachedMsPartitionDescriptor, partitionStats, hasIncrementalStats, numRows, writeId, > line too long (91 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 3 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Steve Carlin Gerrit-Comment-Date: Sat, 20 Feb 2021 17:23:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Steve Carlin has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 --- M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java 5 files changed, 50 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/4 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 4 Gerrit-Owner: Steve Carlin Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8177/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 3 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 20 Feb 2021 15:38:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 ) Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java: http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@699 PS3, Line 699: this(table, partitionIdCounter_.getAndIncrement(), prevId, partName, partitionKeyValues, line too long (92 > 90) http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@701 PS3, Line 701: encodedDeleteFileDescriptors, location, isMarkedCached, accessLevel, hmsParameters, line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@702 PS3, Line 702: cachedMsPartitionDescriptor, partitionStats, hasIncrementalStats, numRows, writeId, line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 3 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 20 Feb 2021 15:17:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
scar...@cloudera.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17092 Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions. .. IMPALA-10524: Changes to HdfsPartition for third party extensions. Some changes are needed to HdfsPartition and other related classes to allow for third party extensions. These changes include: - A protected constructor which will allow a subclass to instantiate HdfsPartition using its own Builder. - Various changes of permissions to methods and variables to allow third party extension visibility. - Creation of the getHostIndex() method to allow the subclass to override how the hostIndexes are retrieved. - Added a new default method "getFileSystem()" to FeFsPartition which will allow the third party extension to override how the filesystem is obtained from the partition object. Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 --- M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java 5 files changed, 50 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/3 -- To view, visit http://gerrit.cloudera.org:8080/17092 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46 Gerrit-Change-Number: 17092 Gerrit-PatchSet: 3 Gerrit-Owner: Anonymous Coward
