[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS HMS tables. The new column is used to differentiate among column stats computed by different engines. The related HMS API calls were changed accordingly. This change is Step 2 in a series of steps to coordinate the introduction of HMS API changes to Hive3 and Impala. Step 4 is also Impala related, it will be covered in IMPALA-8842 part 2. The steps are as follows: 1. Change in Hive3. We push new APIs so Impala can use them. New APIs will simply call old existing methods so there should not be any change of functionality there. Since there were many incompatible changes, new APIs are tagged method_name_V2. 2. Change in Impala. Push changes to use new methods *V2. 3. Change in Hive3. Push patch with complete functionality. *V2 methods contains the new logic. The old existing methods are not used anymore by Impala at this point, hence they can be removed. For every method_name_V2, I will create a corresponding method method_name that calls the former one. 4. Change in Impala Replace *V2 calls by *. 5. Change in Hive3. Remove *V2 methods from API. This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml to work around the Hive3 Hadoop2 dependency. TESTING: Only Step 1 has been merged in into Hive3 at this point, which means that there's no change in Impala and Hive3 functionality yet. Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Reviewed-on: http://gerrit.cloudera.org:8080/14043 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/impala-config.sh M fe/pom.xml M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M shaded-deps/pom.xml 9 files changed, 106 insertions(+), 17 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 19:51:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14 PS7, Line 14: This change is Step 2 in a series of steps to coordinate the : introduction of HMS API changes to Hive3 and Impala. Step 4 is also : Impala related, it will be covered in IMPALA-8842 part 2. : : The steps are as follows: : : 1. Change in Hive3. : We push new APIs so Impala can use them. New APIs will simply call old : existing methods so there should not be any change of functionality : there. Since there were many incompatible changes, new APIs are tagged : method_name_V2. : : 2. Change in Impala. : Push changes to use new methods *V2. : : 3. Change in Hive3. : Push patch with complete functionality. *V2 methods contains the new : logic. The old existing methods are not used anymore by Impala at this : point, hence they can be removed. For every method_name_V2, I will : create a corresponding method method_name that calls the former one. : : 4. Change in Impala : Replace *V2 calls by *. : : 5. Change in Hive3. : Remove *V2 methods from API > I also wanted to explain why there's no change in functionality yet. I am okay with this. This should not block the commit. -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 18:24:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14 PS7, Line 14: This change is Step 2 in a series of steps to coordinate the : introduction of HMS API changes to Hive3 and Impala. Step 4 is also : Impala related, it will be covered in IMPALA-8842 part 2. : : The steps are as follows: : : 1. Change in Hive3. : We push new APIs so Impala can use them. New APIs will simply call old : existing methods so there should not be any change of functionality : there. Since there were many incompatible changes, new APIs are tagged : method_name_V2. : : 2. Change in Impala. : Push changes to use new methods *V2. : : 3. Change in Hive3. : Push patch with complete functionality. *V2 methods contains the new : logic. The old existing methods are not used anymore by Impala at this : point, hence they can be removed. For every method_name_V2, I will : create a corresponding method method_name that calls the former one. : : 4. Change in Impala : Replace *V2 calls by *. : : 5. Change in Hive3. : Remove *V2 methods from API > I recorded the steps in the commit msg because I wanted to explain why we n I also wanted to explain why there's no change in functionality yet. -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 18:01:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14 PS7, Line 14: This change is Step 2 in a series of steps to coordinate the : introduction of HMS API changes to Hive3 and Impala. Step 4 is also : Impala related, it will be covered in IMPALA-8842 part 2. : : The steps are as follows: : : 1. Change in Hive3. : We push new APIs so Impala can use them. New APIs will simply call old : existing methods so there should not be any change of functionality : there. Since there were many incompatible changes, new APIs are tagged : method_name_V2. : : 2. Change in Impala. : Push changes to use new methods *V2. : : 3. Change in Hive3. : Push patch with complete functionality. *V2 methods contains the new : logic. The old existing methods are not used anymore by Impala at this : point, hence they can be removed. For every method_name_V2, I will : create a corresponding method method_name that calls the former one. : : 4. Change in Impala : Replace *V2 calls by *. : : 5. Change in Hive3. : Remove *V2 methods from API > I recorded the steps in the commit msg because I wanted to explain why we n Please let me know if you are okay with this commit msg, as II am planning to merge this in in the next couple of hours. -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 17:58:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14 PS7, Line 14: This change is Step 2 in a series of steps to coordinate the : introduction of HMS API changes to Hive3 and Impala. Step 4 is also : Impala related, it will be covered in IMPALA-8842 part 2. : : The steps are as follows: : : 1. Change in Hive3. : We push new APIs so Impala can use them. New APIs will simply call old : existing methods so there should not be any change of functionality : there. Since there were many incompatible changes, new APIs are tagged : method_name_V2. : : 2. Change in Impala. : Push changes to use new methods *V2. : : 3. Change in Hive3. : Push patch with complete functionality. *V2 methods contains the new : logic. The old existing methods are not used anymore by Impala at this : point, hence they can be removed. For every method_name_V2, I will : create a corresponding method method_name that calls the former one. : : 4. Change in Impala : Replace *V2 calls by *. : : 5. Change in Hive3. : Remove *V2 methods from API > I am not sure why we need this in the commit message. The commit message sh I recorded the steps in the commit msg because I wanted to explain why we need to break IMPALA-8842 down to part 1 and part2. http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml File shaded-deps/pom.xml: http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml@43 PS7, Line 43: : : org.apache.hive.shims : hive-shims-0.20 : : > Is there a particular reason why this needs to be excluded for this patch? This is mentioned briefly in the Commit Message. hive-shims-0.20 brings in Hadoop2 dependencies which will break the Impala build. This is a workaround to avoid build failures until we find a way to stop Hadoop2 dependencies from leaking out. -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 17:45:40 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14 PS7, Line 14: This change is Step 2 in a series of steps to coordinate the : introduction of HMS API changes to Hive3 and Impala. Step 4 is also : Impala related, it will be covered in IMPALA-8842 part 2. : : The steps are as follows: : : 1. Change in Hive3. : We push new APIs so Impala can use them. New APIs will simply call old : existing methods so there should not be any change of functionality : there. Since there were many incompatible changes, new APIs are tagged : method_name_V2. : : 2. Change in Impala. : Push changes to use new methods *V2. : : 3. Change in Hive3. : Push patch with complete functionality. *V2 methods contains the new : logic. The old existing methods are not used anymore by Impala at this : point, hence they can be removed. For every method_name_V2, I will : create a corresponding method method_name that calls the former one. : : 4. Change in Impala : Replace *V2 calls by *. : : 5. Change in Hive3. : Remove *V2 methods from API I am not sure why we need this in the commit message. The commit message should give details what the patch does and why. The coordination of patches between Hive and Impala can be done out-of-band and provides no value by recording the steps in the git commit message. http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml File shaded-deps/pom.xml: http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml@43 PS7, Line 43: : : org.apache.hive.shims : hive-shims-0.20 : : Is there a particular reason why this needs to be excluded for this patch? -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 15 Aug 2019 17:20:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4262/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 6 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Aug 2019 15:57:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Aug 2019 15:42:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4798/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Aug 2019 15:42:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 6: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@7 PS4, Line 7: IMPALA-8842 part 1: (Hive3) Use 'engine' field in H > Done Thanks, I understand it now, I think that it is ok this way. -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 6 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Aug 2019 15:32:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Attila Jeges has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS HMS tables. The new column is used to differentiate among column stats computed by different engines. The related HMS API calls were changed accordingly. This change is Step 2 in a series of steps to coordinate the introduction of HMS API changes to Hive3 and Impala. Step 4 is also Impala related, it will be covered in IMPALA-8842 part 2. The steps are as follows: 1. Change in Hive3. We push new APIs so Impala can use them. New APIs will simply call old existing methods so there should not be any change of functionality there. Since there were many incompatible changes, new APIs are tagged method_name_V2. 2. Change in Impala. Push changes to use new methods *V2. 3. Change in Hive3. Push patch with complete functionality. *V2 methods contains the new logic. The old existing methods are not used anymore by Impala at this point, hence they can be removed. For every method_name_V2, I will create a corresponding method method_name that calls the former one. 4. Change in Impala Replace *V2 calls by *. 5. Change in Hive3. Remove *V2 methods from API. This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml to work around the Hive3 Hadoop2 dependency. TESTING: Only Step 1 has been merged in into Hive3 at this point, which means that there's no change in Impala and Hive3 functionality yet. Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 --- M bin/impala-config.sh M fe/pom.xml M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M shaded-deps/pom.xml 9 files changed, 106 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/6 -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 6 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4260/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 5 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Aug 2019 15:11:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Attila Jeges has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/14043 ) Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API .. IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS HMS tables. The new column is used to differentiate among column stats computed by different engines. The related HMS API calls were changed accordingly. This change is Step 2 in a series of steps to coordinate the introduction of HMS API changes to Hive3 and Impala: 1. Change in Hive3. We push new APIs so Impala can use them. New APIs will simply call old existing methods so there should not be any change of functionality there. Since there were many incompatible changes, new APIs are tagged method_name_V2. 2. Change in Impala. Push changes to use new methods *V2. 3. Change in Hive3. Push patch with complete functionality. *V2 methods contains the new logic. The old existing methods are not used anymore by Impala at this point, hence they can be removed. For every method_name_V2, I will create a corresponding method method_name that calls the former one. 4. Change in Impala Replace *V2 calls by *. 5. Change in Hive3. Remove *V2 methods from API. This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml to work around the Hive3 Hadoop2 dependency. TESTING: Only Step 1 has been merged in into Hive3 at this point, which means that there's no change in Impala and Hive3 functionality yet. Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 --- M bin/impala-config.sh M fe/pom.xml M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M shaded-deps/pom.xml 9 files changed, 106 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/5 -- To view, visit http://gerrit.cloudera.org:8080/14043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4 Gerrit-Change-Number: 14043 Gerrit-PatchSet: 5 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
