[Impala-ASF-CR] IMPALA-5256: Force log rotation when max log size exceeded
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17997 ) Change subject: IMPALA-5256: Force log rotation when max_log_size exceeded .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7608/ -- To view, visit http://gerrit.cloudera.org:8080/17997 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 Gerrit-Change-Number: 17997 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 09 Nov 2021 07:28:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 10 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 09 Nov 2021 07:19:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7610/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 10 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 09 Nov 2021 07:19:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 9: > Patch Set 9: > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7607/ This time, we were hit by different flaky test at test_spilling.py. The failed test was introduced by IMPALA-9725. The test require for the hash join to spill its partitions, but it does not happen in https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/4903 Following recent hash table memory improvement (IMPALA-7635), we might need to lower the buffer_pool_limit for that test so that it spill consistently. -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 09 Nov 2021 03:59:39 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 ) Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9739/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43 Gerrit-Change-Number: 17948 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 09 Nov 2021 03:17:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10943: Add test to verify support for multiple resource and executor pools
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17891 ) Change subject: IMPALA-10943: Add test to verify support for multiple resource and executor pools .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7606/ -- To view, visit http://gerrit.cloudera.org:8080/17891 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If76d386d8de5730da937674ddd9a69aa1aa1355e Gerrit-Change-Number: 17891 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 09 Nov 2021 03:05:21 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
Wenzhe Zhou has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17948 ) Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala .. WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala There are some API changes in the new version of protobuf library. This patch makes necessary changes for Impala code to pass compiling. Upgrade Kudu to 5d39a74a19. With the new version of Kudu, the protobuf symbols in libkudu_client.so are hided. This solved the static initialization conflicts between two copies of protobuf library in impalad's binary - one is linked by Impala itself for KRPC communication, another is linked into Kudu client shared library. Testing: - Passed core DEBUG build and exhaustive release build. - TODO pass a core ASAN build Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43 --- M be/src/gutil/dynamic_annotations.h M be/src/kudu/rpc/exactly_once_rpc-test.cc M be/src/kudu/rpc/inbound_call.cc M be/src/kudu/rpc/outbound_call.cc M be/src/kudu/rpc/result_tracker.h M be/src/kudu/rpc/serialization.cc M be/src/kudu/rpc/serialization.h M be/src/kudu/util/pb_util.cc M be/src/kudu/util/protobuf_util.h M bin/impala-config.sh M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl 11 files changed, 74 insertions(+), 50 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/5 -- To view, visit http://gerrit.cloudera.org:8080/17948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43 Gerrit-Change-Number: 17948 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 9: Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7607/ -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 09 Nov 2021 02:50:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. Patch Set 14: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 14 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Tue, 09 Nov 2021 02:39:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10940: Pick parts of recent gutil changes from Kudu repo
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17897 ) Change subject: IMPALA-10940: Pick parts of recent gutil changes from Kudu repo .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 Gerrit-Change-Number: 17897 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 09 Nov 2021 01:46:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10940: Pick parts of recent gutil changes from Kudu repo
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17897 ) Change subject: IMPALA-10940: Pick parts of recent gutil changes from Kudu repo .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7609/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 Gerrit-Change-Number: 17897 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 09 Nov 2021 01:46:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10940: Pick parts of recent gutil changes from Kudu repo
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17897 ) Change subject: IMPALA-10940: Pick parts of recent gutil changes from Kudu repo .. Patch Set 3: I pick the files with the changes that are required for rebase of be/src/kudu, and the their dependent files. -- To view, visit http://gerrit.cloudera.org:8080/17897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 Gerrit-Change-Number: 17897 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 09 Nov 2021 00:37:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5256: Force log rotation when max log size exceeded
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17997 ) Change subject: IMPALA-5256: Force log rotation when max_log_size exceeded .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7608/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17997 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 Gerrit-Change-Number: 17997 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 09 Nov 2021 00:36:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10926: Improve catalogd consistency and self events detection
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17859 ) Change subject: IMPALA-10926: Improve catalogd consistency and self events detection .. Patch Set 27: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7603/ -- To view, visit http://gerrit.cloudera.org:8080/17859 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36364e401911352c474eb98c8d61bbaae9b9 Gerrit-Change-Number: 17859 Gerrit-PatchSet: 27 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Tue, 09 Nov 2021 00:36:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11007: Replies to HTTP HEAD requests should contain no content.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18002 ) Change subject: IMPALA-11007: Replies to HTTP HEAD requests should contain no content. .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18002 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc3bf72067dfa5bc24e29fe974e5762be507952 Gerrit-Change-Number: 18002 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 23:59:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11007: Replies to HTTP HEAD requests should contain no content.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18002 ) Change subject: IMPALA-11007: Replies to HTTP HEAD requests should contain no content. .. IMPALA-11007: Replies to HTTP HEAD requests should contain no content. If a HEAD request to Impala's webserver, for example curl -I http://localhost:25000/metrics does send content in the response then this results in ugly messages being logged on the server side when the client side closes the connection after it has read the headers in the http response. Fix this by not sending (or generating) the content in reply to the HEAD request. TESTING: Ran all end-to-end tests. Added HEAD calls to various existing tests. Change-Id: I5cc3bf72067dfa5bc24e29fe974e5762be507952 Reviewed-on: http://gerrit.cloudera.org:8080/18002 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/util/webserver.cc M tests/webserver/test_web_pages.py 2 files changed, 20 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18002 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5cc3bf72067dfa5bc24e29fe974e5762be507952 Gerrit-Change-Number: 18002 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10940: Pick parts of recent gutil changes from Kudu repo
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17897 ) Change subject: IMPALA-10940: Pick parts of recent gutil changes from Kudu repo .. Patch Set 3: Code-Review+2 These make sense. After applying these, is our gutil essentially the same as Kudu's for the files we share? -- To view, visit http://gerrit.cloudera.org:8080/17897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 Gerrit-Change-Number: 17897 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 08 Nov 2021 23:45:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10940: Pick parts of recent gutil changes from Kudu repo
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17897 ) Change subject: IMPALA-10940: Pick parts of recent gutil changes from Kudu repo .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9738/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 Gerrit-Change-Number: 17897 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 08 Nov 2021 22:35:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5256: Force log rotation when max log size exceeded
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17997 ) Change subject: IMPALA-5256: Force log rotation when max_log_size exceeded .. Patch Set 2: Code-Review+2 Thanks, this looks good to me -- To view, visit http://gerrit.cloudera.org:8080/17997 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 Gerrit-Change-Number: 17997 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 22:24:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10940: Pick parts of recent gutil changes from Kudu repo
Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17897 ) Change subject: IMPALA-10940: Pick parts of recent gutil changes from Kudu repo .. IMPALA-10940: Pick parts of recent gutil changes from Kudu repo There are various fixes and improvements in gutil that have been made on the Kudu repo. These changes are required for rebase of be/src/kudu. Testing: - Passed core debug build and exhaustive release build. - Passed core ASAN build. Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 --- M be/src/gutil/cpu.cc M be/src/gutil/cpu.h M be/src/gutil/integral_types.h M be/src/gutil/macros.h M be/src/gutil/map-util.h M be/src/gutil/port.h M be/src/gutil/stringprintf.cc M be/src/gutil/stringprintf.h M be/src/gutil/strings/numbers.cc M be/src/gutil/strings/numbers.h M be/src/gutil/strtoint.cc M be/src/gutil/strtoint.h 12 files changed, 407 insertions(+), 188 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/17897/3 -- To view, visit http://gerrit.cloudera.org:8080/17897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 Gerrit-Change-Number: 17897 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-10940: Pick parts of recent gutil changes from Kudu repo
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17897 ) Change subject: IMPALA-10940: Pick parts of recent gutil changes from Kudu repo .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/map-util.h File be/src/gutil/map-util.h: http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/map-util.h@975 PS2, Line 975: return ComputePairIfAbsentReturnAbsense(container, key, compute_pair_func).first; > line too long (107 > 90) fixed http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/port.h File be/src/gutil/port.h: http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/port.h@1179 PS2, Line 1179: // Enable UnalignedLoad and UnalignedStore for numeric types (floats and ints) including int128. > line too long (96 > 90) fixed http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/stringprintf.h File be/src/gutil/stringprintf.h: http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/stringprintf.h@44 PS2, Line 44: extern std::string StringPrintfVector(const char* format, const std::vector& v); > line too long (93 > 90) fixed http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/strings/numbers.cc File be/src/gutil/strings/numbers.cc: http://gerrit.cloudera.org:8080/#/c/17897/2/be/src/gutil/strings/numbers.cc@1076 PS2, Line 1076: static_cast(100) * static_cast(100); > line too long (96 > 90) fixed -- To view, visit http://gerrit.cloudera.org:8080/17897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9230923111af3c1028f0e71cd37cb5ee6fbec654 Gerrit-Change-Number: 17897 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 08 Nov 2021 22:12:10 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17996 ) Change subject: IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9737/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a Gerrit-Change-Number: 17996 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 08 Nov 2021 21:57:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17996 ) Change subject: IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9736/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a Gerrit-Change-Number: 17996 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 08 Nov 2021 21:49:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala
Wenzhe Zhou has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17996 ) Change subject: IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala .. IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala There are some header files deprecated in the new version of Boost library. Need to define BOOST_ALLOW_DEPRECATED_HEADERS in CMakeLists.txt to avoid compiling errors. Also define BOOST_BIND_GLOBAL_PLACEHOLDERS to keep current behaviour of boost::bind and avoid big code changes. Define exception handler for a new boost::throw_exception() API since BOOST_NO_EXCEPTIONS is defined in be/CMakeLists.txt and we have to provide handlers which will be called by codegen'd code. Revert the code change made by IMPALA-2846 and IMPALA-9571 since the bug was fixed in Boost 1.74.0. Testing: - Passed core DEBUG build and exhaustive release build. - Passed core ASAN build. Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a --- M be/CMakeLists.txt M be/src/benchmarks/convert-timestamp-benchmark.cc M be/src/codegen/llvm-codegen.cc M be/src/runtime/bufferpool/buffer-pool-test.cc M be/src/runtime/timestamp-value.cc M be/src/statestore/failure-detector.h M be/src/util/blocking-queue-test.cc M be/src/util/condition-variable.h M be/src/util/filesystem-util.cc M bin/impala-config.sh 10 files changed, 28 insertions(+), 37 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/17996/5 -- To view, visit http://gerrit.cloudera.org:8080/17996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a Gerrit-Change-Number: 17996 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala
Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17996 ) Change subject: WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala .. WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala There are some header files deprecated in the new version of Boost library. Need to define BOOST_ALLOW_DEPRECATED_HEADERS in CMakeLists.txt to avoid compiling errors. Also define BOOST_BIND_GLOBAL_PLACEHOLDERS to keep current behaviour of boost::bind and avoid big code changes. Define exception handler for a new boost::throw_exception() API since BOOST_NO_EXCEPTIONS is defined in be/CMakeLists.txt and we have to provide handlers which will be called by codegen'd code. Revert the code change made by IMPALA-2846 and IMPALA-9571 since the bug was fixed in Boost 1.74.0. Testing: - Passed core DEBUG build and exhaustive release build. - Passed core ASAN build. Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a --- M be/CMakeLists.txt M be/src/benchmarks/convert-timestamp-benchmark.cc M be/src/codegen/llvm-codegen.cc M be/src/runtime/bufferpool/buffer-pool-test.cc M be/src/runtime/timestamp-value.cc M be/src/statestore/failure-detector.h M be/src/util/blocking-queue-test.cc M be/src/util/condition-variable.h M be/src/util/filesystem-util.cc M bin/impala-config.sh 10 files changed, 31 insertions(+), 33 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/17996/4 -- To view, visit http://gerrit.cloudera.org:8080/17996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a Gerrit-Change-Number: 17996 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17996 ) Change subject: WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17996/4/be/src/runtime/timestamp-value.cc File be/src/runtime/timestamp-value.cc: http://gerrit.cloudera.org:8080/#/c/17996/4/be/src/runtime/timestamp-value.cc@40 PS4, Line 40: // Boost stores dates as an uint32_t. Since subtraction is needed, convert to signed. : #pragma clang diagnostic push : #pragma clang diagnostic ignored "-Wunused-const-variable" : const int64_t EPOCH_DAY_NUMBER = : static_cast(date(1970, boost::gregorian::Jan, 1).day_number()); : #pragma clang diagnostic pop > If this is unused, is there any reason we can't just remove it? Will remove it. -- To view, visit http://gerrit.cloudera.org:8080/17996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a Gerrit-Change-Number: 17996 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 08 Nov 2021 21:26:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5256: Force log rotation when max log size exceeded
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17997 ) Change subject: IMPALA-5256: Force log rotation when max_log_size exceeded .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9735/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17997 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 Gerrit-Change-Number: 17997 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 21:14:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5256: Force log rotation when max log size exceeded
Riza Suminto has posted comments on this change. (
http://gerrit.cloudera.org:8080/17997 )
Change subject: IMPALA-5256: Force log rotation when max_log_size exceeded
..
Patch Set 2:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/17997/1/be/src/common/logging.cc
File be/src/common/logging.cc:
http://gerrit.cloudera.org:8080/#/c/17997/1/be/src/common/logging.cc@156
PS1, Line 156: FLAGS_stderrthreshold = google::FATAL + 1;
:
: if (RedirectStdoutStderr()) {
: // We will be redirecting stdout/stderr to INFO/LOG so
override any glog settings
: // that log to stdout/stderr...
: FLAGS_logtostderr = false;
:
> Nit: One question is who is responsible for handling errors and who can mos
Done.
Also remove the declaration from logging.h to avoid the need to include
"common/status.h" there.
http://gerrit.cloudera.org:8080/#/c/17997/1/be/src/common/logging.cc@164
PS1, Line 164:
> Nit: I go back and forth about whether this should return Status or not. It
Done
http://gerrit.cloudera.org:8080/#/c/17997/1/be/src/common/logging.cc@202
PS1, Line 202: lock_guard logging_lock(logging_mut
> Nit: There are two use cases for this function. One is checking the file si
Done
http://gerrit.cloudera.org:8080/#/c/17997/1/be/src/common/logging.cc@211
PS1, Line 211: int log_to_check[2] = {google::INFO, google::ERROR};
> Can you add a comment saying that max_log_size is measured in megabytes (an
Done
http://gerrit.cloudera.org:8080/#/c/17997/1/be/src/util/filesystem-util.h
File be/src/util/filesystem-util.h:
http://gerrit.cloudera.org:8080/#/c/17997/1/be/src/util/filesystem-util.h@111
PS1, Line 111: /// Return the approximate file size of 'path' into output
argument 'file_size',
> Please add a comment for this function
Done
http://gerrit.cloudera.org:8080/#/c/17997/1/tests/custom_cluster/test_breakpad.py
File tests/custom_cluster/test_breakpad.py:
http://gerrit.cloudera.org:8080/#/c/17997/1/tests/custom_cluster/test_breakpad.py@416
PS1, Line 416: o
> flake8: E261 at least two spaces before inline comment
Done
http://gerrit.cloudera.org:8080/#/c/17997/1/tests/custom_cluster/test_breakpad.py@417
PS1, Line 417:
> flake8: E261 at least two spaces before inline comment
Done
http://gerrit.cloudera.org:8080/#/c/17997/1/tests/custom_cluster/test_breakpad.py@427
PS1, Line 427:test_max_log_files = 2
: test_max_log_size = 1 # 1 MB
: test_error_msg = ('123456789abcde_' * 64) # 1 KB error
message
: test_debug_actions = 'LOG_MAINTENANCE_STDERR:FAIL@1.0@' +
test_error_msg
: os.chmod(self.tmp_dir, 0744)
> Two thoughts:
Done.
Increased the wait time to 40s and sanity check the log size and count every
second.
http://gerrit.cloudera.org:8080/#/c/17997/1/tests/custom_cluster/test_breakpad.py@431
PS1, Line 431:
> flake8: W292 no newline at end of file
Done
--
To view, visit http://gerrit.cloudera.org:8080/17997
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3
Gerrit-Change-Number: 17997
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Riza Suminto
Gerrit-Comment-Date: Mon, 08 Nov 2021 20:53:42 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5256: Force log rotation when max log size exceeded
Hello Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17997 to look at the new patch set (#2). Change subject: IMPALA-5256: Force log rotation when max_log_size exceeded .. IMPALA-5256: Force log rotation when max_log_size exceeded Impala daemons allow STDOUT/STDERR redirection into INFO/ERROR log respectively through redirect_stdout_stderr startup flag. If redirect_stdout_stderr is true, daemons redirect STDOUT/STDERR stream to write into the log file symlink created by glog. There are two problems with this approach: 1. Glog updates the symlink to point to the new log file when it does log rotation. However, Impala is not aware that the symlink point to a different file. So cout/cerr write still goes to the oldest log file. 2. When there is a lot of write activity to cout/cerr, the log file can grow big. However, glog is not aware of STDOUT/STDERR activity. It only counts the message bytes written to glog (LOG(INFO), LOG(ERROR)). Thus, it only uses its internal bytes count when deciding to rotate the logs. This commit addresses the issue by monitoring the log file size every second. If Impala sees that the log file has exceeded max_log_size, it will call google::FlushLogFiles(), ahead of logbufsecs. If the log file stays big after the flush, we will force the glog to rotate the log. Since there is no direct way to force glog to rotate, we do this by changing the log extension to random extension through google::SetLogFilenameExtension(), and immediately return them to extensionless (empty string extension). We also check periodically whether the log file symlink has pointed to a new file. If it has changed, we reattach the STDOUT/STDERR stream to the new log file. Testing: - Pass the core test. - Add new exhaustive test TestLogging::test_excessive_cerr. Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 --- M be/src/common/init.cc M be/src/common/logging.cc M be/src/common/logging.h M be/src/util/filesystem-util.cc M be/src/util/filesystem-util.h M tests/custom_cluster/test_breakpad.py 6 files changed, 254 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/17997/2 -- To view, visit http://gerrit.cloudera.org:8080/17997 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 Gerrit-Change-Number: 17997 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7607/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 20:35:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11004 (part 2): Upgrade glog library for Impala
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17951 ) Change subject: IMPALA-11004 (part 2): Upgrade glog library for Impala .. Patch Set 3: Code-Review+2 This is looking good to me. -- To view, visit http://gerrit.cloudera.org:8080/17951 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic57ebb2f75f5bdb223d6e3f6b4de4f2ebaac68de Gerrit-Change-Number: 17951 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 08 Nov 2021 20:34:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10943: Add test to verify support for multiple resource and executor pools
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17891 ) Change subject: IMPALA-10943: Add test to verify support for multiple resource and executor pools .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17891 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If76d386d8de5730da937674ddd9a69aa1aa1355e Gerrit-Change-Number: 17891 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 08 Nov 2021 20:33:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10943: Add test to verify support for multiple resource and executor pools
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17891 ) Change subject: IMPALA-10943: Add test to verify support for multiple resource and executor pools .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7606/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17891 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If76d386d8de5730da937674ddd9a69aa1aa1355e Gerrit-Change-Number: 17891 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 08 Nov 2021 20:33:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10943: Add test to verify support for multiple resource and executor pools
Bikramjeet Vig has removed a vote on this change. Change subject: IMPALA-10943: Add test to verify support for multiple resource and executor pools .. Removed Verified-1 by Impala Public Jenkins -- To view, visit http://gerrit.cloudera.org:8080/17891 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: If76d386d8de5730da937674ddd9a69aa1aa1355e Gerrit-Change-Number: 17891 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9734/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 14 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Mon, 08 Nov 2021 20:32:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. Patch Set 14: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7605/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 14 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Mon, 08 Nov 2021 20:11:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Yu-Wen Lai has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables To enable fine-grained table refreshing, there are three main changes in this commit. 1. Maintain validWriteIdList in Catalogd for transactional tables. We will keep track of write id changes for partitioned tables by AllocWriteIdEvents, CommitTxnEvents, and AbortTxnEvents. 2. Conduct partition level refreshing for transactional tables' addPartitionEvents, dropPartitionEvents, and AlterPartitionEvents. 3. Introduce a config hms_event_incremental_refresh_transactional_table, which can switch on/off the fine-grained table refreshing. Performance Tests: A simple test was performed by running insert into one partition for a partitioned ACID table(50,000 partitions). Below are the time taken to refresh this table by the event. StorageBefore After = S3 50 secs 50 msecs local 3 secs 3 msecs Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/Catalog.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/TableWriteId.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/hive/common/MutableValidReaderWriteIdList.java M fe/src/main/java/org/apache/impala/hive/common/MutableValidWriteIdList.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java A fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/hive/common/MutableValidReaderWriteIdListTest.java 17 files changed, 1,093 insertions(+), 68 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/17858/14 -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 14 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai
[Impala-ASF-CR] [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18004 ) Change subject: [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7602/ -- To view, visit http://gerrit.cloudera.org:8080/18004 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8cae9cccdcd94ef2c0f5cfaf5ddb15eaa8c46f5c Gerrit-Change-Number: 18004 Gerrit-PatchSet: 1 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 19:21:06 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17996 ) Change subject: WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9733/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a Gerrit-Change-Number: 17996 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 08 Nov 2021 18:43:41 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala
Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17996 ) Change subject: WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala .. WIP IMPALA-11005 (part 2): Upgrade Boost library to 1.74.0 for Impala There are some header files deprecated in the new version of Boost library. Need to define BOOST_ALLOW_DEPRECATED_HEADERS in CMakeLists.txt to avoid compiling errors. Also define BOOST_BIND_GLOBAL_PLACEHOLDERS to keep current behaviour of boost::bind and avoid big code changes. Define exception handler for a new boost::throw_exception() API since BOOST_NO_EXCEPTIONS is defined in be/CMakeLists.txt and we have to provide handlers which will be called by codegen'd code. Replace random number generator ranlux64_3 with mt19937_64 for better performance. Revert the code change made by IMPALA-2846 and IMPALA-9571 since the bug was fixed in Boost 1.74.0. Testing: - Passed core DEBUG build and exhaustive release build. - Passed core ASAN build. Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a --- M be/CMakeLists.txt M be/src/benchmarks/convert-timestamp-benchmark.cc M be/src/codegen/llvm-codegen.cc M be/src/exprs/aggregate-functions-ir.cc M be/src/runtime/bufferpool/buffer-pool-test.cc M be/src/runtime/timestamp-value.cc M be/src/statestore/failure-detector.h M be/src/util/blocking-queue-test.cc M be/src/util/condition-variable.h M be/src/util/filesystem-util.cc M bin/impala-config.sh 11 files changed, 36 insertions(+), 37 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/17996/3 -- To view, visit http://gerrit.cloudera.org:8080/17996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I78f32ae3c274274325e7af9e9bc9643814ae346a Gerrit-Change-Number: 17996 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 9: GVO over patch set 9 hit flaky test previously reported at IMPALA-10886. That flaky test is not related with this patch. -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 18:13:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10791 Add batching reading for remote temporary files
Qifan Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/17979 )
Change subject: IMPALA-10791 Add batching reading for remote temporary files
..
Patch Set 5:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/17979/5/be/src/runtime/io/disk-io-mgr.cc
File be/src/runtime/io/disk-io-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17979/5/be/src/runtime/io/disk-io-mgr.cc@396
PS5, Line 396: DCHECK(disk_file_dst_ != nullptr);
: DCHECK(disk_file_src_ != nullptr);
May need to add a comment to describe these two files. An example would be
helpful too.
http://gerrit.cloudera.org:8080/#/c/17979/5/be/src/runtime/io/disk-io-mgr.cc@424
PS5, Line 424: "Mem block '$0' has been deleted, path: '$1'",
buffer_idx, remote_file_path));
Why can't we use this DISABLED memory block here? Looks like we should not
miss the opportunity of a good use of the memory block.
http://gerrit.cloudera.org:8080/#/c/17979/5/be/src/runtime/io/disk-io-mgr.cc@436
PS5, Line 436: ScopedHistogramTimer read_timer(queue->read_latency());
This is not used.
http://gerrit.cloudera.org:8080/#/c/17979/5/be/src/runtime/io/disk-io-mgr.cc@445
PS5, Line 445: status = Status(TErrorCode::DISK_IO_ERROR, GetBackendString(),
: GetHdfsErrorMsg("Error reading from HDFS file: ",
remote_file_path));
Should we set the memory block allocated above to DISABLED here so that it can
be used by reading from other offsets?
Also, I think we should close the file even in the error reading case.
--
To view, visit http://gerrit.cloudera.org:8080/17979
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dcc5d0881ffaeff09c5c514306cd668373ad31b
Gerrit-Change-Number: 17979
Gerrit-PatchSet: 5
Gerrit-Owner: Yida Wu
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Qifan Chen
Gerrit-Reviewer: Yida Wu
Gerrit-Comment-Date: Mon, 08 Nov 2021 17:39:26 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11007: Replies to HTTP HEAD requests should contain no content.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18002 ) Change subject: IMPALA-11007: Replies to HTTP HEAD requests should contain no content. .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7604/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18002 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc3bf72067dfa5bc24e29fe974e5762be507952 Gerrit-Change-Number: 18002 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 17:36:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11007: Replies to HTTP HEAD requests should contain no content.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18002 ) Change subject: IMPALA-11007: Replies to HTTP HEAD requests should contain no content. .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18002 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc3bf72067dfa5bc24e29fe974e5762be507952 Gerrit-Change-Number: 18002 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 17:36:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 9: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7601/ -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 16:58:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11007: Replies to HTTP HEAD requests should contain no content.
Abhishek Rawat has posted comments on this change. ( http://gerrit.cloudera.org:8080/18002 ) Change subject: IMPALA-11007: Replies to HTTP HEAD requests should contain no content. .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18002 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc3bf72067dfa5bc24e29fe974e5762be507952 Gerrit-Change-Number: 18002 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 16:11:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10791 Add batching reading for remote temporary files
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17979 ) Change subject: IMPALA-10791 Add batching reading for remote temporary files .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/17979/5/be/src/runtime/io/disk-file.h File be/src/runtime/io/disk-file.h: http://gerrit.cloudera.org:8080/#/c/17979/5/be/src/runtime/io/disk-file.h@326 PS5, Line 326: int read_buffer_idx = offset / read_buffer_size(); > Can you pls double check? I saw the IF check below which should take care of index being 12. Since the assigned idx is always the last entry, we will have a skew. A better approach is to use % num_of_read_buffers() to distribute it evenly. -- To view, visit http://gerrit.cloudera.org:8080/17979 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1dcc5d0881ffaeff09c5c514306cd668373ad31b Gerrit-Change-Number: 17979 Gerrit-PatchSet: 5 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 08 Nov 2021 15:50:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11006: Impalad crashes during query cancel tests
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17999 ) Change subject: IMPALA-11006: Impalad crashes during query cancel tests .. Patch Set 1: (2 comments) Thanks a lot for the careful review! http://gerrit.cloudera.org:8080/#/c/17999/1/be/src/service/client-request-state.cc File be/src/service/client-request-state.cc: http://gerrit.cloudera.org:8080/#/c/17999/1/be/src/service/client-request-state.cc@1070 PS1, Line 1070: else > What is query_status_ in case of cancellation? Shouldn't it be in an error CTAS is processed async with a difference before the RPC patch. Main steps involved: 1. COMPILE 2. CREATE TABLE 3. POPULATE TABLE Before: 1 and 2 in main thread, 3 in async_exec_thread After: 2 in main thread, 2 and 3 in async_exec_thread GetCoordinator() returns TRUE iff coord_exec_called_ is true which is set only inside ClientRequestState::FinishExecQueryOrDmlRequest(), which is called only if there is no interruption (like cancel)) and step 3 completes successfully. In the core dump case, cancel is received from the client and the above method is never called. So GetCoordinator() is FALSE. My limited understanding of the code is that the GetCoordinator() is available (and to get results from it) only if there is no interruption. We therefore have to check it being FALSE case here. http://gerrit.cloudera.org:8080/#/c/17999/1/be/src/service/client-request-state.cc@1076 PS1, Line 1076: MarkInactive > I think that MarkInactive() is not needed in case of cancellation, as we ha I instrumented the code to observe when MarkActive() and MarkInactive() for the relevant this object and found the following. MarkActive(Exec()): this=0xe62e800, updated ref_count_=1 MarkActive(Finalize()): this=0xe62e800, updated ref_count_=2 MarkInactive(WaitInternal() for ELSE branch): this=0xe62e800, updated ref_count_=1 So if MarkInactive() is not called here, the ref count will be 2. Per code calling convention, EXEC() and WAIT() (which calls WaitInternal() are paired so ideally it is a good idea to decrement ref_count here. But I am Okay with remove MarkInactive() as the reference count is not back to 0 anyway. -- To view, visit http://gerrit.cloudera.org:8080/17999 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia49411f8525734b8d463d9ffbfbce705b90a8d73 Gerrit-Change-Number: 17999 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Mon, 08 Nov 2021 15:23:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. Patch Set 13: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7600/ -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 13 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Mon, 08 Nov 2021 14:35:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10926: Improve catalogd consistency and self events detection
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17859 ) Change subject: IMPALA-10926: Improve catalogd consistency and self events detection .. Patch Set 27: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7603/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17859 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36364e401911352c474eb98c8d61bbaae9b9 Gerrit-Change-Number: 17859 Gerrit-PatchSet: 27 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Mon, 08 Nov 2021 14:35:54 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10920: Unnest function for zipping unnest arrays
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17983 ) Change subject: WiP: IMPALA-10920: Unnest function for zipping unnest arrays .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9732/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic58ff6579ecff03962e7a8698edfbe0684ce6cf7 Gerrit-Change-Number: 17983 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 13:55:39 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10920: Unnest function for zipping unnest arrays
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/17983 ) Change subject: WiP: IMPALA-10920: Unnest function for zipping unnest arrays .. Patch Set 2: (33 comments) http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h File be/src/exec/unnest-node.h: http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@40 PS2, Line 40: SlotRefs If they are always SlotRefs, can't the type of the vector be std::vector? http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@42 PS2, Line 42: 'coll_values_', but instead manually retrieve the slot values to support projection : /// (see class comment) I think it should be mentioned that 'coll_values_' and the class comment are in the UnnestNode class, not this one. (Of course this should have been done when extracting plan node classes.) http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@46 PS2, Line 46: coll_expr_evals_ 'coll_expr_evals_' has been removed. http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@46 PS2, Line 46: Set in : /// Prepare(). These slots are always set to NULL in Open() as a simple projection. It seems no longer to be true that they are set in Prepare() and set to NULL in Open(). http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@50 PS2, Line 50: Set in Prepare() It seems no longer to be the case. http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@109 PS2, Line 109: s Nit: belong. http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@116 PS2, Line 116: don't Nit: doesn't. http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@126 PS2, Line 126: Set in Prepare(). These slots are always : /// set to NULL in Open() as a simple projection Does not seem to be true anymore. http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.h@131 PS2, Line 131: Set in Prepare() Does not seem to be true. http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.cc File be/src/exec/unnest-node.cc: http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.cc@60 PS2, Line 60: DCHECK(slot_ref != nullptr); This DCHECK is superfluous because slot_ref==null iff coll_expr==nullptr, and in that case we have already crashed at L58. This is because static_cast does not perform any runtime check on the actual type of coll_expr and doesn't turn it into nullptr if the type is not SlotRef. It is dynamic_cast that would do it this way. http://gerrit.cloudera.org:8080/#/c/17983/2/be/src/exec/unnest-node.cc@174 PS2, Line 174: collection Nit: should be collections. http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/cup/sql-parser.cup File fe/src/main/cup/sql-parser.cup: http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/cup/sql-parser.cup@3091 PS2, Line 3091: "not enough aliases" What if the user provides more aliases than paths? http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java File fe/src/main/java/org/apache/impala/analysis/Analyzer.java: http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@574 PS2, Line 574: tableRefsFromUnnestExpr Nit: Should end in an underscore (_). http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@706 PS2, Line 706: if (aliasMap_.containsKey(uniqueAlias)) return true; : return false; Nit: 'return aliasMap_.containsKey(uniqueAlias)' would be simpler. http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@966 PS2, Line 966: return Isn't this a bug at this point if the type is not a SlotRef? Shouldn't we assert that it is rather than silently ignore it? http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java File fe/src/main/java/org/apache/impala/analysis/SelectStmt.java: http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java@389 PS2, Line 389: fromClause_.add(tblRef) Does this make it necessary to re-analyse 'fromClause_'? http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java File fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java: http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java@31 PS2, Line 31: UnnestExpr Could we have a class comment saying that this is for zipping unnests in the select list? http://gerrit.cloudera.org:8080/#/c/17983/2/fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java@36 PS2, Line 36: super(path); Don't we have to call removeItemFromPath() here? What if 'path' already contains "item"? Will it be added once again on
[Impala-ASF-CR] WiP: IMPALA-10920: Unnest function for zipping unnest arrays
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/17983 )
Change subject: WiP: IMPALA-10920: Unnest function for zipping unnest arrays
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/17983/3/fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java
File fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java:
http://gerrit.cloudera.org:8080/#/c/17983/3/fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java@49
PS3, Line 49: protected void analyzeImpl(Analyzer analyzer) throws
AnalysisException {
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/17983/3/fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java@50
PS3, Line 50: Preconditions.checkNotNull(rawPath_);
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/17983/3/fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java@109
PS3, Line 109: Type resolvedType =
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/17983/3/tests/query_test/test_nested_types.py
File tests/query_test/test_nested_types.py:
http://gerrit.cloudera.org:8080/#/c/17983/3/tests/query_test/test_nested_types.py@196
PS3, Line 196: class TestZippingUnnest(ImpalaTestSuite):
flake8: E302 expected 2 blank lines, found 1
--
To view, visit http://gerrit.cloudera.org:8080/17983
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic58ff6579ecff03962e7a8698edfbe0684ce6cf7
Gerrit-Change-Number: 17983
Gerrit-PatchSet: 3
Gerrit-Owner: Gabor Kaszab
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Mon, 08 Nov 2021 13:33:34 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10920: Unnest function for zipping unnest arrays
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17983 to look at the new patch set (#3). Change subject: WiP: IMPALA-10920: Unnest function for zipping unnest arrays .. WiP: IMPALA-10920: Unnest function for zipping unnest arrays The SQL standard compliant syntax is kind of ready. TODO: The Postgres kind of syntax where UNNEST() is in the select list. Change-Id: Ic58ff6579ecff03962e7a8698edfbe0684ce6cf7 --- M be/src/exec/unnest-node.cc M be/src/exec/unnest-node.h M common/thrift/PlanNodes.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java A fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/planner/UnnestNode.java M fe/src/main/jflex/sql-scanner.flex M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java A testdata/ComplexTypesTbl/arrays.orc A testdata/ComplexTypesTbl/arrays.parq M testdata/data/README M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/zipping-unnest-in-from-clause.test A testdata/workloads/functional-query/queries/QueryTest/zipping-unnest-in-select-list.test M tests/query_test/test_nested_types.py 29 files changed, 1,347 insertions(+), 132 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/17983/3 -- To view, visit http://gerrit.cloudera.org:8080/17983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic58ff6579ecff03962e7a8698edfbe0684ce6cf7 Gerrit-Change-Number: 17983 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11006: Impalad crashes during query cancel tests
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17999 ) Change subject: IMPALA-11006: Impalad crashes during query cancel tests .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/17999/1/be/src/service/client-request-state.cc File be/src/service/client-request-state.cc: http://gerrit.cloudera.org:8080/#/c/17999/1/be/src/service/client-request-state.cc@1070 PS1, Line 1070: else What is query_status_ in case of cancellation? Shouldn't it be in an error status, which would lead to returning at line 1058? I looked around and I think that we are quite inconsistent here - Cancel() only updates the query status if there is a "cause": https://github.com/apache/impala/blob/9d2ef8564786d858db7786ad338b7daa5386eb20/be/src/service/client-request-state.cc#L1343 Otherwise only is_cancelled_ will become true. This seems intentional: https://github.com/apache/impala/blob/9d2ef8564786d858db7786ad338b7daa5386eb20/be/src/service/client-request-state.h#L175 But if the cancellation happens at some certain times, we do set query_status_ regardless of having a cause or not: https://github.com/apache/impala/blob/9d2ef8564786d858db7786ad338b7daa5386eb20/be/src/service/client-request-state.cc#L612 I think that this logic would be clearer if we would decide first whether GetCoordinator() should return non-null - my understanding is that there are some ddl-s that still won't have a coordinator. In case the it should have a coordinator but it doesn't, then we could check if it was cancelled and hit a DCHECK if it was not. http://gerrit.cloudera.org:8080/#/c/17999/1/be/src/service/client-request-state.cc@1076 PS1, Line 1076: MarkInactive I think that MarkInactive() is not needed in case of cancellation, as we have already closed the query and expect no more RPCs from the client. Calling MarkInactive() means that Impala is no longer active, but we are waiting for more activity from the client. -- To view, visit http://gerrit.cloudera.org:8080/17999 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia49411f8525734b8d463d9ffbfbce705b90a8d73 Gerrit-Change-Number: 17999 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 08 Nov 2021 13:28:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18004 ) Change subject: [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7602/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/18004 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8cae9cccdcd94ef2c0f5cfaf5ddb15eaa8c46f5c Gerrit-Change-Number: 18004 Gerrit-PatchSet: 1 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 13:11:44 + Gerrit-HasComments: No
[Impala-ASF-CR] [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18004 ) Change subject: [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9731/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18004 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8cae9cccdcd94ef2c0f5cfaf5ddb15eaa8c46f5c Gerrit-Change-Number: 18004 Gerrit-PatchSet: 1 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 13:07:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10926: Improve catalogd consistency and self events detection
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17859 ) Change subject: IMPALA-10926: Improve catalogd consistency and self events detection .. Patch Set 27: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9730/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17859 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36364e401911352c474eb98c8d61bbaae9b9 Gerrit-Change-Number: 17859 Gerrit-PatchSet: 27 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Mon, 08 Nov 2021 13:02:17 + Gerrit-HasComments: No
[Impala-ASF-CR] [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18004 ) Change subject: [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/18004/1/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java File fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java: http://gerrit.cloudera.org:8080/#/c/18004/1/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@2402 PS1, Line 2402: batchEvents = eventFactory.createBatchEvents(mockEvents, eventsProcessor_.getMetrics()); line too long (94 > 90) http://gerrit.cloudera.org:8080/#/c/18004/1/fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsSyncToLatestEventIdTest.java File fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsSyncToLatestEventIdTest.java: http://gerrit.cloudera.org:8080/#/c/18004/1/fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsSyncToLatestEventIdTest.java@85 PS1, Line 85: private static boolean flagEnableCatalogCache ,flagInvalidateCache, flagSyncToLatestEventId; line too long (96 > 90) -- To view, visit http://gerrit.cloudera.org:8080/18004 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8cae9cccdcd94ef2c0f5cfaf5ddb15eaa8c46f5c Gerrit-Change-Number: 18004 Gerrit-PatchSet: 1 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Nov 2021 12:45:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints
Sourabh Goyal has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18004 Change subject: [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints .. [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints Change-Id: I8cae9cccdcd94ef2c0f5cfaf5ddb15eaa8c46f5c --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/events/EventFactory.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/NoOpEventProcessor.java M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/catalog/metastore/HmsApiNameEnum.java M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java A fe/src/test/java/org/apache/impala/catalog/MetastoreApiTestUtils.java M fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/catalog/events/SynchronousHMSEventProcessorForTests.java M fe/src/test/java/org/apache/impala/catalog/metastore/AbstractCatalogMetastoreTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsSyncToLatestEventIdTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/custom_cluster/test_metastore_service.py 26 files changed, 3,362 insertions(+), 282 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/18004/1 -- To view, visit http://gerrit.cloudera.org:8080/18004 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8cae9cccdcd94ef2c0f5cfaf5ddb15eaa8c46f5c Gerrit-Change-Number: 18004 Gerrit-PatchSet: 1 Gerrit-Owner: Sourabh Goyal
[Impala-ASF-CR] [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints
Sourabh Goyal has abandoned this change. ( http://gerrit.cloudera.org:8080/17964 ) Change subject: [DO NOT MERGE] IMPALA-10926: Sync db/table in catalog cache to latest HMS event id when performing DDL operations via catalog HMS endpoints .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/17964 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I869268c4c23366ed0719b153252338af9738a5f6 Gerrit-Change-Number: 17964 Gerrit-PatchSet: 5 Gerrit-Owner: Sourabh Goyal Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10926: Improve catalogd consistency and self events detection
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/17859 )
Change subject: IMPALA-10926: Improve catalogd consistency and self events
detection
..
Patch Set 27:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/17859/27/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/17859/27/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2423
PS27, Line 2423:* String, long)} which passes false for {@code
refreshUpdatedPartitions} argument and ignore
line too long (95 > 90)
http://gerrit.cloudera.org:8080/#/c/17859/27/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/17859/27/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@5878
PS27, Line 5878: updatedThriftTable = catalog_.reloadTable(tbl,
req, resultType, cmdString, -1);
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/17859/27/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
File
fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java:
http://gerrit.cloudera.org:8080/#/c/17859/27/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@2471
PS27, Line 2471: batchEvents = eventFactory.createBatchEvents(mockEvents,
eventsProcessor_.getMetrics());
line too long (94 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36364e401911352c474eb98c8d61bbaae9b9
Gerrit-Change-Number: 17859
Gerrit-PatchSet: 27
Gerrit-Owner: Sourabh Goyal
Gerrit-Reviewer: Anonymous Coward
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sourabh Goyal
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Reviewer: Yu-Wen Lai
Gerrit-Comment-Date: Mon, 08 Nov 2021 12:41:03 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10926: Improve catalogd consistency and self events detection
Sourabh Goyal has posted comments on this change. (
http://gerrit.cloudera.org:8080/17859 )
Change subject: IMPALA-10926: Improve catalogd consistency and self events
detection
..
Patch Set 27:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java:
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@328
PS26, Line 328: toryForSyncToLatestEvent(Catalog
> remove if not needed.
Ack
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@919
PS26, Line 919: }
: org.apache.impala.catalog.Table tbl = null;
: t
> Not sure if I understand this condition correctly. Why are evaluating old s
This is not the right condition because if sync to latest event id flag is set
to true and the event is *not* a self event then the code from line no: 922
shall not get executed as tbl.setLastSyncedEventId(getEventId()) would get
called while processing the event.
To make this code more readable I can modify the if condition to:
if (isSelfEvent && BackendConfig.INSTANCE.enableSyncToLatestEventOnDdls() ) {
tbl = catalog_.getTable(getDbName(), getTableName());
if (tbl != null && catalog_.tryWriteLock(tbl)) {
catalog_.getLock().writeLock().unlock();
if (tbl.getLastSyncedEventId() < getEventId()) {
infoLog("is a self event. last synced event id for "
+ "table {} is {}. Setting it to {}", tbl.getFullName(),
tbl.getLastSyncedEventId(), getEventId());
tbl.setLastSyncedEventId(getEventId());
}
}
Thoughts?
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@932
PS26, Line 932:
> Why do we need to set the lastSyncedEventId here? Can we keep the scope of
MetastoreEventFactory (and not EventFactoryForSyncToLatestEvent) skips
processing an event which is self event. In that case, for a table, event
factory should set the last synced event id to this self event id if
enableSyncToLatestEventOnDdls() is set before skipping the processing of an
event (more details in method comments)
I am not sure what is the right place to do that and thats what this overridden
method isSelfEvent does.
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java
File
fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java:
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@3088
PS26, Line 3088: String ms
> remove if not needed.
Ack
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@749
PS26, Line 749: if (existingTable != null) {
: LOG.debug("EventId: {} Table {} was not added
> Is this something that you are working on?
I am not looking into it right now. But when I was working on it, the question
in TODO comment crossed my mind. I am not sure if it is a valid scenario and
for now I am thinking of adding a warning message if existing table's create
event id does not match the event id passed in method argument.
http://gerrit.cloudera.org:8080/#/c/17859/26/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@929
PS26, Line 929: }
> Is this TODO still unresolved? Please remove if it is. My understanding is
Ack
--
To view, visit http://gerrit.cloudera.org:8080/17859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36364e401911352c474eb98c8d61bbaae9b9
Gerrit-Change-Number: 17859
Gerrit-PatchSet: 27
Gerrit-Owner: Sourabh Goyal
Gerrit-Reviewer: Anonymous Coward
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sourabh Goyal
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Reviewer: Yu-Wen Lai
Gerrit-Comment-Date: Mon, 08 Nov 2021 12:40:54 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10926: Improve catalogd consistency and self events detection
Hello Vihang Karajgaonkar, kis...@cloudera.com, Yu-Wen Lai, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17859 to look at the new patch set (#27). Change subject: IMPALA-10926: Improve catalogd consistency and self events detection .. IMPALA-10926: Improve catalogd consistency and self events detection In the current design catalogd cache gets updated from 2 sources: 1. Impala shell 2. MetastoreEventProcessor The updates from the Impala shell are applied in place whereas MetastoreEventProcessor runs as a background thread, polls HMS events and apply them asynchronously. These two stream of updates cause consistency issues. For example consider a following sequence of alter table events on a table t1 as per HMS: 1. alter table t1 from source s1 say other Impala cluster 2. alter table t1 from source s2 say other Hive cluster 3. alter table t1 from local Impala cluster The #3 alter table ddl operation would get reflected in the local cache immediately. However, later on event processor would process events from #1 and #2 above and try to alter the table. In an ideal scenario, these alters should have been applied before #3 i.e in the same order as they appear in HMS notification log. This leaves table t1 in an inconsistent state. Proposed solution: The main idea of the solution is to keep track of the last event id for a given table as eventId which the catalogd has synced to in the Table object. The events processor ignores any event whose EVENT_ID is less than or equal to the eventId stored in the table. Once the events processor successfully processes a given event, it updates the value of eventId in the table before releasing the table lock. Also, any DDL or refresh operation on the catalogd (from both catalog HMS endpoint and Impala shell) will follow the following steps to update the event id for the table: 1. Acquire write lock on the table 2. Perform ddl operation in HMS 3. Sync table till the latest event id (as per HMS) since its last synced event id The above steps ensure that any concurrent updates applied on a same db/table from multiple sources like Hive, Impala or say multiple Impala clusters, get reflected in the local catalogd cache (in the same order as they appear in HMS) thus removing any inconsistencies. Also the solution relies on the existing locking mechanism in the catalogd to prevent any other concurrent updates to the table (even via EventsProcessor). In case of database objects, we will also have a similar eventId which represents the events on the database object (CREATE, DROP, ALTER database) to which the catalogd as synced to. This patch addresses the following: 1. Add a new flag enable_sync_to_latest_event_on_ddls to enable/disable this improvement. It is turned off by default. 2. If flag in #1 is enabled then apart from Impala shell and MetastoreEventProcessor the cache would also get updated for ddls executed via catalog HMS endpoints. While excuting a ddl, db/table will be synced till latest event id. 3. Event processor skips processing an event if db/table is already synced till that event id. Sets that event id in db/table if the event is processed. 4. When EventProcessor detects a self event, it sets the last synced event id in db/table before skipping the processing of an event. 5. Full table refresh sets the last event processed in table cache. Future Work: 1. Sync db/table to latest event id for ddls executed from Impala shell (execDdlRequest() in catalogOpExecutor) Testing: 1. Added new unit tests and modified existing ones 2. Ran exhaustive tests with flag both turned on and off Change-Id: I36364e401911352c474eb98c8d61bbaae9b9 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/events/EventFactory.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/NoOpEventProcessor.java M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/catalog/metastore/HmsApiNameEnum.java M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java A
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 10:43:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7601/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 10:43:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10984: Improve TimestampValue to String casting
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17980 ) Change subject: IMPALA-10984: Improve TimestampValue to String casting .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fcb4545d9c9a3fdb38c4db58bb4b1321a429d61 Gerrit-Change-Number: 17980 Gerrit-PatchSet: 8 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 08 Nov 2021 10:43:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 ) Change subject: IMPALA-10934: Enable table definition over a single file .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7 Gerrit-Change-Number: 17878 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 08 Nov 2021 08:54:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9729/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 13 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Mon, 08 Nov 2021 08:46:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Yu-Wen Lai has uploaded a new patch set (#13). ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables To enable fine-grained table refreshing, there are three main changes in this commit. 1. Maintain validWriteIdList in Catalogd for transactional tables. We will keep track of write id changes for partitioned tables by AllocWriteIdEvents, CommitTxnEvents, and AbortTxnEvents. 2. Conduct partition level refreshing for transactional tables' addPartitionEvents, dropPartitionEvents, and AlterPartitionEvents. 3. Introduce a config hms_event_incremental_refresh_transactional_table, which can switch on/off the fine-grained table refreshing. Performance Tests: A simple test was performed by running insert into one partition for a partitioned ACID table(50,000 partitions). Below are the time taken to refresh this table by the event. StorageBefore After = S3 50 secs 50 msecs local 3 secs 3 msecs Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/Catalog.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/TableWriteId.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/hive/common/MutableValidReaderWriteIdList.java M fe/src/main/java/org/apache/impala/hive/common/MutableValidWriteIdList.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java A fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/hive/common/MutableValidReaderWriteIdListTest.java 17 files changed, 1,089 insertions(+), 68 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/17858/13 -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 13 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai
[Impala-ASF-CR] IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17858 ) Change subject: IMPALA-10923: Fine grained table refreshing at partition level events for transactional tables .. Patch Set 13: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7600/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17858 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6ba07c9a338a25614690e314335ee4b801486da9 Gerrit-Change-Number: 17858 Gerrit-PatchSet: 13 Gerrit-Owner: Yu-Wen Lai Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sourabh Goyal Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Mon, 08 Nov 2021 08:25:17 + Gerrit-HasComments: No
