[kudu-CR] [thirdparty] Upgrade CMake to 3.19
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16785 Change subject: [thirdparty] Upgrade CMake to 3.19 .. [thirdparty] Upgrade CMake to 3.19 CMake 3.19 has some improvements when building on macOS, especially will Xcode 12+. This patch also includeds some fixes for macOS that would likely occur when using this new version of CMake and potentially occured on older versions as well depending on the environment. Change-Id: I125448a3057d0c9da8c82f7e29683ef7844b0aa8 --- M CMakeLists.txt M src/kudu/codegen/CMakeLists.txt M src/kudu/gutil/mathlimits.h M src/kudu/util/env_posix.cc M thirdparty/vars.sh 5 files changed, 19 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/85/16785/1 -- To view, visit http://gerrit.cloudera.org:8080/16785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I125448a3057d0c9da8c82f7e29683ef7844b0aa8 Gerrit-Change-Number: 16785 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [thirdparty] Upgrade Protobuf to 3.14.0
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16784 ) Change subject: [thirdparty] Upgrade Protobuf to 3.14.0 .. [thirdparty] Upgrade Protobuf to 3.14.0 This patch upgrades Protobuf from 3.12.3 to 3.14.0. Change-Id: I46617b4c55fc61b28b2837f7a0f04ba9061038ae Reviewed-on: http://gerrit.cloudera.org:8080/16784 Reviewed-by: Alexey Serbin Tested-by: Kudu Jenkins --- M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16784 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I46617b4c55fc61b28b2837f7a0f04ba9061038ae Gerrit-Change-Number: 16784 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [thirdparty] Upgrade Protobuf to 3.14.0
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16784 ) Change subject: [thirdparty] Upgrade Protobuf to 3.14.0 .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/16784/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16784/1//COMMIT_MSG@7 PS1, Line 7: Upgrade Protobuf > BTW, do you plan to update protobuf version used in Java part of the projec Yeah, I will bump the Java versions in bulk at some point per the usual "[Java] Upgrade Dependencies" commit. http://gerrit.cloudera.org:8080/#/c/16784/1//COMMIT_MSG@9 PS1, Line 9: This patch upgrades Protobuf from 3.12.3 to 3.14.0 > Does it make sense to add more colors about the motivation for this version I am hopeful this bump will fix the UBSAN issue I was seeing in the C++17 patch, but even if it doesn't we can/should merge it. For that reason, I left it out of this message. -- To view, visit http://gerrit.cloudera.org:8080/16784 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46617b4c55fc61b28b2837f7a0f04ba9061038ae Gerrit-Change-Number: 16784 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 25 Nov 2020 17:43:34 + Gerrit-HasComments: Yes
[kudu-CR] [thirdparty] Upgrade Protobuf to 3.14.0
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16784 Change subject: [thirdparty] Upgrade Protobuf to 3.14.0 .. [thirdparty] Upgrade Protobuf to 3.14.0 This patch upgrades Protobuf from 3.12.3 to 3.14.0. Change-Id: I46617b4c55fc61b28b2837f7a0f04ba9061038ae --- M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/16784/1 -- To view, visit http://gerrit.cloudera.org:8080/16784 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I46617b4c55fc61b28b2837f7a0f04ba9061038ae Gerrit-Change-Number: 16784 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] Explictly use boost::make optional
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16772 ) Change subject: Explictly use boost::make_optional .. Explictly use boost::make_optional This patch is in preparation for compilation with C++17. When compiling there are cases where `make_optional` is ambiguous because `make_optional` is also a part of `std::optional`. I fixed this by prefixing all existing usage of `make_optional` with `boost::`. Change-Id: I627fe57b42eb063dcb34057d0ea9f4847296bb8d Reviewed-on: http://gerrit.cloudera.org:8080/16772 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke --- M src/kudu/integration-tests/master_authz-itest.cc M src/kudu/integration-tests/master_hms-itest.cc M src/kudu/master/catalog_manager.cc 3 files changed, 4 insertions(+), 7 deletions(-) Approvals: Kudu Jenkins: Verified Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I627fe57b42eb063dcb34057d0ea9f4847296bb8d Gerrit-Change-Number: 16772 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] Explictly use boost::make optional
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16772 ) Change subject: Explictly use boost::make_optional .. Patch Set 2: Code-Review+2 Carrying Bankim's +2 through the rebase. -- To view, visit http://gerrit.cloudera.org:8080/16772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I627fe57b42eb063dcb34057d0ea9f4847296bb8d Gerrit-Change-Number: 16772 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 24 Nov 2020 20:36:28 + Gerrit-HasComments: No
[kudu-CR] [build] Support and require C++17
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16776 ) Change subject: [build] Support and require C++17 .. Patch Set 6: I ran dist_tests from Centos 7 with devtoolset-8 here: http://dist-test.cloudera.org/job?job_id=root.1606245312.441323 -- To view, visit http://gerrit.cloudera.org:8080/16776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d Gerrit-Change-Number: 16776 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 24 Nov 2020 19:29:58 + Gerrit-HasComments: No
[kudu-CR] KUDU-2700: Add a LSAN suppression
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16781 ) Change subject: KUDU-2700: Add a LSAN suppression .. KUDU-2700: Add a LSAN suppression OpenSSL 1.1 has a leak in libcrypto.so when running tests that use the CLI to connect to a remote server. Previously this didn’t impact CI runs because Jenkins and dist_test were using Ubuntu 14 and therefore OpenSSL 1.0. Now that we are using Ubuntu 18 and OpenSSL 1.1 this leak is reproducable 100% of the time. This patch suppresses the leak to unbreak the ASAN builds. Change-Id: I5d1378a820997d3eabfa7e5cda53c07dd4df917d Reviewed-on: http://gerrit.cloudera.org:8080/16781 Reviewed-by: Andrew Wong Reviewed-by: Alexey Serbin Tested-by: Kudu Jenkins --- M src/kudu/util/sanitizer_options.cc 1 file changed, 4 insertions(+), 0 deletions(-) Approvals: Andrew Wong: Looks good to me, approved Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16781 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5d1378a820997d3eabfa7e5cda53c07dd4df917d Gerrit-Change-Number: 16781 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2700: Add a LSAN suppression
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16781 to look at the new patch set (#2). Change subject: KUDU-2700: Add a LSAN suppression .. KUDU-2700: Add a LSAN suppression OpenSSL 1.1 has a leak in libcrypto.so when running tests that use the CLI to connect to a remote server. Previously this didn’t impact CI runs because Jenkins and dist_test were using Ubuntu 14 and therefore OpenSSL 1.0. Now that we are using Ubuntu 18 and OpenSSL 1.1 this leak is reproducable 100% of the time. This patch suppresses the leak to unbreak the ASAN builds. Change-Id: I5d1378a820997d3eabfa7e5cda53c07dd4df917d --- M src/kudu/util/sanitizer_options.cc 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/16781/2 -- To view, visit http://gerrit.cloudera.org:8080/16781 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5d1378a820997d3eabfa7e5cda53c07dd4df917d Gerrit-Change-Number: 16781 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2700: Add a LSAN suppression
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16781 ) Change subject: KUDU-2700: Add a LSAN suppression .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16781/1/src/kudu/util/sanitizer_options.cc File src/kudu/util/sanitizer_options.cc: http://gerrit.cloudera.org:8080/#/c/16781/1/src/kudu/util/sanitizer_options.cc@199 PS1, Line 199: leak:libcrypto.so.1\n > Would it make sense to narrow down the scope of the suppression to ERR_get_ I can be more specific on libcrypto.so.1.1, but I don't think can't get ERR_get_state() without fast_unwind_on_malloc=0, similar to libgssapi_krb5 below. FWIW the TSAN filter is broad as well. -- To view, visit http://gerrit.cloudera.org:8080/16781 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5d1378a820997d3eabfa7e5cda53c07dd4df917d Gerrit-Change-Number: 16781 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 24 Nov 2020 18:51:09 + Gerrit-HasComments: Yes
[kudu-CR] [build] Support and require C++17
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16776 to look at the new patch set (#6). Change subject: [build] Support and require C++17 .. [build] Support and require C++17 This patch adjust the build to start supporting and requiring C++17 (therefore GCC 7+). It adjusts the documentation and supported OS matrix to match the new requirements as well. I needed to replace std::random_shuffle with std::shuffle. In C++14 `std::random_shuffle` is deprectated and in C++17 it is removed all together. This patch replaces `std::random_shuffle` with `std::shuffle` passing in a generator. Additionally I needed to patch libev due to a small C++17 incompatibility: https://github.com/enki/libev/pull/8 Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d --- M CMakeLists.txt M README.adoc R build-support/ccache-devtoolset-8/c++ R build-support/ccache-devtoolset-8/cc M build-support/enable_devtoolset.sh M build-support/enable_devtoolset_inner.sh M docker/Dockerfile M docker/bootstrap-dev-env.sh M docs/installation.adoc M src/kudu/benchmarks/wal_hiccup.cc M src/kudu/client/client-test.cc M src/kudu/client/predicate-test.cc M src/kudu/client/symbols.map M src/kudu/common/wire_protocol-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/all_types-itest.cc M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/master/catalog_manager.cc M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/tablet.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/tserver/tablet_server-test.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh A thirdparty/patches/libev-c17.patch M thirdparty/preflight.py 27 files changed, 137 insertions(+), 93 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/16776/6 -- To view, visit http://gerrit.cloudera.org:8080/16776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d Gerrit-Change-Number: 16776 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [build] Support and require C++17
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16776 to look at the new patch set (#5). Change subject: [build] Support and require C++17 .. [build] Support and require C++17 This patch adjust the build to start supporting and requiring C++17 (therefore GCC 7+). It adjusts the documentation and supported OS matrix to match the new requirements as well. I needed to replace std::random_shuffle with std::shuffle. In C++14 `std::random_shuffle` is deprectated and in C++17 it is removed all together. This patch replaces `std::random_shuffle` with `std::shuffle` passing in a generator. Additionally I needed to patch libev due to a small C++17 incompatibility: https://github.com/enki/libev/pull/8 Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d --- M CMakeLists.txt M README.adoc R build-support/ccache-devtoolset-7/c++ R build-support/ccache-devtoolset-7/cc M build-support/enable_devtoolset.sh M build-support/enable_devtoolset_inner.sh M docker/Dockerfile M docker/bootstrap-dev-env.sh M docs/installation.adoc M src/kudu/benchmarks/wal_hiccup.cc M src/kudu/client/client-test.cc M src/kudu/client/predicate-test.cc M src/kudu/client/symbols.map M src/kudu/common/wire_protocol-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/all_types-itest.cc M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/master/catalog_manager.cc M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/tablet.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/tserver/tablet_server-test.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh A thirdparty/patches/libev-c17.patch M thirdparty/preflight.py 27 files changed, 137 insertions(+), 93 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/16776/5 -- To view, visit http://gerrit.cloudera.org:8080/16776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d Gerrit-Change-Number: 16776 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2700: Add a LSAN suppression
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16781 Change subject: KUDU-2700: Add a LSAN suppression .. KUDU-2700: Add a LSAN suppression OpenSSL 1.1 has a leak in libcrypto.so when running tests that use the CLI to connect to a remote server. Previously this didn’t impact CI runs because Jenkins and dist_test were using Ubuntu 14 and therefore OpenSSL 1.0. Now that we are using Ubuntu 18 and OpenSSL 1.1 this leak is reproducable 100% of the time. This patch suppresses the leak to unbreak the ASAN builds. Change-Id: I5d1378a820997d3eabfa7e5cda53c07dd4df917d --- M src/kudu/util/sanitizer_options.cc 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/16781/1 -- To view, visit http://gerrit.cloudera.org:8080/16781 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5d1378a820997d3eabfa7e5cda53c07dd4df917d Gerrit-Change-Number: 16781 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-3214 Parallelize DownloadWALs in tablet copy operation
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16726 ) Change subject: KUDU-3214 Parallelize DownloadWALs in tablet copy operation .. Patch Set 9: (1 comment) http://gerrit.cloudera.org:8080/#/c/16726/9/src/kudu/integration-tests/tablet_copy-itest.cc File src/kudu/integration-tests/tablet_copy-itest.cc: http://gerrit.cloudera.org:8080/#/c/16726/9/src/kudu/integration-tests/tablet_copy-itest.cc@2089 PS9, Line 2089: auto scan_func = [=client_, > I saw Grant's email, I guess c14 feature is available. They will be "officially" available once we merge this change: https://gerrit.cloudera.org/#/c/16776/ -- To view, visit http://gerrit.cloudera.org:8080/16726 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib513118d233efdf4be7cbb0b790354c4c3e1bd84 Gerrit-Change-Number: 16726 Gerrit-PatchSet: 9 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: wangning <1994wangn...@gmail.com> Gerrit-Comment-Date: Tue, 24 Nov 2020 13:11:24 + Gerrit-HasComments: Yes
[kudu-CR] WIP: [build] Support and require C++17
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16776 to look at the new patch set (#4). Change subject: WIP: [build] Support and require C++17 .. WIP: [build] Support and require C++17 This patch adjust the build to start supporting and requiring C++17 (therefore GCC 7+). It adjusts the documentation and supported OS matrix to match the new requirements as well. I needed to replace std::random_shuffle with std::shuffle. In C++14 `std::random_shuffle` is deprectated and in C++17 it is removed all together. This patch replaces `std::random_shuffle` with `std::shuffle` passing in a generator. Additionally I needed to patch libev due to a small C++17 incompatibility: https://github.com/enki/libev/pull/8 WIP Because: - I need to test the Centos 7 devtoolset changes Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d --- M CMakeLists.txt M README.adoc R build-support/ccache-devtoolset-7/c++ R build-support/ccache-devtoolset-7/cc M build-support/enable_devtoolset.sh M build-support/enable_devtoolset_inner.sh M docker/Dockerfile M docker/bootstrap-dev-env.sh M docs/installation.adoc M src/kudu/benchmarks/wal_hiccup.cc M src/kudu/client/client-test.cc M src/kudu/client/predicate-test.cc M src/kudu/client/symbols.map M src/kudu/common/wire_protocol-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/all_types-itest.cc M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/master/catalog_manager.cc M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/tablet.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/tserver/tablet_server-test.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh A thirdparty/patches/libev-c17.patch M thirdparty/preflight.py 27 files changed, 137 insertions(+), 93 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/16776/4 -- To view, visit http://gerrit.cloudera.org:8080/16776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d Gerrit-Change-Number: 16776 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] WIP: [build] Support and require C++17
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16776 to look at the new patch set (#3). Change subject: WIP: [build] Support and require C++17 .. WIP: [build] Support and require C++17 This patch adjust the build to start supporting and requiring C++17 (therefore GCC 7+). It adjusts the documentation and supported OS matrix to match the new requirements as well. I needed to replace std::random_shuffle with std::shuffle. In C++14 `std::random_shuffle` is deprectated and in C++17 it is removed all together. This patch replaces `std::random_shuffle` with `std::shuffle` passing in a generator. Additionally I needed to patch libev due to a small C++17 incompatibility: https://github.com/enki/libev/pull/8 WIP Because: - I need to test the Centos 7 devtoolset changes Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d --- M CMakeLists.txt M README.adoc R build-support/ccache-devtoolset-7/c++ R build-support/ccache-devtoolset-7/cc M build-support/enable_devtoolset.sh M build-support/enable_devtoolset_inner.sh M docker/Dockerfile M docker/bootstrap-dev-env.sh M docs/installation.adoc M src/kudu/benchmarks/wal_hiccup.cc M src/kudu/client/client-test.cc M src/kudu/client/predicate-test.cc M src/kudu/client/symbols.map M src/kudu/common/wire_protocol-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/all_types-itest.cc M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/tablet.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/tserver/tablet_server-test.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh A thirdparty/patches/libev-c17.patch M thirdparty/preflight.py 26 files changed, 128 insertions(+), 90 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/16776/3 -- To view, visit http://gerrit.cloudera.org:8080/16776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d Gerrit-Change-Number: 16776 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] WIP: [build] Support and require C++17
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16776 to look at the new patch set (#2). Change subject: WIP: [build] Support and require C++17 .. WIP: [build] Support and require C++17 This patch adjust the build to start supporting and requiring C++17 (therefore GCC 7+). It adjusts the documentation and supported OS matrix to match the new requirements as well. I needed to replace std::random_shuffle with std::shuffle. In C++14 `std::random_shuffle` is deprectated and in C++17 it is removed all together. This patch replaces `std::random_shuffle` with `std::shuffle` passing in a generator. Additionally I needed to patch libev due to a small C++17 incompatibility: https://github.com/enki/libev/pull/8 WIP Because: - I need to test the Centos 7 devtoolset changes Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d --- M CMakeLists.txt M README.adoc R build-support/ccache-devtoolset-7/c++ R build-support/ccache-devtoolset-7/cc M build-support/enable_devtoolset.sh M build-support/enable_devtoolset_inner.sh M docker/Dockerfile M docker/bootstrap-dev-env.sh M docs/installation.adoc M src/kudu/benchmarks/wal_hiccup.cc M src/kudu/client/client-test.cc M src/kudu/client/predicate-test.cc M src/kudu/client/symbols.map M src/kudu/common/wire_protocol-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/all_types-itest.cc M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/tablet.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/tserver/tablet_server-test.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh A thirdparty/patches/libev-c17.patch M thirdparty/preflight.py 26 files changed, 128 insertions(+), 90 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/16776/2 -- To view, visit http://gerrit.cloudera.org:8080/16776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d Gerrit-Change-Number: 16776 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] WIP: [build] Support and require C++17
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16776 Change subject: WIP: [build] Support and require C++17 .. WIP: [build] Support and require C++17 This patch adjust the build to start supporting and requiring C++17 (therefore GCC 7+). It adjusts the documentation and supported OS matrix to match the new requirements as well. I needed to replace std::random_shuffle with std::shuffle. In C++14 `std::random_shuffle` is deprectated and in C++17 it is removed all together. This patch replaces `std::random_shuffle` with `std::shuffle` passing in a generator. WIP Because: - I need to test the Centos 7 devtoolset changes Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d --- M CMakeLists.txt M README.adoc R build-support/ccache-devtoolset-7/c++ R build-support/ccache-devtoolset-7/cc M build-support/enable_devtoolset.sh M build-support/enable_devtoolset_inner.sh M docker/Dockerfile M docker/bootstrap-dev-env.sh M docs/installation.adoc M src/kudu/benchmarks/wal_hiccup.cc M src/kudu/client/client-test.cc M src/kudu/client/predicate-test.cc M src/kudu/client/symbols.map M src/kudu/common/wire_protocol-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/all_types-itest.cc M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/tablet.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/tserver/tablet_server-test.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/preflight.py 24 files changed, 87 insertions(+), 88 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/16776/1 -- To view, visit http://gerrit.cloudera.org:8080/16776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I15312473bda8c743eac228cd23c337fae5acf99d Gerrit-Change-Number: 16776 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] Replace std::random shuffle with std::shuffle
Grant Henke has abandoned this change. ( http://gerrit.cloudera.org:8080/16773 ) Change subject: Replace std::random_shuffle with std::shuffle .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/16773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I04d532f9bd199405128d911262062100db0fd994 Gerrit-Change-Number: 16773 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] Replace std::random shuffle with std::shuffle
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16773 Change subject: Replace std::random_shuffle with std::shuffle .. Replace std::random_shuffle with std::shuffle In C++14 `std::random_shuffle` is deprectated and in C++17 it is removed all together. This patch replaces `std::random_shuffle` with `std::shuffle` passing in a generator. Change-Id: I04d532f9bd199405128d911262062100db0fd994 --- M src/kudu/benchmarks/wal_hiccup.cc M src/kudu/client/client-test.cc M src/kudu/client/predicate-test.cc M src/kudu/common/wire_protocol-test.cc M src/kudu/fs/log_block_manager-test.cc M src/kudu/integration-tests/all_types-itest.cc M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/tablet.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/tserver/tablet_server-test.cc 11 files changed, 38 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/16773/1 -- To view, visit http://gerrit.cloudera.org:8080/16773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I04d532f9bd199405128d911262062100db0fd994 Gerrit-Change-Number: 16773 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] Explictly use boost::make optional
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16772 Change subject: Explictly use boost::make_optional .. Explictly use boost::make_optional This patch is in preparation for compilation with C++17. When compiling there are cases where `make_optional` is ambiguous because `make_optional` is also a part of `std::optional`. I fixed this by prefixing all existing usage of `make_optional` with `boost::`. Change-Id: I627fe57b42eb063dcb34057d0ea9f4847296bb8d --- M src/kudu/integration-tests/master_authz-itest.cc M src/kudu/integration-tests/master_hms-itest.cc M src/kudu/master/catalog_manager.cc 3 files changed, 4 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/72/16772/1 -- To view, visit http://gerrit.cloudera.org:8080/16772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I627fe57b42eb063dcb34057d0ea9f4847296bb8d Gerrit-Change-Number: 16772 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] WIP: Add abseil
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16769 Change subject: WIP: Add abseil .. WIP: Add abseil Change-Id: I64a2660a20da1054ac5fcd650a829216182fc659 --- M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 4 files changed, 49 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/69/16769/1 -- To view, visit http://gerrit.cloudera.org:8080/16769 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I64a2660a20da1054ac5fcd650a829216182fc659 Gerrit-Change-Number: 16769 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [build] Upgrade to LLVM 10.0.1 and IWYU 0.14
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16768 Change subject: [build] Upgrade to LLVM 10.0.1 and IWYU 0.14 .. [build] Upgrade to LLVM 10.0.1 and IWYU 0.14 This patch upgrade LLVM to version 10.0.1 and IWYU to version 0.14. It also bumps the language level to C++14 and the minimum gcc version accordingly. Follow on patches will upgrade the system requirements documentation to drop deprecated OS versions and GCC versions. Change-Id: Id9c32abe256978158617a4fe3a3c34e9bfd00fb2 --- M CMakeLists.txt M src/kudu/codegen/code_generator.cc M thirdparty/download-thirdparty.sh M thirdparty/package-llvm.sh M thirdparty/vars.sh 5 files changed, 14 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/68/16768/1 -- To view, visit http://gerrit.cloudera.org:8080/16768 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id9c32abe256978158617a4fe3a3c34e9bfd00fb2 Gerrit-Change-Number: 16768 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [test] fix compilation warning
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16766 ) Change subject: [test] fix compilation warning .. Patch Set 1: Verified+1 Code-Review+2 Failures are due to temporary Jenkins upgrade issues -- To view, visit http://gerrit.cloudera.org:8080/16766 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16ab253ab99808c2a69b0520a3a6b6253f27f2f8 Gerrit-Change-Number: 16766 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 23 Nov 2020 00:01:45 + Gerrit-HasComments: No
[kudu-CR] [test] fix compilation warning
Grant Henke has removed a vote on this change. Change subject: [test] fix compilation warning .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16766 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I16ab253ab99808c2a69b0520a3a6b6253f27f2f8 Gerrit-Change-Number: 16766 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] IGNORE: A small patch to test dist test
Grant Henke has removed a vote on this change. Change subject: IGNORE: A small patch to test dist_test .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16754 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I133627195155eaab7939b2822f5bb60f33da13c5 Gerrit-Change-Number: 16754 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] IGNORE: A small patch to test dist test
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16754 Change subject: IGNORE: A small patch to test dist_test .. IGNORE: A small patch to test dist_test Change-Id: I133627195155eaab7939b2822f5bb60f33da13c5 --- M CMakeLists.txt 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/54/16754/1 -- To view, visit http://gerrit.cloudera.org:8080/16754 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I133627195155eaab7939b2822f5bb60f33da13c5 Gerrit-Change-Number: 16754 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [build] Fix LLVM build on Xcode 12.2
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16746 ) Change subject: [build] Fix LLVM build on Xcode 12.2 .. [build] Fix LLVM build on Xcode 12.2 When building with the latest Xcode version (12.2) I found that the LLVM build failed to build `sanitizer_platform_limits_posix.cpp` due to the `stat64` type missing. Given we don’t need the sanitizers, I disabled building them to work around the issue. Example error message: kudu/thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc:169:31: error: invalid application of 'sizeof' to an incomplete type 'struct stat64' unsigned struct_stat64_sz = sizeof(struct stat64); Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 Reviewed-on: http://gerrit.cloudera.org:8080/16746 Reviewed-by: Alexey Serbin Reviewed-by: Andrew Wong Tested-by: Kudu Jenkins --- M thirdparty/build-definitions.sh 1 file changed, 9 insertions(+), 0 deletions(-) Approvals: Alexey Serbin: Looks good to me, but someone else must approve Andrew Wong: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 Gerrit-Change-Number: 16746 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy
[kudu-CR] [build] Fix LLVM build on Xcode 12.2
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16746 ) Change subject: [build] Fix LLVM build on Xcode 12.2 .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/16746/2/thirdparty/build-definitions.sh File thirdparty/build-definitions.sh: http://gerrit.cloudera.org:8080/#/c/16746/2/thirdparty/build-definitions.sh@287 PS2, Line 287: # Xcode 12.2 fails to buld the sanitzers and they are not needed. > nit: sanitizers Done http://gerrit.cloudera.org:8080/#/c/16746/2/thirdparty/build-definitions.sh@290 PS2, Line 290: sanitzer_common > nit: sanitizer here too Done http://gerrit.cloudera.org:8080/#/c/16746/2/thirdparty/build-definitions.sh@290 PS2, Line 290: sanitzer_common > nit: sanitizer_common ? Done http://gerrit.cloudera.org:8080/#/c/16746/2/thirdparty/build-definitions.sh@291 PS2, Line 291: github > Curious whether you tried disabling COMPILER_RT_HAS_SANITIZER_COMMON? Or wo That isn't settable, it is set earlier in the LLVM build. -- To view, visit http://gerrit.cloudera.org:8080/16746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 Gerrit-Change-Number: 16746 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Comment-Date: Wed, 18 Nov 2020 23:58:54 + Gerrit-HasComments: Yes
[kudu-CR] [build] Fix LLVM build on Xcode 12.2
Hello Mahesh Reddy, Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16746 to look at the new patch set (#3). Change subject: [build] Fix LLVM build on Xcode 12.2 .. [build] Fix LLVM build on Xcode 12.2 When building with the latest Xcode version (12.2) I found that the LLVM build failed to build `sanitizer_platform_limits_posix.cpp` due to the `stat64` type missing. Given we don’t need the sanitizers, I disabled building them to work around the issue. Example error message: kudu/thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc:169:31: error: invalid application of 'sizeof' to an incomplete type 'struct stat64' unsigned struct_stat64_sz = sizeof(struct stat64); Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 --- M thirdparty/build-definitions.sh 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/46/16746/3 -- To view, visit http://gerrit.cloudera.org:8080/16746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 Gerrit-Change-Number: 16746 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy
[kudu-CR] [build] Fix LLVM build on Xcode 12.2
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16746 to look at the new patch set (#2). Change subject: [build] Fix LLVM build on Xcode 12.2 .. [build] Fix LLVM build on Xcode 12.2 When building with the latest Xcode version (12.2) I found that the LLVM build failed to build `sanitizer_platform_limits_posix.cpp` due to the `stat64` type missing. Given we don’t need the sanitizers, I disabled building them to work around the issue. Example error message: kudu/thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc:169:31: error: invalid application of 'sizeof' to an incomplete type 'struct stat64' unsigned struct_stat64_sz = sizeof(struct stat64); Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 --- M thirdparty/build-definitions.sh 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/46/16746/2 -- To view, visit http://gerrit.cloudera.org:8080/16746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 Gerrit-Change-Number: 16746 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [build] Fix LLVM build on Xcode 12.2
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16746 Change subject: [build] Fix LLVM build on Xcode 12.2 .. [build] Fix LLVM build on Xcode 12.2 When building with the latest Xcode version (12.2) I found thatL the LVVM build failed to build sanitizer_platform_limits_posix.cpp due to the stats64 type missing. Given we don’t need the sanitizers, I disabled building them to work around the issue. Example error message: kudu/thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc:169:31: error: invalid application of 'sizeof' to an incomplete type 'struct stat64' unsigned struct_stat64_sz = sizeof(struct stat64); Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 --- M thirdparty/build-definitions.sh 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/46/16746/1 -- To view, visit http://gerrit.cloudera.org:8080/16746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I17ea2de460aac4142469c95af201b17554c4f751 Gerrit-Change-Number: 16746 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] Revert "KUDU-3210 Add lock before verifying signature"
Grant Henke has uploaded a new patch set (#3) to the change originally created by Attila Bukor. ( http://gerrit.cloudera.org:8080/16731 ) Change subject: Revert "KUDU-3210 Add lock before verifying signature" .. Revert "KUDU-3210 Add lock before verifying signature" This reverts commit f9f3189a6dbe0636d578d80b1d8e60cf7b2e6686. Change-Id: I02daa5ca95de697714d5e893f8bc813bdc9530d3 --- M src/kudu/security/CMakeLists.txt M src/kudu/security/crypto.cc M src/kudu/security/openssl_util.cc M src/kudu/security/openssl_util.h M src/kudu/security/tls_context.cc M src/kudu/security/tls_handshake.cc 6 files changed, 13 insertions(+), 96 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/16731/3 -- To view, visit http://gerrit.cloudera.org:8080/16731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I02daa5ca95de697714d5e893f8bc813bdc9530d3 Gerrit-Change-Number: 16731 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou
[kudu-CR] Revert "KUDU-3210 Add lock before verifying signature"
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16731 ) Change subject: Revert "KUDU-3210 Add lock before verifying signature" .. Revert "KUDU-3210 Add lock before verifying signature" This reverts commit f9f3189a6dbe0636d578d80b1d8e60cf7b2e6686. Change-Id: I02daa5ca95de697714d5e893f8bc813bdc9530d3 Reviewed-on: http://gerrit.cloudera.org:8080/16731 Reviewed-by: Grant Henke Reviewed-by: Alexey Serbin Tested-by: Kudu Jenkins --- M src/kudu/security/CMakeLists.txt M src/kudu/security/crypto.cc M src/kudu/security/openssl_util.cc M src/kudu/security/openssl_util.h M src/kudu/security/tls_context.cc M src/kudu/security/tls_handshake.cc 6 files changed, 13 insertions(+), 96 deletions(-) Approvals: Grant Henke: Looks good to me, approved Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I02daa5ca95de697714d5e893f8bc813bdc9530d3 Gerrit-Change-Number: 16731 Gerrit-PatchSet: 4 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou
[kudu-CR] KUDU-3210 Add thread ID callback to OpenSSL init
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16730 ) Change subject: KUDU-3210 Add thread ID callback to OpenSSL init .. KUDU-3210 Add thread ID callback to OpenSSL init It seems the race condition bug we worked around in f9f3189a6dbe0636d578d80b1d8e60cf7b2e6686 was caused by using the default thread ID callback. It seems it's not a bug in SafeLogic after all, but this is likely reproducible in upstream OpenSSL as well. We didn't find this before as we always tested in older OpenSSL versions, while the commit[1] responsible for this behavior was included only in OpenSSL 1.0.2i[2]. The threads(3) man page claims that "If the application does not register such a callback using CRYPTO_THREADID_set_callback(), then a default implementation is used - on Windows and BeOS this uses the system's default thread identifying APIs, and on all other platforms it uses the address of errno. The latter is satisfactory for thread-safety if and only if the platform has a thread-local error number facility." This seems to be no longer true in 1.0.2i and later. Redefining the thread ID callback seems to fix the problem without any additional locking and f9f3189a6dbe0636d578d80b1d8e60cf7b2e6686 can be reverted safely. I tested these changes on the host I discovered the race condition. [1] https://github.com/openssl/openssl/commit/a43cfd7bb1fc681d563e5efa75cc926d7e8e5c36 [2] https://mta.openssl.org/pipermail/openssl-commits/2016-September/010743.html Change-Id: Icec6da3a9380206fe6ba4a31ea8fb4dcbc34dd00 Reviewed-on: http://gerrit.cloudera.org:8080/16730 Reviewed-by: Grant Henke Reviewed-by: Alexey Serbin Tested-by: Kudu Jenkins --- M src/kudu/security/openssl_util.cc 1 file changed, 9 insertions(+), 0 deletions(-) Approvals: Grant Henke: Looks good to me, but someone else must approve Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icec6da3a9380206fe6ba4a31ea8fb4dcbc34dd00 Gerrit-Change-Number: 16730 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou
[kudu-CR] KUDU-3210 Add thread ID callback to OpenSSL init
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16730 ) Change subject: KUDU-3210 Add thread ID callback to OpenSSL init .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/16730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icec6da3a9380206fe6ba4a31ea8fb4dcbc34dd00 Gerrit-Change-Number: 16730 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 16 Nov 2020 22:46:35 + Gerrit-HasComments: No
[kudu-CR] KUDU-3210 Add thread ID callback to OpenSSL init
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16730 ) Change subject: KUDU-3210 Add thread ID callback to OpenSSL init .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/16730/2/src/kudu/security/openssl_util.cc File src/kudu/security/openssl_util.cc: http://gerrit.cloudera.org:8080/#/c/16730/2/src/kudu/security/openssl_util.cc@53 PS2, Line 53: #if OPENSSL_VERSION_NUMBER < 0x1010L > Why is this based on openssl version? Oh, nevermind. I missed the matching check below. -- To view, visit http://gerrit.cloudera.org:8080/16730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icec6da3a9380206fe6ba4a31ea8fb4dcbc34dd00 Gerrit-Change-Number: 16730 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 16 Nov 2020 22:46:02 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3210 Add thread ID callback to OpenSSL init
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16730 ) Change subject: KUDU-3210 Add thread ID callback to OpenSSL init .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/16730/2/src/kudu/security/openssl_util.cc File src/kudu/security/openssl_util.cc: http://gerrit.cloudera.org:8080/#/c/16730/2/src/kudu/security/openssl_util.cc@53 PS2, Line 53: #if OPENSSL_VERSION_NUMBER < 0x1010L Why is this based on openssl version? -- To view, visit http://gerrit.cloudera.org:8080/16730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icec6da3a9380206fe6ba4a31ea8fb4dcbc34dd00 Gerrit-Change-Number: 16730 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 16 Nov 2020 22:43:35 + Gerrit-HasComments: Yes
[kudu-CR] Revert "KUDU-3210 Add lock before verifying signature"
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16731 ) Change subject: Revert "KUDU-3210 Add lock before verifying signature" .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I02daa5ca95de697714d5e893f8bc813bdc9530d3 Gerrit-Change-Number: 16731 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 16 Nov 2020 22:41:48 + Gerrit-HasComments: No
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. [dist_test] Ship security libraries to dist_test This patch adjusts the lib whitelist to allow shipping the security libraries to dist_test and allow more flexibility when versions do not match the dist_test images versions. This was already happening for rhel6 installs due to the rhel6 workaround script linked below. With this change the libraries will be shipped even when not in thirdparty. https://github.com/apache/kudu/blob/master/thirdparty/install-openssl-el6-workaround.sh I also needed to adjust run_dist_test.py in order to set the SASL_PATH environment variable if SASL modules are present. Otherwise the system modules were still used. Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Reviewed-on: http://gerrit.cloudera.org:8080/16716 Tested-by: Kudu Jenkins Reviewed-by: Andrew Wong Reviewed-by: Alexey Serbin --- M build-support/dist_test.py M build-support/run_dist_test.py M cmake_modules/FindCyrusSASL.cmake 3 files changed, 68 insertions(+), 3 deletions(-) Approvals: Kudu Jenkins: Verified Andrew Wong: Looks good to me, approved Alexey Serbin: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 7 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist test] Ship security libraries to dist test
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16716 to look at the new patch set (#6). Change subject: [dist_test] Ship security libraries to dist_test .. [dist_test] Ship security libraries to dist_test This patch adjusts the lib whitelist to allow shipping the security libraries to dist_test and allow more flexibility when versions do not match the dist_test images versions. This was already happening for rhel6 installs due to the rhel6 workaround script linked below. With this change the libraries will be shipped even when not in thirdparty. https://github.com/apache/kudu/blob/master/thirdparty/install-openssl-el6-workaround.sh I also needed to adjust run_dist_test.py in order to set the SASL_PATH environment variable if SASL modules are present. Otherwise the system modules were still used. Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 --- M build-support/dist_test.py M build-support/run_dist_test.py M cmake_modules/FindCyrusSASL.cmake 3 files changed, 68 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/16716/6 -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/16716/5/build-support/dist_test.py File build-support/dist_test.py: http://gerrit.cloudera.org:8080/#/c/16716/5/build-support/dist_test.py@274 PS5, Line 274: def add_sasl_module_deps(deps): > Would you mind commenting why sasl needs special treatment here? Done -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sat, 14 Nov 2020 04:50:00 + Gerrit-HasComments: Yes
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. Patch Set 5: With patch set 5 I was able to submit tests from Centos7 against dist_test (Running ubuntu 14). I am running full suites now to validate further. -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sat, 14 Nov 2020 03:44:29 + Gerrit-HasComments: No
[kudu-CR] [dist test] Ship security libraries to dist test
Hello Alexey Serbin, Kudu Jenkins, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16716 to look at the new patch set (#5). Change subject: [dist_test] Ship security libraries to dist_test .. [dist_test] Ship security libraries to dist_test This patch adjusts the lib whitelist to allow shipping the security libraries to dist_test and allow more flexibility when versions do not match the dist_test images versions. This was already happening for rhel6 installs due to the rhel6 workaround script linked below. With this change the libraries will be shipped even when not in thirdparty. https://github.com/apache/kudu/blob/master/thirdparty/install-openssl-el6-workaround.sh I also needed to adjust run_dist_test.py in order to set the SASL_PATH environment variable if SASL modules are present. Otherwise the system modules were still used. Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 --- M build-support/dist_test.py M build-support/run_dist_test.py M cmake_modules/FindCyrusSASL.cmake 3 files changed, 62 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/16716/5 -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/16716/4/build-support/dist_test.py File build-support/dist_test.py: http://gerrit.cloudera.org:8080/#/c/16716/4/build-support/dist_test.py@709 PS4, Line 709: print(json.dumps(relocated_deps)) > Only the relocated_deps are now dumped compared to all earlier, is this int They are all appended to relocated_deps. However, only the system libraries are actually changed. -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sat, 14 Nov 2020 00:20:15 + Gerrit-HasComments: Yes
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has removed a vote on this change. Change subject: [dist_test] Ship security libraries to dist_test .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 13 Nov 2020 23:48:39 + Gerrit-HasComments: No
[kudu-CR] [dist test] Ship security libraries to dist test
Hello Alexey Serbin, Kudu Jenkins, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16716 to look at the new patch set (#4). Change subject: [dist_test] Ship security libraries to dist_test .. [dist_test] Ship security libraries to dist_test This patch adjusts the lib whitelist to allow shipping the security libraries to dist_test and allow more flexibility when versions do not match the dist_test images versions. This was already happening for rhel6 installs due to the rhel6 workaround script linked below. With this change the libraries will be shipped even when not in thirdparty. https://github.com/apache/kudu/blob/master/thirdparty/install-openssl-el6-workaround.sh Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 --- M build-support/dist_test.py M cmake_modules/FindCyrusSASL.cmake 2 files changed, 44 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/16716/4 -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. Patch Set 3: Yeah, the motivation is not to guarantee that submitting from other OS versions will work, but to at least reduce the cases where it fails when system library versions don't align. Primarily to simplify upgrading the dist_test os version. Right now dist_test runs on ubuntu 14 and we can submit from ubuntu 14 and rhel 6. I am hoping that submitting from centos 7 is something that can be workable at some point. -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 13 Nov 2020 23:15:02 + Gerrit-HasComments: No
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/16716/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16716/1//COMMIT_MSG@13 PS1, Line 13: 6 workaro > nit: workaround Done http://gerrit.cloudera.org:8080/#/c/16716/1//COMMIT_MSG@13 PS1, Line 13: rhe > nit: rhel6 here as well? Done -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 13 Nov 2020 19:55:32 + Gerrit-HasComments: Yes
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16716 ) Change subject: [dist_test] Ship security libraries to dist_test .. Patch Set 3: I need to figure out why this is failing all the java tests -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 13 Nov 2020 19:55:44 + Gerrit-HasComments: No
[kudu-CR] [dist test] Ship security libraries to dist test
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16716 to look at the new patch set (#3). Change subject: [dist_test] Ship security libraries to dist_test .. [dist_test] Ship security libraries to dist_test This patch adjusts the lib whitelist to allow shipping the security libraries to dist_test and allow more flexibility when versions do not match the dist_test images versions. This was already happening for rhel6 installs due to the rhel6 workaround script linked below. With this change the libraries will be shipped even when not in thirdparty. https://github.com/apache/kudu/blob/master/thirdparty/install-openssl-el6-workaround.sh Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 --- M build-support/dist_test.py M cmake_modules/FindCyrusSASL.cmake 2 files changed, 36 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/16716/3 -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist test] Ship security libraries to dist test
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16716 to look at the new patch set (#2). Change subject: [dist_test] Ship security libraries to dist_test .. [dist_test] Ship security libraries to dist_test This patch adjusts the lib whitelist to allow shipping the security libraries to dist_test and allow more flexibility when versions do not match the dist_test images versions. This was already happening for rhel6 installs due to the rhel6 workaround script linked below. With this change the libraries will be shipped even when not in thirdparty. https://github.com/apache/kudu/blob/master/thirdparty/install-openssl-el6-workaround.sh Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 --- M build-support/dist_test.py M cmake_modules/FindCyrusSASL.cmake 2 files changed, 38 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/16716/2 -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist test] Ship security libraries to dist test
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16716 Change subject: [dist_test] Ship security libraries to dist_test .. [dist_test] Ship security libraries to dist_test This patch adjusts the lib whitelist to allow shipping the security libraries to dist_test and allow more flexibility when versions do not match the dist_test images versions. This was already happening for rhel6 installs due to the el6 workaroud script linked below. With this change the libraries will be shipped even when not in thirdparty. https://github.com/apache/kudu/blob/master/thirdparty/install-openssl-el6-workaround.sh Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 --- M build-support/dist_test.py 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/16716/1 -- To view, visit http://gerrit.cloudera.org:8080/16716 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id10afab6e9c48b9ffcf0da905993c7f2a1e606a6 Gerrit-Change-Number: 16716 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] docs: update known issues based on recent features
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16713 ) Change subject: docs: update known issues based on recent features .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16713 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57a7ae753f541f06cdbf8990b8e9717a37486e41 Gerrit-Change-Number: 16713 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 12 Nov 2020 18:10:29 + Gerrit-HasComments: No
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16681 ) Change subject: KUDU-1563 Support ignore operations in kudu-spark .. KUDU-1563 Support ignore operations in kudu-spark This patch adds support for the INSERT_IGNORE, UPDATE_IGNORE, and DELETE_IGNORE operations into the Kudu Spark integration. It leverages `AsyncKuduClient.supportsIgnoreOperations()` to handle INSERT_IGNORE operations in a compatible way. Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Reviewed-on: http://gerrit.cloudera.org:8080/16681 Reviewed-by: Attila Bukor Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala 4 files changed, 83 insertions(+), 5 deletions(-) Approvals: Attila Bukor: Looks good to me, approved Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Gerrit-Change-Number: 16681 Gerrit-PatchSet: 7 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16683 ) Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 09 Nov 2020 15:07:34 + Gerrit-HasComments: No
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16683 ) Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. KUDU-1563 Use DELETE_IGNORE in KuduRestore job This patch changes the KuduRestore job to use DELETE_IGNORE operations instead of DELETE when the cluster supports ignore operations. `session.setIgnoreAllNotFoundRows(true)` is retained to support falling back to DELETE operations for backward compatibility. Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Reviewed-on: http://gerrit.cloudera.org:8080/16683 Reviewed-by: Attila Bukor Reviewed-by: Alexey Serbin Tested-by: Grant Henke --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 2 files changed, 26 insertions(+), 4 deletions(-) Approvals: Attila Bukor: Looks good to me, approved Alexey Serbin: Looks good to me, approved Grant Henke: Verified -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 7 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Grant Henke has removed a vote on this change. Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16683 to look at the new patch set (#6). Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. KUDU-1563 Use DELETE_IGNORE in KuduRestore job This patch changes the KuduRestore job to use DELETE_IGNORE operations instead of DELETE when the cluster supports ignore operations. `session.setIgnoreAllNotFoundRows(true)` is retained to support falling back to DELETE operations for backward compatibility. Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 2 files changed, 26 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/16683/6 -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16681 ) Change subject: KUDU-1563 Support ignore operations in kudu-spark .. Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/16681/5/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala: http://gerrit.cloudera.org:8080/#/c/16681/5/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala@247 PS5, Line 247: case "insert_ignore" => InsertIgnore > nit: can you make the order of underscore and hyphen versions consistent? Done http://gerrit.cloudera.org:8080/#/c/16681/5/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/16681/5/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@486 PS5, Line 486: "kudu.ignoreDuplicateRowErrors is deprecated and slow. Use the insert_ignore operation instead.") > should we emit this warning even if it is not supported in the cluster? "ignoreDuplicateRowErrors" is the old style and is always "supported". If you mean should we emit this when ignore ops aren't supported, I think we should since it indicates to the user that the old style is being used. -- To view, visit http://gerrit.cloudera.org:8080/16681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Gerrit-Change-Number: 16681 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 09 Nov 2020 14:06:51 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16681 to look at the new patch set (#6). Change subject: KUDU-1563 Support ignore operations in kudu-spark .. KUDU-1563 Support ignore operations in kudu-spark This patch adds support for the INSERT_IGNORE, UPDATE_IGNORE, and DELETE_IGNORE operations into the Kudu Spark integration. It leverages `AsyncKuduClient.supportsIgnoreOperations()` to handle INSERT_IGNORE operations in a compatible way. Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala 4 files changed, 83 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/16681/6 -- To view, visit http://gerrit.cloudera.org:8080/16681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Gerrit-Change-Number: 16681 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16698 ) Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. KUDU-1563. Add a feature flag for IGNORE operations This patch adds a master server feature flag to indicate that the cluster supports `IGNORE` operations. This includes INSERT_IGNORE, DELETE_IGNORE, and UPDATE_IGNORE. Though this is technically a tserver feature, it is unreasonable to check if every tablet server supports this feature. Instead we use the master as a proxy. In the future KUDU-3211 will add more complete cluster feature flag support. Additionally this patch leverages the feature flag in the Java client on a PingRequest to implement a `supportsIgnoreOperations()` method. This functionality will be used in follow on patches to add ignore operation support to the Spark and Backup integrations in a compatible way. Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Reviewed-on: http://gerrit.cloudera.org:8080/16698 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin Reviewed-by: Attila Bukor --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M src/kudu/master/master.proto M src/kudu/master/master_service.cc 6 files changed, 104 insertions(+), 2 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved Attila Bukor: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Fix build when using default base OS
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16694 ) Change subject: [docker] Fix build when using default base OS .. [docker] Fix build when using default base OS When using the default base OS the string is not utf8 type and `isnumeric` is not defined. This small fix changes to `isdigit` which works with all python strings. Change-Id: I81069970caabce1c7b9e0cdabaee403fdf921bad Reviewed-on: http://gerrit.cloudera.org:8080/16694 Tested-by: Kudu Jenkins Reviewed-by: Attila Bukor --- M docker/docker-build.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Kudu Jenkins: Verified Attila Bukor: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16694 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I81069970caabce1c7b9e0cdabaee403fdf921bad Gerrit-Change-Number: 16694 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Fix build when using default base OS
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16694 ) Change subject: [docker] Fix build when using default base OS .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16694/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16694/1//COMMIT_MSG@9 PS1, Line 9: not utf8 type > just curious, why is it not utf8 type? Python string constants aren't utf-8 by default. -- To view, visit http://gerrit.cloudera.org:8080/16694 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I81069970caabce1c7b9e0cdabaee403fdf921bad Gerrit-Change-Number: 16694 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 09 Nov 2020 14:01:48 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16681 ) Change subject: KUDU-1563 Support ignore operations in kudu-spark .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/16681/4/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/16681/4/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@159 PS4, Line 159: transient lazy val supportsIgnoreOperations > Does the '@transient' attribute mean that every worker calls this since thi This is only called on the driver node before the tasks are created. If it were called inside `rdd.foreachPartition` in the writeRows method, then it would be on each worker. -- To view, visit http://gerrit.cloudera.org:8080/16681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Gerrit-Change-Number: 16681 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sat, 07 Nov 2020 20:26:46 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Grant Henke has removed a vote on this change. Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/16683 )
Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job
..
Patch Set 5: Verified+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16683/3/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala
File
java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/16683/3/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala@594
PS3, Line 594: def testDeleteIgnore(): Unit = {
> nit: similar feedback to the previous patch -- might be nice to add a hint
Done
--
To view, visit http://gerrit.cloudera.org:8080/16683
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22
Gerrit-Change-Number: 16683
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 07 Nov 2020 18:58:11 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16698 to look at the new patch set (#5). Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. KUDU-1563. Add a feature flag for IGNORE operations This patch adds a master server feature flag to indicate that the cluster supports `IGNORE` operations. This includes INSERT_IGNORE, DELETE_IGNORE, and UPDATE_IGNORE. Though this is technically a tserver feature, it is unreasonable to check if every tablet server supports this feature. Instead we use the master as a proxy. In the future KUDU-3211 will add more complete cluster feature flag support. Additionally this patch leverages the feature flag in the Java client on a PingRequest to implement a `supportsIgnoreOperations()` method. This functionality will be used in follow on patches to add ignore operation support to the Spark and Backup integrations in a compatible way. Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M src/kudu/master/master.proto M src/kudu/master/master_service.cc 6 files changed, 104 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/16698/5 -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16698 ) Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/16698/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16698/4//COMMIT_MSG@17 PS4, Line 17: Additionality > Additionally Done http://gerrit.cloudera.org:8080/#/c/16698/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java: http://gerrit.cloudera.org:8080/#/c/16698/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@2647 PS3, Line 2647: @return true if the cluster supports ignore operations > One more thought: since the information about the status of the IGNORE oper Done -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sat, 07 Nov 2020 18:00:28 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/16698 )
Change subject: KUDU-1563. Add a feature flag for IGNORE operations
..
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/16698/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/16698/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@2662
PS3, Line 2662: // The server returns an RpcRemoteException when
: // the required feature is not supported.
: if (resp instanceof RpcRemoteException) {
: return false;
: }
> Does it make sense to be more specific here? As I understand, the server s
Done
http://gerrit.cloudera.org:8080/#/c/16698/3/java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java
File java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java:
http://gerrit.cloudera.org:8080/#/c/16698/3/java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java@43
PS3, Line 43: makeMasterPingRequest
> I don't have good understanding of exact workflow here, but would it be mor
It's not something that is done or required at connection time or could even be
set at connection time. If you look at the follow on patches it is only used in
certain use cases and runtime for an application.
http://gerrit.cloudera.org:8080/#/c/16698/3/src/kudu/master/master_service.cc
File src/kudu/master/master_service.cc:
http://gerrit.cloudera.org:8080/#/c/16698/3/src/kudu/master/master_service.cc@106
PS3, Line 106: TAG_FLAG
> nit: does it make sense to add the 'runtime' tag as well?
Done
--
To view, visit http://gerrit.cloudera.org:8080/16698
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a
Gerrit-Change-Number: 16698
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 07 Nov 2020 03:33:42 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16698 to look at the new patch set (#4). Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. KUDU-1563. Add a feature flag for IGNORE operations This patch adds a master server feature flag to indicate that the cluster supports `IGNORE` operations. This includes INSERT_IGNORE, DELETE_IGNORE, and UPDATE_IGNORE. Though this is technically a tserver feature, it is unreasonable to check if every tablet server supports this feature. Instead we use the master as a proxy. In the future KUDU-3211 will add more complete cluster feature flag support. Additionality this patch leverages the feature flag in the Java client on a PingRequest to implement a `supportsIgnoreOperations()` method. This functionality will be used in follow on patches to add ignore operation support to the Spark and Backup integrations in a compatible way. Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M src/kudu/master/master.proto M src/kudu/master/master_service.cc 6 files changed, 102 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/16698/4 -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16698 ) Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 06 Nov 2020 14:39:45 + Gerrit-HasComments: No
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Grant Henke has removed a vote on this change. Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16683 to look at the new patch set (#4). Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. KUDU-1563 Use DELETE_IGNORE in KuduRestore job This patch changes the KuduRestore job to use DELETE_IGNORE operations instead of DELETE when the cluster supports ignore operations. `session.setIgnoreAllNotFoundRows(true)` is retained to support falling back to DELETE operations for backward compatibility. Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 2 files changed, 52 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/16683/4 -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/16681 )
Change subject: KUDU-1563 Support ignore operations in kudu-spark
..
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16681/3/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
File
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala:
http://gerrit.cloudera.org:8080/#/c/16681/3/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala@289
PS3, Line 289: def testInsertIgnoreRowsWriteOption() {
> nit: maybe add a hint to future readers like, "Identical to the above test,
I did this but swapped it so that the legacy test is second.
--
To view, visit http://gerrit.cloudera.org:8080/16681
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4
Gerrit-Change-Number: 16681
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 06 Nov 2020 13:57:43 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16681 to look at the new patch set (#4). Change subject: KUDU-1563 Support ignore operations in kudu-spark .. KUDU-1563 Support ignore operations in kudu-spark This patch adds support for the INSERT_IGNORE, UPDATE_IGNORE, and DELETE_IGNORE operations into the Kudu Spark integration. It leverages `AsyncKuduClient.supportsIgnoreOperations()` to handle INSERT_IGNORE operations in a compatible way. Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala 4 files changed, 83 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/16681/4 -- To view, visit http://gerrit.cloudera.org:8080/16681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Gerrit-Change-Number: 16681 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/16698 )
Change subject: KUDU-1563. Add a feature flag for IGNORE operations
..
Patch Set 2:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/16698/2/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/16698/2/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@2666
PS2, Line 2666: else {
> nit: don't need to 'else' after a return
Done
http://gerrit.cloudera.org:8080/#/c/16698/2/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@2667
PS2, Line 2667: new IllegalStateException((Exception) resp);
> nit: why not just throw 'resp'?
I need to throw an unchecked exception.
http://gerrit.cloudera.org:8080/#/c/16698/2/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@2675
PS2, Line 2675: feature
> nit: it'll be obvious to us because there's only one so far, but for non-de
Done
http://gerrit.cloudera.org:8080/#/c/16698/2/java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java
File java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java:
http://gerrit.cloudera.org:8080/#/c/16698/2/java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java@46
PS2, Line 46: static PingRequest makeMasterPingRequest(KuduTable masterTable,
Timer timer, long timeoutMillis) {
: return new PingRequest(masterTable, MASTER_SERVICE_NAME,
timer, timeoutMillis);
> Just checking based on the other files -- were these constructors necessary
Yes. Without passing the masterTable, sendRpcToTablet doesn't know how to route
the request to the correct master server.
http://gerrit.cloudera.org:8080/#/c/16698/2/src/kudu/master/master_service.cc
File src/kudu/master/master_service.cc:
http://gerrit.cloudera.org:8080/#/c/16698/2/src/kudu/master/master_service.cc@107
PS2, Line 107: TAG_FLAG(master_support_ignore_operations, unsafe);
> nit: probably doesn't need to be unsafe -- pretty sure the writes will erro
Done
--
To view, visit http://gerrit.cloudera.org:8080/16698
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a
Gerrit-Change-Number: 16698
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 06 Nov 2020 13:47:40 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16698 to look at the new patch set (#3). Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. KUDU-1563. Add a feature flag for IGNORE operations This patch adds a master server feature flag to indicate that the cluster supports `IGNORE` operations. This includes INSERT_IGNORE, DELETE_IGNORE, and UPDATE_IGNORE. Though this is technically a tserver feature, it is unreasonable to check if every tablet server supports this feature. Instead we use the master as a proxy. In the future KUDU-3211 will add more complete cluster feature flag support. Additionality this patch leverages the feature flag in the Java client on a PingRequest to implement a `supportsIgnoreOperations()` method. This functionality will be used in follow on patches to add ignore operation support to the Spark and Backup integrations in a compatible way. Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M src/kudu/master/master.proto M src/kudu/master/master_service.cc 6 files changed, 96 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/16698/3 -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16683 to look at the new patch set (#3). Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. KUDU-1563 Use DELETE_IGNORE in KuduRestore job This patch changes the KuduRestore job to use DELETE_IGNORE operations instead of DELETE when the cluster supports ignore operations. `session.setIgnoreAllNotFoundRows(true)` is retained to support falling back to DELETE operations for backward compatibility. Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 2 files changed, 48 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/16683/3 -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Use DELETE IGNORE in KuduRestore job
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16683 to look at the new patch set (#2). Change subject: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. KUDU-1563 Use DELETE_IGNORE in KuduRestore job This patch changes the KuduRestore job to use DELETE_IGNORE operations instead of DELETE when the cluster supports ignore operations. `session.setIgnoreAllNotFoundRows(true)` is retained to support falling back to DELETE operations for backward compatibility. Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala 1 file changed, 10 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/16683/2 -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16681 ) Change subject: KUDU-1563 Support ignore operations in kudu-spark .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/16681/2/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala File java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala: http://gerrit.cloudera.org:8080/#/c/16681/2/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala@266 PS2, Line 266: ess.getMaster > There is also `insert_ignore` at line 297. What's the difference? There is no difference. I made the hyphen and underscore interchangeable in the Spark properties like Kudu's other flags/configs. Testing using the string literal ensures we don't change a constant and break existing applications. -- To view, visit http://gerrit.cloudera.org:8080/16681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Gerrit-Change-Number: 16681 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 06 Nov 2020 05:15:20 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563 Support ignore operations in kudu-spark
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16681 to look at the new patch set (#3). Change subject: KUDU-1563 Support ignore operations in kudu-spark .. KUDU-1563 Support ignore operations in kudu-spark This patch adds support for the INSERT_IGNORE, UPDATE_IGNORE, and DELETE_IGNORE operations into the Kudu Spark integration. It leverages `AsyncKuduClient.supportsIgnoreOperations()` to handle INSERT_IGNORE operations in a compatible way. Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala 4 files changed, 79 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/16681/3 -- To view, visit http://gerrit.cloudera.org:8080/16681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If4b4dc0ec996a88afead0f9da0024457e568b0f4 Gerrit-Change-Number: 16681 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16698 to look at the new patch set (#2). Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. KUDU-1563. Add a feature flag for IGNORE operations This patch adds a master server feature flag to indicate that the cluster supports `IGNORE` operations. This includes INSERT_IGNORE, DELETE_IGNORE, and UPDATE_IGNORE. Though this is technically a tserver feature, it is unreasonable to check if every tablet server supports this feature. Instead we use the master as a proxy. In the future KUDU-3211 will add more complete cluster feature flag support. Additionality this patch leverages the feature flag in the Java client on a PingRequest to implement a `supportsIgnoreOperations()` method. This functionality will be used in follow on patches to add ignore operation support to the Spark and Backup integrations in a compatible way. Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M src/kudu/master/master.proto M src/kudu/master/master_service.cc 6 files changed, 98 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/16698/2 -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563. Add a feature flag for IGNORE operations
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16698 Change subject: KUDU-1563. Add a feature flag for IGNORE operations .. KUDU-1563. Add a feature flag for IGNORE operations This patch adds a master server feature flag to indicate that the cluster supports `IGNORE` operations. This includes INSERT_IGNORE, DELETE_IGNORE, and UPDATE_IGNORE. Though this is technically a tserver feature, it is unreasonable to check if every tablet server supports this feature. Instead we use the master as a proxy. In the future KUDU-3211 will add more complete cluster feature flag support. Additionality this patch leverages the feature flag in the Java client on a PingRequest to implement a `supportsIgnoreOperations()` method. This functionality will be used in follow on patches to add ignore operation support to the Spark and Backup integrations in a compatible way. Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/PingRequest.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M src/kudu/master/master.proto M src/kudu/master/master_service.cc 5 files changed, 83 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/16698/1 -- To view, visit http://gerrit.cloudera.org:8080/16698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I329bd8bde73d247240ae597b677e2cc20a92343a Gerrit-Change-Number: 16698 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [examples] Update examples dependencies
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16695 ) Change subject: [examples] Update examples dependencies .. [examples] Update examples dependencies This patch updates the examples builds to use more current dependencies. Change-Id: I74aa9ea4375d3aee9b08122b8568ca4d45c686ec Reviewed-on: http://gerrit.cloudera.org:8080/16695 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M examples/java/collectl/pom.xml M examples/java/insert-loadgen/pom.xml M examples/java/java-example/pom.xml M examples/scala/sbt-int-test-example/build.sbt M examples/scala/spark-example/pom.xml 5 files changed, 27 insertions(+), 27 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16695 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I74aa9ea4375d3aee9b08122b8568ca4d45c686ec Gerrit-Change-Number: 16695 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3210 Add lock before verifying signature
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16659 ) Change subject: KUDU-3210 Add lock before verifying signature .. KUDU-3210 Add lock before verifying signature It seems there is a race condition somewhere in OpenSSL FIPS Object Module 2.0, or at least in SafeLogic CryptoComply for Servers 1.0.2v-fips, as a certificate can get corrupted when multiple certificates are being verified in the same time, throwing an error during TLS handshake similar to this: error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01:rsa_pk1.c:102 error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed:rsa_eay.c:786 error:1408D07B:SSL routines:ssl3_get_key_exchange:bad signature:s3_clnt.c:2038 This means OpenSSL is trying to verify a signature with a public key that doesn't match the private key, resulting in an invalid message. According to a mailing list thread[1] this can happen in multi-threaded applications, such as ours, even though we seem to have proper locking in our verify signature function and the OpenSSL locking callbacks are properly registered too. Unfortunately, OpenSSL 1.0.2 doesn't guarantee complete thread safety[2]: "OpenSSL can generally be used safely in multi-threaded applications provided that at least two callback functions are set, the locking_function and threadid_func. Note that OpenSSL is not completely thread-safe, and unfortunately not all global resources have the necessary locks. Further, the thread-safety does not extend to things like multiple threads using the same SSL object at the same time." In addition to the TLS negotiation this can also get triggered when verifying authn token signatures, further suggesting a race condition in the underlying OpenSSL library. This commit adds additional locking conditionally depending on a hidden runtime flag "openssl_defensive_locking" to crypto and TLS context/handshake which seems to prevent this from happening in both places. [1] https://groups.google.com/u/1/g/mailing.openssl.users/c/u0JyAuogrc0 [2] https://www.openssl.org/docs/man1.0.2/man3/threads.html Change-Id: Ifafc7dcf91db910123276b657515e410bb7f6fcd Reviewed-on: http://gerrit.cloudera.org:8080/16659 Reviewed-by: Wenzhe Zhou Reviewed-by: Alexey Serbin Reviewed-by: Grant Henke Tested-by: Grant Henke --- M src/kudu/security/CMakeLists.txt M src/kudu/security/crypto.cc M src/kudu/security/openssl_util.cc M src/kudu/security/openssl_util.h M src/kudu/security/tls_context.cc M src/kudu/security/tls_handshake.cc 6 files changed, 96 insertions(+), 13 deletions(-) Approvals: Wenzhe Zhou: Looks good to me, but someone else must approve Alexey Serbin: Looks good to me, approved Grant Henke: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16659 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifafc7dcf91db910123276b657515e410bb7f6fcd Gerrit-Change-Number: 16659 Gerrit-PatchSet: 18 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wenzhe Zhou
[kudu-CR] KUDU-3210 Add lock before verifying signature
Grant Henke has removed a vote on this change. Change subject: KUDU-3210 Add lock before verifying signature .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16659 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Ifafc7dcf91db910123276b657515e410bb7f6fcd Gerrit-Change-Number: 16659 Gerrit-PatchSet: 17 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wenzhe Zhou
[kudu-CR] KUDU-3210 Add lock before verifying signature
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16659 ) Change subject: KUDU-3210 Add lock before verifying signature .. Patch Set 17: Verified+1 Code-Review+2 ASAN failure looks like KUDU-3171 -- To view, visit http://gerrit.cloudera.org:8080/16659 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifafc7dcf91db910123276b657515e410bb7f6fcd Gerrit-Change-Number: 16659 Gerrit-PatchSet: 17 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 04 Nov 2020 01:51:23 + Gerrit-HasComments: No
[kudu-CR] KUDU-3210 Increase key size in tests and EMC
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16658 ) Change subject: KUDU-3210 Increase key size in tests and EMC .. Patch Set 15: Code-Review+2 This is likely a separate patch, but do we also want to add KUDU_USE_LARGE_KEYS_IN_TESTS support to the Java tests? It might just be removing this section when KUDU_USE_LARGE_KEYS_IN_TESTS is true: https://github.com/apache/kudu/blob/3343144fefaad5a30e95e21297c64c78e308fa1f/java/kudu-test-utils/src/main/java/org/apache/kudu/test/cluster/MiniKuduCluster.java#L145-L151 -- To view, visit http://gerrit.cloudera.org:8080/16658 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I45b96e0b4499ea1d80db5871a529c732ad41220e Gerrit-Change-Number: 16658 Gerrit-PatchSet: 15 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 04 Nov 2020 01:49:36 + Gerrit-HasComments: No
[kudu-CR] KUDU-3210 Disable digest authn in FIPS mode
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16675 ) Change subject: KUDU-3210 Disable digest authn in FIPS mode .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16675 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4a446aa8d95a67658c727d3a6f85943d64c79ecf Gerrit-Change-Number: 16675 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 04 Nov 2020 01:45:24 + Gerrit-HasComments: No
[kudu-CR] [examples] Update examples dependencies
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16695 Change subject: [examples] Update examples dependencies .. [examples] Update examples dependencies This patch updates the examples builds to use more current dependencies. Change-Id: I74aa9ea4375d3aee9b08122b8568ca4d45c686ec --- M examples/java/collectl/pom.xml M examples/java/insert-loadgen/pom.xml M examples/java/java-example/pom.xml M examples/scala/sbt-int-test-example/build.sbt M examples/scala/spark-example/pom.xml 5 files changed, 27 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/16695/1 -- To view, visit http://gerrit.cloudera.org:8080/16695 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I74aa9ea4375d3aee9b08122b8568ca4d45c686ec Gerrit-Change-Number: 16695 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [docker] Fix build when using default base OS
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16694 Change subject: [docker] Fix build when using default base OS .. [docker] Fix build when using default base OS When using the default base OS the string is not utf8 type and `isnumeric` is not defined. This small fix changes to `isdigit` which works with all python strings. Change-Id: I81069970caabce1c7b9e0cdabaee403fdf921bad --- M docker/docker-build.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/16694/1 -- To view, visit http://gerrit.cloudera.org:8080/16694 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I81069970caabce1c7b9e0cdabaee403fdf921bad Gerrit-Change-Number: 16694 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-1563: Fix DELETE IGNORE operations with extra columns set
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16682 ) Change subject: KUDU-1563: Fix DELETE_IGNORE operations with extra columns set .. KUDU-1563: Fix DELETE_IGNORE operations with extra columns set In commit 17fb5a1 the Java client was enhanced to support extra columns set on DELETE operations. This patch ensures the same functionality works with DELETE_IGNORE operations. Change-Id: I179c2085c5061c71dc010929056247340f062b8c Reviewed-on: http://gerrit.cloudera.org:8080/16682 Tested-by: Grant Henke Reviewed-by: Andrew Wong Reviewed-by: Alexey Serbin --- M java/kudu-client/src/main/java/org/apache/kudu/client/Operation.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduSession.java 2 files changed, 13 insertions(+), 3 deletions(-) Approvals: Grant Henke: Verified Andrew Wong: Looks good to me, approved Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16682 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I179c2085c5061c71dc010929056247340f062b8c Gerrit-Change-Number: 16682 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563: Fix DELETE IGNORE operations with extra columns set
Grant Henke has removed a vote on this change. Change subject: KUDU-1563: Fix DELETE_IGNORE operations with extra columns set .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16682 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I179c2085c5061c71dc010929056247340f062b8c Gerrit-Change-Number: 16682 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1563: Fix DELETE IGNORE operations with extra columns set
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16682 ) Change subject: KUDU-1563: Fix DELETE_IGNORE operations with extra columns set .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16682 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I179c2085c5061c71dc010929056247340f062b8c Gerrit-Change-Number: 16682 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 30 Oct 2020 20:35:37 + Gerrit-HasComments: No
[kudu-CR] KUDU-1563: Fix DELETE IGNORE operations with extra columns set
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16682 ) Change subject: KUDU-1563: Fix DELETE_IGNORE operations with extra columns set .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16682/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduSession.java File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduSession.java: http://gerrit.cloudera.org:8080/#/c/16682/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduSession.java@211 PS1, Line 211: /** Regression test for KUDU-3198. Delete with full row from a 64-column table. */ > Is it worth updating this one too? Done -- To view, visit http://gerrit.cloudera.org:8080/16682 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I179c2085c5061c71dc010929056247340f062b8c Gerrit-Change-Number: 16682 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 30 Oct 2020 20:03:04 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1563: Fix DELETE IGNORE operations with extra columns set
Hello Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16682 to look at the new patch set (#2). Change subject: KUDU-1563: Fix DELETE_IGNORE operations with extra columns set .. KUDU-1563: Fix DELETE_IGNORE operations with extra columns set In commit 17fb5a1 the Java client was enhanced to support extra columns set on DELETE operations. This patch ensures the same functionality works with DELETE_IGNORE operations. Change-Id: I179c2085c5061c71dc010929056247340f062b8c --- M java/kudu-client/src/main/java/org/apache/kudu/client/Operation.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduSession.java 2 files changed, 13 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/82/16682/2 -- To view, visit http://gerrit.cloudera.org:8080/16682 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I179c2085c5061c71dc010929056247340f062b8c Gerrit-Change-Number: 16682 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [python] KUDU-1563. Add support for UPDATE IGNORE and DELETE IGNORE
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16667 ) Change subject: [python] KUDU-1563. Add support for UPDATE_IGNORE and DELETE_IGNORE .. [python] KUDU-1563. Add support for UPDATE_IGNORE and DELETE_IGNORE Implements Python support for the `UPDATE_IGNORE' and `DELETE_IGNORE ` operations Change-Id: Iad0fbd1db1edc140b88e1559ecf5d89bf6d9cbb9 Reviewed-on: http://gerrit.cloudera.org:8080/16667 Tested-by: Kudu Jenkins Reviewed-by: Andrew Wong --- M python/kudu/__init__.py M python/kudu/client.pyx M python/kudu/libkudu_client.pxd M python/kudu/tests/test_client.py 4 files changed, 80 insertions(+), 2 deletions(-) Approvals: Kudu Jenkins: Verified Andrew Wong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16667 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iad0fbd1db1edc140b88e1559ecf5d89bf6d9cbb9 Gerrit-Change-Number: 16667 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] WIP: KUDU-1563 Use DELETE IGNORE in KuduRestore job
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16683 Change subject: WIP: KUDU-1563 Use DELETE_IGNORE in KuduRestore job .. WIP: KUDU-1563 Use DELETE_IGNORE in KuduRestore job This patch changes the KuduRestore job to use DELETE_IGNORE operations instead of DELETE with kudu.setIgnoreAllNotFoundRows. TODO: After KUDU-3211 automatically detect if DELETE_IGNORE is supported and fall back to DELETE with `kudu.setIgnoreAllNotFoundRows = true`. Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala 1 file changed, 4 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/16683/1 -- To view, visit http://gerrit.cloudera.org:8080/16683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib6f6d5a31be77630e79ff1566e796eb5183a5d22 Gerrit-Change-Number: 16683 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
