[kudu-CR] KUDU-3371 [fs] Use RocksDB to store LBM metadata
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/18569 )
Change subject: KUDU-3371 [fs] Use RocksDB to store LBM metadata
..
Patch Set 61:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/18569/61//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18569/61//COMMIT_MSG@50
PS61, Line 50: contructed
nit: constructed
http://gerrit.cloudera.org:8080/#/c/18569/61//COMMIT_MSG@52
PS61, Line 52: Contruct
nit: Construct
http://gerrit.cloudera.org:8080/#/c/18569/61//COMMIT_MSG@56
PS61, Line 56: logr
"logr" a little short and not clear. use 'rocksdb' directly?
http://gerrit.cloudera.org:8080/#/c/18569/61/src/kudu/fs/fs_report.h
File src/kudu/fs/fs_report.h:
http://gerrit.cloudera.org:8080/#/c/18569/61/src/kudu/fs/fs_report.h@291
PS61, Line 291: corrupted_rdb_record_check
This field seems no used. So add a TODO?
http://gerrit.cloudera.org:8080/#/c/18569/61/src/kudu/fs/log_block_manager.cc
File src/kudu/fs/log_block_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18569/61/src/kudu/fs/log_block_manager.cc@2093
PS61, Line 2093: tmp_key = Substitute("$0.$1", id_,
lb->block_id().ToString());
this statement happens several times in this file, may be a simple function to
construct this key is better.
http://gerrit.cloudera.org:8080/#/c/18569/61/src/kudu/fs/log_block_manager.cc@4035
PS61, Line 4035: key(e.block_id);
Should this key be id + "." + block_id ?
--
To view, visit http://gerrit.cloudera.org:8080/18569
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie72f6914eb5653a9c034766c6cd3741a8340711f
Gerrit-Change-Number: 18569
Gerrit-PatchSet: 61
Gerrit-Owner: Yingchun Lai
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Sat, 23 Dec 2023 15:59:47 +
Gerrit-HasComments: Yes
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19909 ) Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. Patch Set 8: > Patch Set 8: > > (1 comment) Hello Alexey. I am very glad receive your feedback. In my docuement: "In fact, kudu duplication implements a cdc(capture data changes) at kudu. CDC is a common requirement." "destination system It’s the destination for records of Kudu duplication. It can be a local kudu engine, remote kudu cluster, kafka and other storage systems. In this document, it refers to Kafka." I think, this whole patch (https://gerrit.cloudera.org/c/20151) has included what Mr Greber want to do except I first implement cdc to kafka using a kafka lib while Mr Greber what to use kudu client lib. It has included my design. Maybe some details and ideas may be difference, we can communicate with each other more carefully. -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 8 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 20 Nov 2023 01:44:17 + Gerrit-HasComments: No
[kudu-CR] KUDU-3371 [fs] make LBMCorruptor a base class
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20504 ) Change subject: KUDU-3371 [fs] make LBMCorruptor a base class .. Patch Set 5: Code-Review+1 LGTM -- To view, visit http://gerrit.cloudera.org:8080/20504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1f2842598f46e88fbd08273d8fd19ed34a9cc5 Gerrit-Change-Number: 20504 Gerrit-PatchSet: 5 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 13 Oct 2023 08:48:02 + Gerrit-HasComments: No
[kudu-CR] KUDU-3371 [fs] make LBMCorruptor a base class
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20504 ) Change subject: KUDU-3371 [fs] make LBMCorruptor a base class .. Patch Set 3: Verified+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/20504/3/src/kudu/fs/log_block_manager-test-util.h File src/kudu/fs/log_block_manager-test-util.h: http://gerrit.cloudera.org:8080/#/c/20504/3/src/kudu/fs/log_block_manager-test-util.h@155 PS3, Line 155: // 'error_type' is in the following types of errors: : // 0. No block offset. : // 1. No block length. : // 2. Negative block offset. : // 3. Negative block length. : // 4. Offset + length > data file size. : // 5. Unrecognized op type. What about consider uniting them into an enum structure ? -- To view, visit http://gerrit.cloudera.org:8080/20504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1f2842598f46e88fbd08273d8fd19ed34a9cc5 Gerrit-Change-Number: 20504 Gerrit-PatchSet: 3 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 12 Oct 2023 10:22:15 + Gerrit-HasComments: Yes
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Hello Tidy Bot, Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, KeDeng, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#8). Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. [duplication] KUDU-3290 Support write ops to kafka with kafka client Kudu duplication is a large patch, design document shown at https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. This patch is the first patch for duplication, it includes: 1. adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. write records to kafka by librdkafka/cppkafka. I will commit about 9 patches for duplication. 1. Support write ops to kafka with kafka client 2. Add a new raft role DUPLICATOR and raft processing 3. registe kafka uri and validate and unregiste 4. recovering duplication when a leader switches 5. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication The whole patch is like this: https://gerrit.cloudera.org/c/20151. This patch is to help reviewers a overview about the duplication feature. [1] https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. [2] https://issues.apache.org/jira/browse/KUDU-3290. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml M java/kudu-proto/build.gradle M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/metadata.proto A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/integration-tests/single_broker_kafka.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/tablet/ops/write_op.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 26 files changed, 2,216 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/8 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 8 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [common] minor optimization on ComputeHashBuckets
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/20288 )
Change subject: [common] minor optimization on ComputeHashBuckets
..
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20288/1/src/kudu/common/partition_pruner.cc
File src/kudu/common/partition_pruner.cc:
http://gerrit.cloudera.org:8080/#/c/20288/1/src/kudu/common/partition_pruner.cc@225
PS1, Line 225: if (std::all_of(hash_bucket_bitset->begin(),
: hash_bucket_bitset->end(),
: [](bool b) { return b; })) {
: return;
: }
> Is it equal to the below? std::any_of will early return if find a false val
Both are right.
It seems no obvious distinction in algorithm implements for 'all_of' and
'any_of'.
https://github.com/llvm-mirror/libcxx/blob/a12cb9d211019d99b5875b6d8034617cbc24c2cc/include/algorithm#L838
--
To view, visit http://gerrit.cloudera.org:8080/20288
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iecd9a739f408646c0dab74727f919e4f6ce3ce7e
Gerrit-Change-Number: 20288
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Tue, 01 Aug 2023 07:51:48 +
Gerrit-HasComments: Yes
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#7). Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. [duplication] KUDU-3290 Support write ops to kafka with kafka client Kudu duplication is a large patch, design document shown at https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. This patch is the first patch for duplication, it includes: 1. adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. write records to kafka by librdkafka/cppkafka. I will commit about 9 patches for duplication. 1. Support write ops to kafka with kafka client 2. Add a new raft role DUPLICATOR and raft processing 3. registe kafka uri and validate and unregiste 4. recovering duplication when a leader switches 5. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication The whole patch is like this: https://gerrit.cloudera.org/c/20151. This patch is to help reviewers a overview about the duplication feature. [1] https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. [2] https://issues.apache.org/jira/browse/KUDU-3290. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/metadata.proto A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/integration-tests/single_broker_kafka.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/tablet/ops/write_op.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 25 files changed, 2,215 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/7 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 7 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#6). Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. [duplication] KUDU-3290 Support write ops to kafka with kafka client Kudu duplication is a large patch, design document shown at https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. This patch is the first patch for duplication, it includes: 1. adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. write records to kafka by librdkafka/cppkafka. I will commit about 9 patches for duplication. 1. Support write ops to kafka with kafka client 2. Add a new raft role DUPLICATOR and raft processing 3. registe kafka uri and validate and unregiste 4. recovering duplication when a leader switches 5. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication The whole patch is like this: https://gerrit.cloudera.org/c/20151. This patch is to help reviewers a overview about the duplication feature. [1] https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. [2] https://issues.apache.org/jira/browse/KUDU-3290. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/metadata.proto A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/integration-tests/single_broker_kafka.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_replica.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 28 files changed, 2,241 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/6 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 6 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#5). Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. [duplication] KUDU-3290 Support write ops to kafka with kafka client Kudu duplication is a large patch, design document shown at https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. This patch is the first patch for duplication, it includes: 1. adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. write records to kafka by librdkafka/cppkafka. I will commit about 9 patches for duplication. 1. Support write ops to kafka with kafka client 2. Add a new raft role DUPLICATOR and raft processing 3. registe kafka uri and validate and unregiste 4. recovering duplication when a leader switches 5. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication I have provided an united patch https://gerrit.cloudera.org/c/20151. Its purpose is to help reviewers a overview about this function. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/metadata.proto A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/integration-tests/single_broker_kafka.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/tablet_server.h M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h M src/kudu/util/blocking_queue.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 35 files changed, 2,793 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/5 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 5 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/19909 )
Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka
client
..
Patch Set 5:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19909/3/src/kudu/integration-tests/single_broker_kafka.h
File src/kudu/integration-tests/single_broker_kafka.h:
http://gerrit.cloudera.org:8080/#/c/19909/3/src/kudu/integration-tests/single_broker_kafka.h@39
PS3, Line 39: static const int kZookeeperSessionTimeoutS = 10;
> warning: method 'InitKafka' can be made const [readability-make-member-func
Done
http://gerrit.cloudera.org:8080/#/c/19909/3/src/kudu/integration-tests/single_broker_kafka.h@48
PS3, Line 48:
> warning: method 'StartKafka' can be made const [readability-make-member-fun
Done
http://gerrit.cloudera.org:8080/#/c/19909/3/src/kudu/integration-tests/single_broker_kafka.h@57
PS3, Line 57: string command = Substitute("$0 start $1", command_,
offset_port_);
> warning: method 'StopKafka' can be made const [readability-make-member-func
Done
http://gerrit.cloudera.org:8080/#/c/19909/3/src/kudu/integration-tests/single_broker_kafka.h@66
PS3, Line 66: }
> warning: method 'DestroyKafka' can be made const [readability-make-member-f
Done
http://gerrit.cloudera.org:8080/#/c/19909/3/src/kudu/tablet/tablet_replica.cc
File src/kudu/tablet/tablet_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19909/3/src/kudu/tablet/tablet_replica.cc@199
PS3, Line 199:
> error: constructor for 'kudu::tablet::TabletReplica' must explicitly initia
Done
--
To view, visit http://gerrit.cloudera.org:8080/19909
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49
Gerrit-Change-Number: 19909
Gerrit-PatchSet: 5
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Mon, 31 Jul 2023 13:32:37 +
Gerrit-HasComments: Yes
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#4). Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. [duplication] KUDU-3290 Support write ops to kafka with kafka client Kudu duplication is a large patch, design document shown at https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing. This patch is the first patch for duplication, it includes: 1. adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. write records to kafka by librdkafka/cppkafka. I will commit about 9 patches for duplication. 1. Support write ops to kafka with kafka client 2. Add a new raft role DUPLICATOR and raft processing 3. registe kafka uri and validate and unregiste 4. recovering duplication when a leader switches 5. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication I have provided an united patch https://gerrit.cloudera.org/c/20151. Its purpose is to help reviewers a overview about this function. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/metadata.proto A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/integration-tests/single_broker_kafka.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/tablet_server.h M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h M src/kudu/util/blocking_queue.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 35 files changed, 2,812 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/4 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 4 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20151 ) Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. Patch Set 16: Code-Review-2 Make it not push to remote repo. -- To view, visit http://gerrit.cloudera.org:8080/20151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2ec587821f41eef75078152d7912c11456906ecf Gerrit-Change-Number: 20151 Gerrit-PatchSet: 16 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 31 Jul 2023 07:49:17 + Gerrit-HasComments: No
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20151 ) Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. Patch Set 16: > Patch Set 14: > > I'm not sure it make sense to pile like patches like this if you want them to > be reviewed in eventually pushed into the repo. > > It's hard to review and revv these. > > Please prefer to maintain a stack of patches, each being logically separated > and doing a well-defined part of the overall task. > > Also, have you addressed the feedback on the design document for this feature? > > Thanks! This patch is for passed all tests and then I'll split this patch into several individual patches. The document: https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit -- To view, visit http://gerrit.cloudera.org:8080/20151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2ec587821f41eef75078152d7912c11456906ecf Gerrit-Change-Number: 20151 Gerrit-PatchSet: 16 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 28 Jul 2023 09:19:22 + Gerrit-HasComments: No
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#16). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#15). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#14). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#13). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#12). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#11). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] remove the supportng of EOL macOS 10.x
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20185 ) Change subject: remove the supportng of EOL macOS 10.x .. Patch Set 1: (1 comment) Basically, I check the output binaries(using mac os 12.4), it's ok. I'll try to compile it and run it. http://gerrit.cloudera.org:8080/#/c/20185/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20185/1//COMMIT_MSG@7 PS1, Line 7: supportng nit: supporting -- To view, visit http://gerrit.cloudera.org:8080/20185 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifd0658eef8ee8b275e50d49f504c23de9615a5d7 Gerrit-Change-Number: 20185 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 13 Jul 2023 15:36:17 + Gerrit-HasComments: Yes
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#10). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#9). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#8). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20151 ) Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. Patch Set 7: (3 comments) http://gerrit.cloudera.org:8080/#/c/20151/4/src/kudu/rebalance/cluster_status.h File src/kudu/rebalance/cluster_status.h: http://gerrit.cloudera.org:8080/#/c/20151/4/src/kudu/rebalance/cluster_status.h@76 PS4, Line 76: std::optional opid_index, > warning: pass by value and use std::move [modernize-pass-by-value] Done http://gerrit.cloudera.org:8080/#/c/20151/4/src/kudu/rebalance/cluster_status.h@83 PS4, Line 83: opid_index(std::move(opid_index)), > warning: parameter 'opid_index' is passed by value and only copied once; co Done http://gerrit.cloudera.org:8080/#/c/20151/4/src/kudu/tablet/tablet_bootstrap.cc File src/kudu/tablet/tablet_bootstrap.cc: http://gerrit.cloudera.org:8080/#/c/20151/4/src/kudu/tablet/tablet_bootstrap.cc@1404 PS4, Line 1404: *all_skipped &= action != NEEDS_REPLAY; > warning: The left operand of '!=' is a garbage value [clang-analyzer-core.U Done -- To view, visit http://gerrit.cloudera.org:8080/20151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2ec587821f41eef75078152d7912c11456906ecf Gerrit-Change-Number: 20151 Gerrit-PatchSet: 7 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 06 Jul 2023 03:59:40 + Gerrit-HasComments: Yes
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#7). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#6). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#5). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/config/spotbugs/excludeFilter.xml A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M
[kudu-CR](gh-pages) [web-site] Add a new PMC member
Hello Yingchun Lai, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20159 to look at the new patch set (#2). Change subject: [web-site] Add a new PMC member .. [web-site] Add a new PMC member Change-Id: I07f0f90ece010f2a3bbe2746b1ea78cfe58817c3 --- M committers.md 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/20159/2 -- To view, visit http://gerrit.cloudera.org:8080/20159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I07f0f90ece010f2a3bbe2746b1ea78cfe58817c3 Gerrit-Change-Number: 20159 Gerrit-PatchSet: 2 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Yingchun Lai
[kudu-CR](gh-pages) [web-site] Add a new PMC member
Yuqi Du has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20159 Change subject: [web-site] Add a new PMC member .. [web-site] Add a new PMC member Change-Id: I07f0f90ece010f2a3bbe2746b1ea78cfe58817c3 --- M committers.md 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/20159/1 -- To view, visit http://gerrit.cloudera.org:8080/20159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newchange Gerrit-Change-Id: I07f0f90ece010f2a3bbe2746b1ea78cfe58817c3 Gerrit-Change-Number: 20159 Gerrit-PatchSet: 1 Gerrit-Owner: Yuqi Du
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#4). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_bootstrap.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#3). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_bootstrap.cc M
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20151 to look at the new patch set (#2). Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_bootstrap.cc M
[kudu-CR] WIP KUDU-3290 support duplication to kafka
Yuqi Du has abandoned this change. ( http://gerrit.cloudera.org:8080/18350 ) Change subject: WIP KUDU-3290 support duplication to kafka .. Abandoned I abandon it. New patch is https://gerrit.cloudera.org/c/20151/ -- To view, visit http://gerrit.cloudera.org:8080/18350 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I842476216f09150383d06c7a6f4c53bf7f2ec018 Gerrit-Change-Number: 18350 Gerrit-PatchSet: 12 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du
[kudu-CR] WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches
Yuqi Du has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20151 Change subject: WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches .. WIP [duplication] KUDU-3290 Unite all patches as one to prepare these patches This patch is for duplication, it includes all about duplication. The purpose of this patch is do some preparing works for duplication. Some refactors need to do such as tests. This patch is a little too many codes, so I plan to split this patch into 10 patches. They are: 1. Adding 'Duplicator' which will write records to kafka, adding extra 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Add a new raft role DUPLICATOR and raft processing 3. recovering duplication when a leader switches 4. creating/altering in server-side for duplication 5. cpp client apis and CLI tool 6. java client apis 7. suppports kerberos 8. alter a duplication 9. registe kafka uri and validate and unregiste During this progressing, plans maybe adjust according to advices. Change-Id: I2ec587821f41eef75078152d7912c11456906ecf --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake A java/kudu-client/src/main/java/org/apache/kudu/DuplicationDownstreamType.java M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTablesResponse.java M java/kudu-client/src/main/java/org/apache/kudu/client/ListTabletsResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-proto/build.gradle M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica-internal.cc M src/kudu/client/replica-internal.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/scan_token-internal.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.h M src/kudu/common/common.proto M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/consensus_meta_manager.cc M src/kudu/consensus/consensus_meta_manager.h M src/kudu/consensus/consensus_queue.cc M src/kudu/consensus/metadata.proto M src/kudu/consensus/peer_manager.cc M src/kudu/consensus/quorum_util-test.cc M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplication_replay-test.cc A src/kudu/duplicator/duplication_replay.cc A src/kudu/duplicator/duplication_replay.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/log_segment_reader.cc A src/kudu/duplicator/log_segment_reader.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_fuzzy_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kafka-itest.cc A src/kudu/integration-tests/duplication_with_kerberos_kafka-itest.cc M src/kudu/integration-tests/fuzz-itest.cc A src/kudu/integration-tests/single_broker_kafka.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/sys_catalog.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/cluster_status.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/subprocess/subprocess_server-test.cc M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/diskrowset-test.cc A src/kudu/tablet/ops/duplication_op.cc A src/kudu/tablet/ops/duplication_op.h M src/kudu/tablet/ops/op.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/op_tracker-test.cc M src/kudu/tablet/ops/op_tracker.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M
[kudu-CR](branch-1.17.x) Bump versions in examples to 1.17.0
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20140 ) Change subject: Bump versions in examples to 1.17.0 .. Patch Set 1: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc File examples/java/collectl/README.adoc: http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc@90 PS1, Line 90: spark2_2.11 > in java/gradle/dependencies.gradle: Yes . I have no questions. -- To view, visit http://gerrit.cloudera.org:8080/20140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: Icf4b023c19de278c52f931574af6029de8de57ec Gerrit-Change-Number: 20140 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 03 Jul 2023 07:42:58 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.17.x) Bump versions in examples to 1.17.0
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20140 ) Change subject: Bump versions in examples to 1.17.0 .. Patch Set 1: (1 comment) > Patch Set 1: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc File examples/java/collectl/README.adoc: http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc@90 PS1, Line 90: spark2_2.11 > in java/gradle/dependencies.gradle: in java/gradle/dependencies.gradle: b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 56) scala211 : "2.11.12", 10e3cec1 (Attila Bukor 2022-01-19 08:42:55 +0100 57) scala : "2.12.15", which scala do we use ? If we use scala211 and spark2 it is kudu-spark2_2.11:1.17.0. In my default environ scala is "2.12.15". What is it determined by ? -- To view, visit http://gerrit.cloudera.org:8080/20140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: Icf4b023c19de278c52f931574af6029de8de57ec Gerrit-Change-Number: 20140 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 03 Jul 2023 07:36:15 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.17.x) Bump versions in examples to 1.17.0
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/20140 )
Change subject: Bump versions in examples to 1.17.0
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc
File examples/java/collectl/README.adoc:
http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc@90
PS1, Line 90: spark2_2.11
> Kudu supports two versions: Spark2 and Spark3. In this example, this has t
in java/gradle/dependencies.gradle:
10e3cec1 (Attila Bukor 2022-01-19 08:42:55 +0100 57) scala :
"2.12.15",
10e3cec1 (Attila Bukor 2022-01-19 08:42:55 +0100 61) spark2 :
"2.4.8",
10e3cec1 (Attila Bukor 2022-01-19 08:42:55 +0100 62) spark :
"3.1.2",
b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 72) // If the `spark2`
property is passed, override the `spark` and `scala` version
b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 73) // to use the `spark2`
and `scala211` versions.
b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 74) if
(propertyExists("spark2")) {
b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 75) versions["spark"] =
"$versions.spark2"
b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 76) versions["scala"] =
"$versions.scala211"
b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 77) }
b2039d51 (Grant Henke2020-10-12 10:08:43 -0500 78)
e5202d30 (Grant Henke2017-06-19 09:35:55 -0500 79) // Add base Scala
version
e5202d30 (Grant Henke2017-06-19 09:35:55 -0500 80) versions["scalaBase"] =
versions.scala.substring(0, versions.scala.lastIndexOf("."))
e5202d30 (Grant Henke2017-06-19 09:35:55 -0500 81)
fa953d05 (Grant Henke2017-08-16 14:36:47 -0500 82) // Add base Spark
version
e5202d30 (Grant Henke2017-06-19 09:35:55 -0500 83) versions["sparkBase"] =
versions.spark.substring(0, versions.spark.indexOf("."))
java/kudu-spark/build.gradle:
74b9ac67 (Grant Henke 2019-11-19 15:20:52 -0600 41) archivesBaseName =
"kudu-spark${versions.sparkBase}_${versions.scalaBase}"
If we use 'spark', its version is "3.1.2" and versions["sparkBase"] is 3.
If we use 'spark2', its version is "2.4.8", and versions["sparkBase"] is 2.
versions["scalaBase"] are the same which is 2.12, whether spark or spark2.
So, spark${versions.sparkBase}_${versions.scalaBase}":
kudu-spark3_2.12 if 'spark'
kudu-spark2_2.12 if 'spark2' . Right?
If spark2, it'skudu-spark2_2.12:1.17.0 ?
--
To view, visit http://gerrit.cloudera.org:8080/20140
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.17.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Icf4b023c19de278c52f931574af6029de8de57ec
Gerrit-Change-Number: 20140
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Mon, 03 Jul 2023 07:09:29 +
Gerrit-HasComments: Yes
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has removed a vote on this change. Change subject: [java] allow disabling TLS transportation from java client .. Removed Code-Review-2 by Attila Bukor -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 11 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. Patch Set 11: > Patch Set 5: > > > Patch Set 5: Code-Review-2 > > (changed my vote to -2 to make sure we don't accidentally submit) I'll remove you -2. This patch remove the environment variable and use a AsyncKuduClient option variable. It need review again. -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 11 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com> Gerrit-Comment-Date: Sun, 02 Jul 2023 14:35:21 + Gerrit-HasComments: No
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has removed a vote on this change. Change subject: [java] allow disabling TLS transportation from java client .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 11 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. Patch Set 11: Verified+1 A test in DEBUG: LocationAwareRebalancingBasicTest.IntraLocationNoConcurrency. failed due to a tcmalloc coredump. It does not matter with this patch. -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 11 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com> Gerrit-Comment-Date: Sun, 02 Jul 2023 14:28:13 + Gerrit-HasComments: No
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Yuqi Du has removed a vote on this change. Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 19 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR](branch-1.17.x) Bump versions in examples to 1.17.0
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20140 ) Change subject: Bump versions in examples to 1.17.0 .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc File examples/java/collectl/README.adoc: http://gerrit.cloudera.org:8080/#/c/20140/1/examples/java/collectl/README.adoc@90 PS1, Line 90: spark2_2.11 I study this version, this version may spark3_2.12:1.17.0 ? and all other this spark jar version changes. in java/gradle/dependencies.gradle: 10e3cec1 (Attila Bukor 2022-01-19 08:42:55 +0100 57) scala : "2.12.15", 10e3cec1 (Attila Bukor 2022-01-19 08:42:55 +0100 62) spark : "3.1.2", http://gerrit.cloudera.org:8080/#/c/20140/1/examples/quickstart/nifi/README.adoc File examples/quickstart/nifi/README.adoc: http://gerrit.cloudera.org:8080/#/c/20140/1/examples/quickstart/nifi/README.adoc@166 PS1, Line 166: spark2_2.11 The same as above. http://gerrit.cloudera.org:8080/#/c/20140/1/examples/quickstart/spark/README.adoc File examples/quickstart/spark/README.adoc: http://gerrit.cloudera.org:8080/#/c/20140/1/examples/quickstart/spark/README.adoc@72 PS1, Line 72: spark2_2.11 Mark -- To view, visit http://gerrit.cloudera.org:8080/20140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: Icf4b023c19de278c52f931574af6029de8de57ec Gerrit-Change-Number: 20140 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Sat, 01 Jul 2023 16:44:40 + Gerrit-HasComments: Yes
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has uploaded a new patch set (#11) to the change originally created by wangning. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. [java] allow disabling TLS transportation from java client In a cluster which is running in intranet, sometimes security is less concerned than performance in case port is not exposed. As mentioned in [1], on JAVA8 the TLS cypher performance is poor. And to disable TLS from server side requires to reboot cluster, it worths a trade off for a working cluster. This patch introduces a new method 'disableRpcTls' in AsyncKuduClientBuilder to allow disabling use TLS transportation in client side. It's default value is true. [1] https://issues.apache.org/jira/browse/KUDU-3133 Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiation.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/KuduTestHarness.java 7 files changed, 126 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/87/16887/11 -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 11 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has uploaded a new patch set (#10) to the change originally created by wangning. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. [java] allow disabling TLS transportation from java client In a cluster which is running in intranet, sometimes security is less concerned than performance in case port is not exposed. As mentioned in [1], on JAVA8 the TLS cypher performance is poor. And to disable TLS from server side requires to reboot cluster, it worths a trade off for a working cluster. This patch introduces a new method 'disableRpcTls' in AsyncKuduClientBuilder to allow disabling use TLS transportation in client side. It's default value is true. [1] https://issues.apache.org/jira/browse/KUDU-3133 Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiation.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/KuduTestHarness.java 7 files changed, 125 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/87/16887/10 -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 10 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has uploaded a new patch set (#9) to the change originally created by wangning. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. [java] allow disabling TLS transportation from java client In a cluster which is running in intranet, sometimes security is less concerned than performance in case port is not exposed. As mentioned in [1], on JAVA8 the TLS cypher performance is poor. And to disable TLS from server side requires to reboot cluster, it worths a trade off for a working cluster. This commit introduces a new variable 'rpcEnableTls' in AsyncKuduClientBuilder variable to allow disabling use TLS transportation in client side. The default value is true. [1] https://issues.apache.org/jira/browse/KUDU-3133 Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiation.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/KuduTestHarness.java 7 files changed, 125 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/87/16887/9 -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 9 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has uploaded a new patch set (#8) to the change originally created by wangning. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. [java] allow disabling TLS transportation from java client In a cluster which is running in intranet, sometimes security is less concerned than performance in case port is not exposed. As mentioned in [1], on JAVA8 the TLS cypher performance is poor. And to disable TLS from server side requires to reboot cluster, it worths a trade off for a working cluster. This commit introduces a new variable 'rpcEnableTls' in AsyncKuduClientBuilder variable to allow disabling use TLS transportation in client side. The default value is true. [1] https://issues.apache.org/jira/browse/KUDU-3133 Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiation.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/KuduTestHarness.java 7 files changed, 125 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/87/16887/8 -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 8 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has uploaded a new patch set (#7) to the change originally created by wangning. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. [java] allow disabling TLS transportation from java client In a cluster which is running in intranet, sometimes security is less concerned than performance in case port is not exposed. As mentioned in [1], on JAVA8 the TLS cypher performance is poor. And to disable TLS from server side requires to reboot cluster, it worths a trade off for a working cluster. This commit introduces a new variable 'rpcEnableTls' in AsyncKuduClientBuilder variable to allow disabling use TLS transportation in client side. The default value is true. [1] https://issues.apache.org/jira/browse/KUDU-3133 Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiation.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/KuduTestHarness.java 7 files changed, 125 insertions(+), 24 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/87/16887/7 -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 7 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] KUDU-2724 build macOS mini cluster binaries on newer macOS
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20141 ) Change subject: KUDU-2724 build macOS mini_cluster binaries on newer macOS .. Patch Set 3: Verified+1 Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20141 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id7d7ae2c55ffeac2ebc11513e6909bf6b0b8d6d7 Gerrit-Change-Number: 20141 Gerrit-PatchSet: 3 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 30 Jun 2023 10:40:32 + Gerrit-HasComments: No
[kudu-CR](branch-1.17.x) Bump version to 1.17.0
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20139 ) Change subject: Bump version to 1.17.0 .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I948139a36be528b09d18c77d4379bcce30184327 Gerrit-Change-Number: 20139 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 30 Jun 2023 10:36:34 + Gerrit-HasComments: No
[kudu-CR](branch-1.17.x) Bump version to 1.17.0
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20139 ) Change subject: Bump version to 1.17.0 .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I948139a36be528b09d18c77d4379bcce30184327 Gerrit-Change-Number: 20139 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 30 Jun 2023 10:36:27 + Gerrit-HasComments: No
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/18330 ) Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. Patch Set 19: > Patch Set 19: Verified+1 > > > Patch Set 19: Verified-1 > > > > Build Failed > > > > http://jenkins.kudu.apache.org/job/kudu-gerrit/28112/ : FAILURE > > > There are 2 failure. > > 1. DEBUG, rebalance_tool a test kudu is coredump. > > 2. ASAN. There was 1 failure: > 1) testFlushBySize(org.apache.kudu.client.TestAsyncKuduSession) > com.stumbleupon.async.TimeoutException: Timed out after 5ms when joining > Deferred@398562602(state=PENDING, result=null, callback=wakeup thread > Time-limited test, errback=wakeup thread Time-limited test) > at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1183) > at com.stumbleupon.async.Deferred.join(Deferred.java:1042) > at > org.apache.kudu.client.TestAsyncKuduSession.testFlushBySize(TestAsyncKuduSession.java:409) > > FAILURES!!! > Tests run: 12, Failures: 1 > > > I'll review the 2 unsteady tests later. The two fails do not matter with this patch. -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 19 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 30 Jun 2023 07:06:25 + Gerrit-HasComments: No
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/18330 ) Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. Patch Set 19: (1 comment) http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc File src/kudu/util/pb_util.cc: http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc@989 PS14, Line 989: // when users use '--json or --json_pr > I should study about this Thanks for your advices. I have added a new extend. You can review it again. And this fix will cause metadata file's header becomes a litter large. So I extract a fs_with_extends.proto file to avoid this disadvantage changes. -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 19 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 30 Jun 2023 07:04:52 + Gerrit-HasComments: Yes
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. Patch Set 6: > Patch Set 5: Code-Review-1 > > This is a nice feature, but right now we don't use many environment > variables, especially on the client-side. I was also thinking about adding a > similar config to *require* TLS as right now it's trivial to MITM and > downgrade to clear text. If we used env vars, these two vars would collide. > > For these reasons, I'd rather we used code config which is a lot more robust > and I feel like it's more widespread in the Java world. Env vars can still be > added on the application-side if a user decides they want to control this > behavior by env vars, but it's also possible to read the config from an XML > or YAML config file as well if we opt to use code-configs. > > I realize this patch might not be the best place to discuss this, so maybe we > should bring this up on dev@ instead. In the meantime, I'd recommend against > merging this so I'm voting -1. I'll remove the variable environment and pass a clientBuilder variable to control whether rpc use TLS. -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 6 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com> Gerrit-Comment-Date: Fri, 30 Jun 2023 06:53:50 + Gerrit-HasComments: No
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/18330 ) Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. Patch Set 19: Verified+1 > Patch Set 19: Verified-1 > > Build Failed > > http://jenkins.kudu.apache.org/job/kudu-gerrit/28112/ : FAILURE There are 2 failure. 1. DEBUG, rebalance_tool a test kudu is coredump. 2. ASAN. There was 1 failure: 1) testFlushBySize(org.apache.kudu.client.TestAsyncKuduSession) com.stumbleupon.async.TimeoutException: Timed out after 5ms when joining Deferred@398562602(state=PENDING, result=null, callback=wakeup thread Time-limited test, errback=wakeup thread Time-limited test) at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1183) at com.stumbleupon.async.Deferred.join(Deferred.java:1042) at org.apache.kudu.client.TestAsyncKuduSession.testFlushBySize(TestAsyncKuduSession.java:409) FAILURES!!! Tests run: 12, Failures: 1 I'll review the 2 unsteady tests later. -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 19 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 30 Jun 2023 06:48:59 + Gerrit-HasComments: No
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,
Abhishek Chennaka, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18330
to look at the new patch set (#19).
Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using
-json/-json_pretty
..
[tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty
The output of two commands below about the 'uuid' field are not the same
1. kudu pbc dump xxx/instance -debug/default
2. kudu pbc dump xxx/instance -json/-json_pretty
For example:
1. kudu pbc dump instance, which is a plain output format.
Message 0
---
uuid: "ac6b3e392cb34b4681e54a33c95fdf9c"
format_stamp: "Formatted at 2023-03-10 09:16:22 on 2a43db713d56"
2. kudu pbc dump instance --json, which is a json output format
{"uuid":"YWM2YjNlMzkyY2IzNGI0NjgxZTU0YTMzYzk1ZmRmOWM=","formatStamp":"Formatted
at 2023-03-10 09:16:22 on 2a43db713d56"}
The reason is the type of 'uuid' is bytes and protobuf's bytes type is
base64-encoded.
'--debug/default' mode shows uuid's plain text while '--debug/default'
mode uses protobuf's API and shows base64-encoded text.
This patch fixes this problem.
Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
---
M src/kudu/fs/CMakeLists.txt
M src/kudu/fs/fs.proto
M src/kudu/fs/fs_manager-test.cc
M src/kudu/fs/fs_manager.cc
A src/kudu/fs/fs_with_extend.proto
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_pbc.cc
M src/kudu/util/easy_json.cc
M src/kudu/util/easy_json.h
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
M src/kudu/util/pb_util.proto
14 files changed, 197 insertions(+), 80 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/19
--
To view, visit http://gerrit.cloudera.org:8080/18330
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
Gerrit-Change-Number: 18330
Gerrit-PatchSet: 19
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19794 ) Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. Patch Set 14: Code-Review+1 > Patch Set 13: > > > Patch Set 13: Code-Review+2 > > Just FYI, Yuqi: it's not a good practice to add +2 for your own patches > (unless you carrying over somebody's else +2 from prior PS just after re-base > or minor updates). OK. Thanks for your advice. I would follow this rule. -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 14 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 26 Jun 2023 09:00:34 + Gerrit-HasComments: No
[kudu-CR] KUDU-3371 [fs] make LogBlockContainer a base class
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19610 ) Change subject: KUDU-3371 [fs] make LogBlockContainer a base class .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19610 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieb199de95973aaeba76947ad095907272d84ca67 Gerrit-Change-Number: 19610 Gerrit-PatchSet: 5 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 26 Jun 2023 07:22:44 + Gerrit-HasComments: No
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19794 ) Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. Patch Set 14: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 14 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 26 Jun 2023 07:17:05 + Gerrit-HasComments: No
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/19794 )
Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up
hash partition pruning for in-list predicate
..
Patch Set 13:
(6 comments)
Thanks for your advices. You can review it again.
http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner-test.cc
File src/kudu/common/partition_pruner-test.cc:
http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner-test.cc@984
PS12, Line 984: generate some in-list predicates for these key columns. The old
algorith
> Sorry for the late reply. I will take a further more review of this patch a
The unit tests which tables with hash bucket partitions has tested the old
algorithm. At this, they are also tested the new algorithms, such as
PartitionPrunerTest::TestHashPruning, PartitionPrunerTest::TestPruning,
PartitionPrunerTest::TestHashSchemasPerRangeWithPartialPrimaryKeyRangePruning.
They can make sure the new algorithm's results are the same as the old one, in
other words, they has make sure the correctness of new algorithm.
So based on the old one algorithm is correct, I randomly construct a lots of
in-list cases to cover more combinations to which is impossible to check all
the combinations, make sure the general sensors and at the same time compare
with the performance.
http://gerrit.cloudera.org:8080/#/c/19794/13/src/kudu/common/partition_pruner-test.cc
File src/kudu/common/partition_pruner-test.cc:
http://gerrit.cloudera.org:8080/#/c/19794/13/src/kudu/common/partition_pruner-test.cc@1026
PS13, Line 1026: vector()
> nit: can be replaced by '{}'.
Done
http://gerrit.cloudera.org:8080/#/c/19794/13/src/kudu/common/partition_pruner-test.cc@1047
PS13, Line 1047: CHECK_EQ(v1.size(), v2.size());
: for (int i = 0; i < v1.size(); i++) {
: ASSERT_EQ(v1[i], v2[i]);
: }
> nit: they can be replaced by ASSERT_EQ(v1, v2);
Done
http://gerrit.cloudera.org:8080/#/c/19794/13/src/kudu/common/partition_pruner-test.cc@1069
PS13, Line 1069: kMaxSafeLength
> kMaxInListLength?
Done
http://gerrit.cloudera.org:8080/#/c/19794/13/src/kudu/common/partition_pruner-test.cc@1084
PS13, Line 1084: temp_values[temp_values.size() - 1]
> nit: temp_values.back() ?
Done
http://gerrit.cloudera.org:8080/#/c/19794/13/src/kudu/common/partition_pruner-test.cc@1085
PS13, Line 1085: j++;
> nit: Move it to line 1081 which is more conventional.
Done
--
To view, visit http://gerrit.cloudera.org:8080/19794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9
Gerrit-Change-Number: 19794
Gerrit-PatchSet: 13
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Mon, 26 Jun 2023 07:12:29 +
Gerrit-HasComments: Yes
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Hello Tidy Bot, Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, KeDeng, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19794 to look at the new patch set (#14). Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate This patch comes from a committed patch, its committed id is 'b69dbeb'. As that patch said, logic of pruning hash partitions for in-list predicate in Kudu cpp client has also a high space complexity and slow. Old algorithm must keep all intermedium objects because they are incomplete until they are completed and can be computed hash. This patch fixes the problems and provides a recursive algorithm the same as the one used by java client. Compared with java client, the cpp client is less likely to cause the OOM condition because it does not keep too many intermediate results. This optimization has good benefits too. The benefits are related to the in-list length and the number of primary keys, the performance would be better if in-list length is longer. For example, PartitionPrunerTest::TestMultiColumnInListHashPruningManyValues, using 10 key columns and kMaxInListLength=50, old algorithm memory cost may reach 600MB, while new algorithm's memory cost can be ignored (it only need one objects and a few stacks for contexts). At the same time, new algorithm has a good speedup, some effect as below: combination_count: 555400692, old cost: 428238us, new cost: 713us, speedup: 600.6x combination_count: 89083783664568, old cost: 2764924us, new cost: 1145us, speedup: 2414.7x combination_count: 27194091724800, old cost: 1610475us, new cost: 1151us, speedup: 1399.2x combination_count: 7116622216704, old cost: 34544289us, new cost: 375us, speedup: 92118.1x combination_count: 37570734489600, old cost: 1733205us, new cost: 901us, speedup: 1923.6x Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 --- M src/kudu/common/partition_pruner-test.cc M src/kudu/common/partition_pruner.cc M src/kudu/common/partition_pruner.h 3 files changed, 240 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19794/14 -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 14 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. Patch Set 6: This patch's author is wangning, I have obtained his consent and continue his work. Welcome to review this again. -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 6 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com> Gerrit-Comment-Date: Mon, 26 Jun 2023 06:31:34 + Gerrit-HasComments: No
[kudu-CR] [java] allow disabling TLS transportation from java client
Yuqi Du has uploaded a new patch set (#6) to the change originally created by wangning. ( http://gerrit.cloudera.org:8080/16887 ) Change subject: [java] allow disabling TLS transportation from java client .. [java] allow disabling TLS transportation from java client In a cluster which is running in intranet, sometimes security is less concerned than performance in case port is not exposed. As mentioned in [1], on JAVA8 the TLS cypher performance is poor. And to disable TLS from server side requires to reboot cluster, it worths a trade off for a working cluster. This commit introduces KUDU_CLIENT_DISABLE_TLS environment variable to allow disabling use TLS transportation in client side. [1] https://issues.apache.org/jira/browse/KUDU-3133 Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed --- M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java A java/kudu-client/src/main/java/org/apache/kudu/util/SystemEnvUtil.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiation.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/junit/ResultReporter.java 4 files changed, 199 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/87/16887/6 -- To view, visit http://gerrit.cloudera.org:8080/16887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I762b9d6ef9257a7853684dd0aef315e5c5bba8ed Gerrit-Change-Number: 16887 Gerrit-PatchSet: 6 Gerrit-Owner: wangning <1994wangn...@gmail.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: wangning <1994wangn...@gmail.com>
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19794 ) Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. Patch Set 13: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 13 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 21 Jun 2023 11:09:27 + Gerrit-HasComments: No
[kudu-CR] [java] add buffer space limit for KuduSession
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19918 ) Change subject: [java] add buffer space limit for KuduSession .. Patch Set 9: Verified+1 (2 comments) LGTM http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG@11 PS6, Line 11: configuring > Same spelling? sorry for mistake http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@649 PS6, Line 649: isExcessMaxSiz > Done Done -- To view, visit http://gerrit.cloudera.org:8080/19918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9 Gerrit-Change-Number: 19918 Gerrit-PatchSet: 9 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 15 Jun 2023 07:18:47 + Gerrit-HasComments: Yes
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#3). Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. [duplication] KUDU-3290 Support write ops to kafka with kafka client This patch is the first patch for duplication, it includes: 1. adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Duplicator: write records to kafka As my plans, I will commit about 9 commits for duplication. 1. Support write ops to kafka with kafka client 2. Add a new raft role DUPLICATOR and raft processing 3. registe kafka uri and validate and unregiste 4. recovering duplication when a leader switches 5. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication During this progressing, plans maybe adjust according to advices. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/metadata.proto A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/integration-tests/single_broker_kafka.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/tablet_server.h M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 33 files changed, 2,126 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/3 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 3 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,
Abhishek Chennaka, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18330
to look at the new patch set (#17).
Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using
-json/-json_pretty
..
[tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty
The output of two commands below about the 'uuid' field are not the same
1. kudu pbc dump xxx/instance -debug/default
2. kudu pbc dump xxx/instance -json/-json_pretty
For example:
1. kudu pbc dump instance, which is a plain output format.
Message 0
---
uuid: "ac6b3e392cb34b4681e54a33c95fdf9c"
format_stamp: "Formatted at 2023-03-10 09:16:22 on 2a43db713d56"
2. kudu pbc dump instance --json, which is a json output format
{"uuid":"YWM2YjNlMzkyY2IzNGI0NjgxZTU0YTMzYzk1ZmRmOWM=","formatStamp":"Formatted
at 2023-03-10 09:16:22 on 2a43db713d56"}
The reason is the type of 'uuid' is bytes and protobuf's bytes type is
base64-encoded.
'--debug/default' mode shows uuid's plain text while '--debug/default'
mode uses protobuf's API and shows base64-encoded text.
This patch fixes this problem.
Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
---
M src/kudu/fs/fs.proto
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_pbc.cc
M src/kudu/util/easy_json.cc
M src/kudu/util/easy_json.h
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
M src/kudu/util/pb_util.proto
8 files changed, 103 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/17
--
To view, visit http://gerrit.cloudera.org:8080/18330
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
Gerrit-Change-Number: 18330
Gerrit-PatchSet: 17
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
[kudu-CR] [java] add buffer space limit for KuduSession
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/19918 )
Change subject: [java] add buffer space limit for KuduSession
..
Patch Set 6:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG@11
PS6, Line 11: configuring
nit: configuring
http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG@11
PS6, Line 11: Buffer Space
buffer space
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@649
PS6, Line 649: isAboveMaxSize
what about isExcessMaxSize ?
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@894
PS6, Line 894: with
nit: from
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java
File
java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java@410
PS6, Line 410: while (session.hasPendingOperations()) {
If session.hasPendingOperations() is always true, it will be an infinity loop.
Our expecting is session.hasPendingOperations() = false very soon. So should
give it a limit time, assertEventuallyTrue is ok.
I think you can comments why sleep to wait at this
--
To view, visit http://gerrit.cloudera.org:8080/19918
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9
Gerrit-Change-Number: 19918
Gerrit-PatchSet: 6
Gerrit-Owner: KeDeng
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Wed, 14 Jun 2023 08:39:26 +
Gerrit-HasComments: Yes
[kudu-CR] [server] KUDU-1945 Auto incrementing column Upsert support
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20061 ) Change subject: [server] KUDU-1945 Auto_incrementing column Upsert support .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc File src/kudu/common/row_operations.cc: http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc@478 PS1, Line 478: op->type == RowOperationsPB_Type_UPSERT || : op->type == RowOperationsPB_Type_UPSERT_IGNORE RowOperationsPB_Type_UPSERT means RowOperationsPB_Type_INSERT or RowOperationsPB_Type_UPDATE, according to context, I guess it is for RowOperationsPB_Type_UPDATE. So should we add RowOperationsPB_Type_UPDATE at this? http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc@491 PS1, Line 491: "column value"); nit: alignment http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc@495 PS1, Line 495: // Check if the provided counter value is less than what is in memory : // and update the counter in memory. This auto-incrementing counter is set by the user at client? If a user set a smaller value, server side will ignore this set and change it to a bigger one using an obscure way for users. In kudu, this is correct, but for users it seems not good enough. Whether it is necessary to provide more response messages or return not ok? -- To view, visit http://gerrit.cloudera.org:8080/20061 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib5cc4d80f77c165452572948f68c76fc70394d47 Gerrit-Change-Number: 20061 Gerrit-PatchSet: 1 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 14 Jun 2023 08:07:02 + Gerrit-HasComments: Yes
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/18330 )
Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using
-json/-json_pretty
..
Patch Set 16:
(13 comments)
Thanks for your advices and I fixed them. You can review it again.
http://gerrit.cloudera.org:8080/#/c/18330/14//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18330/14//COMMIT_MSG@12
PS14, Line 12:
> It is better to describe the difference of the first command and the second
Done
http://gerrit.cloudera.org:8080/#/c/18330/14//COMMIT_MSG@13
PS14, Line 13:
> base64-encoded
Done
http://gerrit.cloudera.org:8080/#/c/18330/14//COMMIT_MSG@14
PS14, Line 14: . kudu pbc dump instance, which is a plain output format.
> Here you need to compare '--debug/default' mode with '-json/-json_pretty' m
I guess you mean I should paste the two commands output at this. So I do this.
http://gerrit.cloudera.org:8080/#/c/18330/14//COMMIT_MSG@15
PS14, Line 15:
> base64-encoded
Done
http://gerrit.cloudera.org:8080/#/c/18330/14//COMMIT_MSG@15
PS14, Line 15:
> API
Done
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.h
File src/kudu/util/pb_util.h:
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.h@422
PS14, Line 422: // 'BytesTextDumpFlag
> Should the name of this enum contain some reference on the context where it
Done
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.h@423
PS14, Line 423: when u
> That's not the best name for the enum tag. Maybe, name this as BASE64_ENCO
ok
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.h@427
PS14, Line 427: enum class BytesFieldDumpFlag {
> Use DECODE_UUID as the default value is a little confusing. There is a Dump
Yes. I fixed the names.
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc
File src/kudu/util/pb_util.cc:
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc@989
PS14, Line 989: const std::string kUuidField = "uuid";
> Instead of having this field name hard-coded here, consider something simil
I should study about this
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc@990
PS14, Line 990: decode it from base6
> decode it from base64 format
Done
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc@994
PS14, Line 994: // Maybe it's necessary to consider other 'bytes' fields
which can be dumped.
> If the whole message is encoded in base64 format, can we decode whole messa
No. Only bytes fields, in 'kudu pbc dump instance' is the uuid. Other field is
normal , not based64-encoded.
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc@1027
PS14, Line 1027: JsonFormat json_format = JsonFormat::JSON;
> nit: is it possible to move this to line 1024, to be after Format::JSON but
It's hard to make the two 'case' using a common variable except moving this
statement before switch.
L1029 has an example.
So, I think this statement at here it ok.
http://gerrit.cloudera.org:8080/#/c/18330/14/src/kudu/util/pb_util.cc@1044
PS14, Line 1044: RETURN_N
> Here and below: why not to use RETURN_NOT_OK() here? I.e., why to crash if
Yes. This CHECK is not necessary. I fix them.
--
To view, visit http://gerrit.cloudera.org:8080/18330
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
Gerrit-Change-Number: 18330
Gerrit-PatchSet: 16
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Tue, 13 Jun 2023 16:37:06 +
Gerrit-HasComments: Yes
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,
Abhishek Chennaka, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18330
to look at the new patch set (#16).
Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using
-json/-json_pretty
..
[tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty
The output of two commands below about the 'uuid' field are not the same
1. kudu pbc dump xxx/instance -debug/default
2. kudu pbc dump xxx/instance -json/-json_pretty
For example:
1. kudu pbc dump instance, which is a plain output format.
Message 0
---
uuid: "ac6b3e392cb34b4681e54a33c95fdf9c"
format_stamp: "Formatted at 2023-03-10 09:16:22 on 2a43db713d56"
2. kudu pbc dump instance --json, which is a json output format
{"uuid":"YWM2YjNlMzkyY2IzNGI0NjgxZTU0YTMzYzk1ZmRmOWM=","formatStamp":"Formatted
at 2023-03-10 09:16:22 on 2a43db713d56"}
The reason is the type of 'uuid' is bytes and protobuf's bytes type is
base64-encoded.
'--debug/default' mode shows uuid's plain text while '--debug/default'
mode uses protobuf's API and shows base64-encoded text.
This patch fixes this problem.
Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_pbc.cc
M src/kudu/util/easy_json.cc
M src/kudu/util/easy_json.h
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
6 files changed, 87 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/16
--
To view, visit http://gerrit.cloudera.org:8080/18330
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
Gerrit-Change-Number: 18330
Gerrit-PatchSet: 16
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,
Abhishek Chennaka, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18330
to look at the new patch set (#15).
Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using
-json/-json_pretty
..
[tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty
The output of two commands below about the 'uuid' field are not the same
1. kudu pbc dump xxx/instance -debug/default
2. kudu pbc dump xxx/instance -json/-json_pretty
For example:
1. kudu pbc dump instance, which is a plain output format.
Message 0
---
uuid: "ac6b3e392cb34b4681e54a33c95fdf9c"
format_stamp: "Formatted at 2023-03-10 09:16:22 on 2a43db713d56"
2. kudu pbc dump instance --json, which is a json output format
{"uuid":"YWM2YjNlMzkyY2IzNGI0NjgxZTU0YTMzYzk1ZmRmOWM=","formatStamp":"Formatted
at 2023-03-10 09:16:22 on 2a43db713d56"}
The reason is the type of 'uuid' is bytes and protobuf's bytes type is
base64-encoded.
'--debug/default' mode shows uuid's plain text while '--debug/default'
mode uses protobuf's API and shows base64-encoded text.
This patch fixes this problem.
Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_pbc.cc
M src/kudu/util/easy_json.cc
M src/kudu/util/easy_json.h
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
6 files changed, 87 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/15
--
To view, visit http://gerrit.cloudera.org:8080/18330
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
Gerrit-Change-Number: 18330
Gerrit-PatchSet: 15
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19794 ) Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. Patch Set 13: Verified+1 (7 comments) Thanks for your advices, I fixed them. http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner-test.cc File src/kudu/common/partition_pruner-test.cc: http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner-test.cc@96 PS12, Line 96: Return a bitset indicates which hash buckets are > How about: Done http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner-test.cc@193 PS12, Line 193: // The old algorithm. It moved from 'partition_pruner.cc'. > Is it copied from partition_pruner.cc? Yes. http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner-test.cc@984 PS12, Line 984: generate some in-list predicates for these key columns. The old algorith > How about add some tests before this patch, the tests will pass by the 'old It seems more confuse. I add some comments using other words. http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner.cc File src/kudu/common/partition_pruner.cc: http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner.cc@204 PS12, Line 204: // A combination of the predicate values is selected to compute the hash buckets. > Could you please add some comments to describe what do 'predicate_values_pi Done http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner.cc@206 PS12, Line 206: predicate_values_selected.reserve(hash_dimension.column_ids.size()); > nit: rename to 'hash_bucket_selected' ? Done http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner.cc@219 PS12, Line 219: > How about renaming it closer to what it stands for? Like 'all_hash_bucket_n Done http://gerrit.cloudera.org:8080/#/c/19794/12/src/kudu/common/partition_pruner.cc@224 PS12, Line 224: all_hash_bucket_needed = true; > If it has a more meaningful name, I gusee these comments can be omitted? Removed. -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 13 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Tue, 13 Jun 2023 08:55:57 + Gerrit-HasComments: Yes
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Hello Tidy Bot, Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, KeDeng, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19794 to look at the new patch set (#13). Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate This patch comes from a committed patch, its committed id is 'b69dbeb'. As that patch said, logic of pruning hash partitions for in-list predicate in Kudu cpp client has also a high space complexity and slow. Old algorithm must keep all intermedium objects because they are incomplete until they are completed and can be computed hash. This patch fixes the problems and provides a recursive algorithm the same as the one used by java client. Compared with java client, the cpp client is less likely to cause the OOM condition because it does not keep too many intermediate results. This optimization has good benefits too. The benefits are related to the in-list length and the number of primary keys, the performance would be better if in-list length is longer. For example, PartitionPrunerTest::TestMultiColumnInListHashPruningManyValues, using 10 key columns and kMaxInListLength=50, old algorithm memory cost may reach 600MB, while new algorithm's memory cost can be ignored (it only need one objects and a few stacks for contexts). At the same time, new algorithm has a good speedup, some effect as below: combination_count: 555400692, old cost: 428238us, new cost: 713us, speedup: 600.6x combination_count: 89083783664568, old cost: 2764924us, new cost: 1145us, speedup: 2414.7x combination_count: 27194091724800, old cost: 1610475us, new cost: 1151us, speedup: 1399.2x combination_count: 7116622216704, old cost: 34544289us, new cost: 375us, speedup: 92118.1x combination_count: 37570734489600, old cost: 1733205us, new cost: 901us, speedup: 1923.6x Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 --- M src/kudu/common/partition_pruner-test.cc M src/kudu/common/partition_pruner.cc M src/kudu/common/partition_pruner.h 3 files changed, 245 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19794/13 -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 13 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] KUDU-3371 [fs] make LogBlockContainer a base class
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19610 ) Change subject: KUDU-3371 [fs] make LogBlockContainer a base class .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19610 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieb199de95973aaeba76947ad095907272d84ca67 Gerrit-Change-Number: 19610 Gerrit-PatchSet: 4 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 08 Jun 2023 10:15:41 + Gerrit-HasComments: No
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/19794 )
Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up
hash partition pruning for in-list predicate
..
Patch Set 12: Verified+1
(4 comments)
http://gerrit.cloudera.org:8080/#/c/19794/11//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19794/11//COMMIT_MSG@16
PS11, Line 16: the one used by java
> nit: the one used by java client
Done
http://gerrit.cloudera.org:8080/#/c/19794/11/src/kudu/common/partition_pruner-test.cc
File src/kudu/common/partition_pruner-test.cc:
http://gerrit.cloudera.org:8080/#/c/19794/11/src/kudu/common/partition_pruner-test.cc@1081
PS11, Line 1081: temp_values.push_back(t_value);
: test_case.push_back(reinterpret_cast(_values[temp_values.size() - 1]));
> Can this be simplified to 'test_case.push_back(reinterpret_casthttp://gerrit.cloudera.org:8080/#/c/19794/11/src/kudu/common/partition_pruner-test.cc@1091
PS11, Line 1091: for (int i = 0; i < kKeyColumnSize; i++) {
:
predicates.emplace_back(ColumnPredicate::InList(schema.column(i),
_cases[i]));
: }
> Maybe this can be moved to the for loop at L1079? And then the 'test_cases'
Can not move it.
The purpose of lines L1075-L1087 is generating the test_cases, and
L1091-1093 use this cases to generator predicates.
http://gerrit.cloudera.org:8080/#/c/19794/11/src/kudu/common/partition_pruner.cc
File src/kudu/common/partition_pruner.cc:
http://gerrit.cloudera.org:8080/#/c/19794/11/src/kudu/common/partition_pruner.cc@224
PS11, Line 224: // All bits in the bitset have been set, we should keep all
partitions for
> nit: Need to document this short circuit condition.
DONE. Add a comment for this.
--
To view, visit http://gerrit.cloudera.org:8080/19794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9
Gerrit-Change-Number: 19794
Gerrit-PatchSet: 12
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Wed, 07 Jun 2023 02:47:02 +
Gerrit-HasComments: Yes
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Hello Tidy Bot, Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, KeDeng, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19794 to look at the new patch set (#12). Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate This patch comes from a committed patch, its committed id is 'b69dbeb'. As that patch said, logic of pruning hash partitions for in-list predicate in Kudu cpp client has also a high space complexity and slow. Old algorithm must keep all intermedium objects because they are incomplete until they are completed and can be computed hash. This patch fixes the problems and provides a recursive algorithm the same as the one used by java client. Compared with java client, the cpp client is less likely to cause the OOM condition because it does not keep too many intermediate results. This optimization has good benefits too. The benefits are related to the in-list length and the number of primary keys, the performance would be better if in-list length is longer. For example, PartitionPrunerTest::TestMultiColumnInListHashPruningManyValues, using 10 key columns and kMaxInListLength=50, old algorithm memory cost may reach 600MB, while new algorithm's memory cost can be ignored (it only need one objects and a few stacks for contexts). At the same time, new algorithm has a good speedup, some effect as below: combination_count: 555400692, old cost: 428238us, new cost: 713us, speedup: 600.6x combination_count: 89083783664568, old cost: 2764924us, new cost: 1145us, speedup: 2414.7x combination_count: 27194091724800, old cost: 1610475us, new cost: 1151us, speedup: 1399.2x combination_count: 7116622216704, old cost: 34544289us, new cost: 375us, speedup: 92118.1x combination_count: 37570734489600, old cost: 1733205us, new cost: 901us, speedup: 1923.6x Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 --- M src/kudu/common/partition_pruner-test.cc M src/kudu/common/partition_pruner.cc M src/kudu/common/partition_pruner.h 3 files changed, 240 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19794/12 -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 12 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [tools] remove the duplicated function GetReplicas
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19998 ) Change subject: [tools] remove the duplicated function GetReplicas .. Patch Set 1: Code-Review+2 LGTM -- To view, visit http://gerrit.cloudera.org:8080/19998 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I069997fa566d70fccf6df3f3decd1aa990417048 Gerrit-Change-Number: 19998 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 02 Jun 2023 10:05:53 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/19949 )
Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed
..
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19949/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19949/3//COMMIT_MSG@20
PS3, Line 20: This patch will validate the schema of multiple rows which belong
: to the same tablet. If the schema is different, it puts them into
: the different groups as different batches.
> Great! This bug is a serious problem. This way seems ok and there is somet
I have found another need a common schema when sending a write batch rpc.
```
static Tserver.WriteRequestPB.Builder
createAndFillWriteRequestPB(List operations) {
Schema schema = operations.get(0).table.getSchema();
Tserver.WriteRequestPB.Builder requestBuilder =
Tserver.WriteRequestPB.newBuilder();
requestBuilder.setSchema(ProtobufHelper.schemaToPb(schema,
EnumSet.of(SchemaPBConversionFlags.SCHEMA_PB_WITHOUT_COMMENT,
SchemaPBConversionFlags.SCHEMA_PB_WITHOUT_ID)));
```
If there are two different schemas, it is an error schema for some ops. So this
method may not be ok. I'll think it again.
--
To view, visit http://gerrit.cloudera.org:8080/19949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db
Gerrit-Change-Number: 19949
Gerrit-PatchSet: 3
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Fri, 02 Jun 2023 07:19:33 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/19949 )
Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed
..
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19949/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19949/3//COMMIT_MSG@21
PS3, Line 21: the
a
http://gerrit.cloudera.org:8080/#/c/19949/3//COMMIT_MSG@20
PS3, Line 20: This patch will validate the schema of multiple rows which belong
: to the same tablet. If the schema is different, it puts them into
: the different groups as different batches.
Great! This bug is a serious problem. This way seems ok and there is something
worried. I comment this at other comment.
I reviewed what you say, and provide another way to fix this bug. You can
review and considering it whether it is ok, like this:
```
// OperationsEncoder in Operation.java.
private void init(List operations) {
int rowsSize = 0;
int indirectSize = 0;
for (Operation op : operations) {
Schema schema = op.table.getSchema();
final int columnBitSetSize =
Bytes.getBitSetSize(schema.getColumnCount());
int sizePerRow = 1 /* for the op type */ + schema.getRowSize() +
columnBitSetSize;
if (schema.hasNullableColumns()) {
// nullsBitSet is the same size as the columnBitSet
sizePerRow += columnBitSetSize;
}
rowsSize += sizePerRow;
indirectSize += schema.getVarLengthColumnCount();
}
this.rows = ByteBuffer.allocate(rowsSize).order(ByteOrder.LITTLE_ENDIAN);
this.indirect = new ArrayList<>(indirectSize);
}
```
http://gerrit.cloudera.org:8080/#/c/19949/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19949/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@371
PS3, Line 371: Extral
I guess you mean 'Extra' ?
http://gerrit.cloudera.org:8080/#/c/19949/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@372
PS3, Line 372: extral_batches
extra_batches
http://gerrit.cloudera.org:8080/#/c/19949/3/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@419
PS3, Line 419: extral_batches
every op different from batch.operations.get(0) need a write rpc? It seems not
good.
If this op can merge into an extra batch, it may be better.
By the way, I am not sure a potential problem because I am not care about txn
in kudu. I worry about this fixing way may cause another problem, there is a
txn Id. This fixing maybe split a txn into 2 or more rpcs.
--
To view, visit http://gerrit.cloudera.org:8080/19949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db
Gerrit-Change-Number: 19949
Gerrit-PatchSet: 3
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Fri, 02 Jun 2023 04:14:21 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19949 ) Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/19949/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19949/1//COMMIT_MSG@7 PS1, Line 7: [KUDU-3483] as kudu's style, this would be better: [java-client] KUDU-3483 ... http://gerrit.cloudera.org:8080/#/c/19949/1//COMMIT_MSG@12 PS1, Line 12: server nit: servers http://gerrit.cloudera.org:8080/#/c/19949/1//COMMIT_MSG@16 PS1, Line 16: has nit: have http://gerrit.cloudera.org:8080/#/c/19949/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19949/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@411 PS1, Line 411: diffrent nit: different -- To view, visit http://gerrit.cloudera.org:8080/19949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db Gerrit-Change-Number: 19949 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 02 Jun 2023 03:04:08 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.17.x) [tests] fix flakiness in TestNoMoreRetryWithWongServerUuid
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19947 ) Change subject: [tests] fix flakiness in TestNoMoreRetryWithWongServerUuid .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19947 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I3d3dee89b32d1e33d1f0f41e8b83835b02eae336 Gerrit-Change-Number: 19947 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 31 May 2023 02:53:06 + Gerrit-HasComments: No
[kudu-CR](branch-1.17.x) KUDU-3248 improve replica selection initialization
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19948 ) Change subject: KUDU-3248 improve replica selection initialization .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I11c2a29ef69a8c97c68330d261fdff64accebb0b Gerrit-Change-Number: 19948 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 31 May 2023 02:53:35 + Gerrit-HasComments: No
[kudu-CR](branch-1.17.x) [client] fix clang warning
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19946 ) Change subject: [client] fix clang warning .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19946 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: Iae586b683e06a6af1e44c9859769309d3b6a1a94 Gerrit-Change-Number: 19946 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 31 May 2023 02:52:46 + Gerrit-HasComments: No
[kudu-CR](branch-1.17.x) [util] use include guard macro in int128.h header
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19945 ) Change subject: [util] use include guard macro in int128.h header .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19945 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: Ica18bbdc8eff034965e1a69404df2180cacf7bb4 Gerrit-Change-Number: 19945 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 31 May 2023 02:52:26 + Gerrit-HasComments: No
[kudu-CR] [duplication] KUDU-3290 add 3rd-party libraries
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#2). Change subject: [duplication] KUDU-3290 add 3rd-party libraries .. [duplication] KUDU-3290 add 3rd-party libraries This patch is the first patch for duplication, adding 2 3rd-party libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. For easy review, Split this ‘kudu duplication’ patch into several small independent patches: 1. add 3rd-party libraries, introduce librdkafka, cppkafka 2. Add duplication engine, support write to kafka 3. Add a new raft role Duplicator and raft processing 4. creating/altering in server-side for duplication 5. recovering duplication when a leader switches 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication 10. register kafka, unregister kafka and validate During this progressing, plans maybe adjust according to advices. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 7 files changed, 169 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/2 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 2 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [duplication] KUDU-3290 add thirdparty libraries
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19909 ) Change subject: [duplication] KUDU-3290 add thirdparty libraries .. Patch Set 1: (1 comment) > Patch Set 1: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/19909/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19909/1//COMMIT_MSG@26 PS1, Line 26: During this progressing, plans maybe adjust according to advices. > I guess the very first step should be publishing a design document and aski Ok. There is a document in jira KUDU-3290. https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit# It's not marked, so I add it in the first line, and I fix this document again, and you can review it. Of course, I have sent this design document to d...@kudu.apache.org to collect advices. About 3rd-party library, yes, you are right. This patch can stay here not commit, I will submit next patches based this patch. All patches may take long time, so now it's not urgent to commit to kudu. We can review them first and commit this patch when necessary. -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 1 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Tue, 30 May 2023 09:52:37 + Gerrit-HasComments: Yes
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, KeDeng, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18330 to look at the new patch set (#14). Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty The two commands below: 1. kudu pbc dump xxx/instance -debug/default 2. kudu pbc dump xxx/instance -json/-json_pretty These two commands' outputs about the 'uuid' field are not the same, because the type of 'uuid' is bytes and protobuf's bytes type is base64 encoded. '--debug/default' mode shows uuid's plain text while '--debug/default' mode uses protobuf's apis and shows base64 encoded text. This patch fixes this problem. Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_pbc.cc M src/kudu/util/easy_json.cc M src/kudu/util/easy_json.h M src/kudu/util/pb_util.cc M src/kudu/util/pb_util.h 6 files changed, 75 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/14 -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 14 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR](branch-1.17.x) [master] Exclude tservers in MAINTENANCE MODE when leader rebalancing
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19923 ) Change subject: [master] Exclude tservers in MAINTENANCE_MODE when leader rebalancing .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19923 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I2f85a675e69fd02a62e2625881dad2ca5e27acd9 Gerrit-Change-Number: 19923 Gerrit-PatchSet: 2 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 26 May 2023 06:45:11 + Gerrit-HasComments: No
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, KeDeng, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18330 to look at the new patch set (#13). Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty The two commands below: 1. kudu pbc dump xxx/instance -debug/default 2. kudu pbc dump xxx/instance -json/-json_pretty These two commands' outputs about the 'uuid' field are not the same, because the type of 'uuid' is bytes and protobuf's bytes type is base64 encoded. '--debug/default' mode shows uuid's plain text while '--debug/default' mode uses protobuf's apis and shows base64 encoded text. This patch fixes this problem. Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_pbc.cc M src/kudu/util/easy_json.cc M src/kudu/util/easy_json.h M src/kudu/util/pb_util.cc M src/kudu/util/pb_util.h 6 files changed, 75 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/13 -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 13 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [ut] Add '--log cleanup interval sec' flag and speedup an unit test
Hello Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19915 to look at the new patch set (#3). Change subject: [ut] Add '--log_cleanup_interval_sec' flag and speedup an unit test .. [ut] Add '--log_cleanup_interval_sec' flag and speedup an unit test LogRollingITest.TestLogCleanupOnStartup is a little slow, it takes 116s or so. This patch try to speed up this test and avoid coredump when flushing logs asynchronously. Adding '--log_cleanup_interval_sec' flag replaces the magic number kWait(60s), set it 6s in the unit test. It takes half(56s) of original time(116s). This new flag can be changed at runtime if needed. Independent related works before this patch: 1. https://gerrit.cloudera.org/c/19842/, this patch has merged and solved this unsteady unit test. This is an original aim of this patch. 2. https://gerrit.cloudera.org/c/19871/ by KeDeng, this patch has same idea with this patch, it's abandoned by KeDeng because of ut may be fail. By KeDeng' reminder, This patch fixed this potential problem. Change-Id: Ib2be9f88c5d47bc889a4711f5aa2143b8288ba60 --- M src/kudu/integration-tests/log-rolling-itest.cc M src/kudu/server/server_base.cc 2 files changed, 18 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/19915/3 -- To view, visit http://gerrit.cloudera.org:8080/19915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib2be9f88c5d47bc889a4711f5aa2143b8288ba60 Gerrit-Change-Number: 19915 Gerrit-PatchSet: 3 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [ut] Fix an unsteady unit test
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19915 ) Change subject: [ut] Fix an unsteady unit test .. Patch Set 2: > Patch Set 1: > > I submitted similar modifications > before,https://gerrit.cloudera.org/#/c/19871/. > > As Mahesh Reddy said in the reviews, this modification can still cause errors > during multiple tests. Thanks for your advice. I have study your case. And indeed your fix unfortunately, encountering another problem because of interval 3s, it's so small and cause coredump. If you set >= 5s. I have written details in comments. -- To view, visit http://gerrit.cloudera.org:8080/19915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib2be9f88c5d47bc889a4711f5aa2143b8288ba60 Gerrit-Change-Number: 19915 Gerrit-PatchSet: 2 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 24 May 2023 03:46:57 + Gerrit-HasComments: No
[kudu-CR] [ut] Fix an unsteady unit test
Hello Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19915 to look at the new patch set (#2). Change subject: [ut] Fix an unsteady unit test .. [ut] Fix an unsteady unit test LogRollingITest.TestLogCleanupOnStartup is a little unsteady. Because Deleting logs every 60s(kWait), but in this test, it may wait 30s(ASSERT_EVENTUALLY) at most. This patch fix it. Meanwhile, try to speed up this test, it takes half(56s) of original time(116s). Using FLAGS_log_cleanup_interval_sec replace the magic number kWait and it can be changed at runtime. Change-Id: Ib2be9f88c5d47bc889a4711f5aa2143b8288ba60 --- M src/kudu/integration-tests/log-rolling-itest.cc M src/kudu/server/server_base.cc 2 files changed, 12 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/19915/2 -- To view, visit http://gerrit.cloudera.org:8080/19915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib2be9f88c5d47bc889a4711f5aa2143b8288ba60 Gerrit-Change-Number: 19915 Gerrit-PatchSet: 2 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [ut] Fix an unsteady unit test
Yuqi Du has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19915 Change subject: [ut] Fix an unsteady unit test .. [ut] Fix an unsteady unit test I found ut: LogRollingITest.TestLogCleanupOnStartup is a little unsteady. Delete logs every 30s, but in this test, it may wait 30s at most. This patch fix it and try to speed up this test. Change-Id: Ib2be9f88c5d47bc889a4711f5aa2143b8288ba60 --- M src/kudu/integration-tests/log-rolling-itest.cc M src/kudu/server/server_base.cc 2 files changed, 15 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/19915/1 -- To view, visit http://gerrit.cloudera.org:8080/19915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib2be9f88c5d47bc889a4711f5aa2143b8288ba60 Gerrit-Change-Number: 19915 Gerrit-PatchSet: 1 Gerrit-Owner: Yuqi Du
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Yingchun Lai, Yifan Zhang, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18330 to look at the new patch set (#11). Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty The two commands below: 1. kudu pbc dump xxx/instance -debug/default 2. kudu pbc dump xxx/instance -json/-json_pretty These two commands' outputs about the 'uuid' field are not the same, because the type of 'uuid' is bytes and protobuf's bytes type is base64 encoded. '--debug/default' mode shows uuid's plain text while '--debug/default' mode uses protobuf's apis and shows base64 encoded text. This patch fixes this problem. Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_pbc.cc M src/kudu/util/easy_json.cc M src/kudu/util/easy_json.h M src/kudu/util/pb_util.cc M src/kudu/util/pb_util.h 6 files changed, 75 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/11 -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 11 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Yingchun Lai, Yifan Zhang, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18330 to look at the new patch set (#9). Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty The two commands below: 1. kudu pbc dump xxx/instance -debug/default 2. kudu pbc dump xxx/instance -json/-json_pretty These two commands' outputs about the 'uuid' field are not the same, because the type of 'uuid' is bytes and protobuf's bytes type is base64 encoded. '--debug/default' mode shows uuid's plain text while '--debug/default' mode uses protobuf's apis and shows base64 encoded text. This patch fixes this problem. Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_pbc.cc M src/kudu/util/easy_json.cc M src/kudu/util/easy_json.h M src/kudu/util/pb_util.cc M src/kudu/util/pb_util.h 6 files changed, 77 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/9 -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 9 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Yingchun Lai, Yifan Zhang, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18330 to look at the new patch set (#8). Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty .. [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty The two commands below: kudu pbc dump xxx/instance -debug kudu pbc dump xxx/instance -json/-json_pretty These two commands' outputs, the uuid is not the same, because uuid is bytes and pb' bytes is base64 encoded. This patch fixes this problem. Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_pbc.cc M src/kudu/util/easy_json.cc M src/kudu/util/easy_json.h M src/kudu/util/pb_util.cc M src/kudu/util/pb_util.h 6 files changed, 77 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/8 -- To view, visit http://gerrit.cloudera.org:8080/18330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf Gerrit-Change-Number: 18330 Gerrit-PatchSet: 8 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19794 ) Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. Patch Set 9: Code-Review+1 (2 comments) Thanks for your advices. http://gerrit.cloudera.org:8080/#/c/19794/9/src/kudu/common/partition_pruner.h File src/kudu/common/partition_pruner.h: http://gerrit.cloudera.org:8080/#/c/19794/9/src/kudu/common/partition_pruner.h@95 PS9, Line 95: // hash_bucket_bitset saves the hash buckets result. > nit: the result is stored in hash_bucket_bitset Done http://gerrit.cloudera.org:8080/#/c/19794/9/src/kudu/common/partition_pruner.cc File src/kudu/common/partition_pruner.cc: http://gerrit.cloudera.org:8080/#/c/19794/9/src/kudu/common/partition_pruner.cc@243 PS9, Line 243: CHECK_LT(level, predicate_values_list.size()); > CHECK_LT is used in release mode. Are there assertion similar used in debug Done -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 9 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 22 May 2023 07:27:23 + Gerrit-HasComments: Yes
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Hello Tidy Bot, Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, KeDeng, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19794 to look at the new patch set (#10). Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate This patch comes from a committed patch, its committed id is 'b69dbeb'. As that patch said, logic of pruning hash partitions for in-list predicate in Kudu cpp client has also a high space complexity and slow. Old algorithm must keep all intermedium objects because they are incomplete until they are completed and can be computed hash. This patch fixes the problems and provides a recursive algorithm the same as one using java client. Compared with java client, the cpp client is less likely to cause the OOM condition because it does not keep too many intermediate results. This optimization has good benefits too. The benefits are related to the in-list length and the number of primary keys, the performance would be better if in-list length is longer. For example, PartitionPrunerTest::TestMultiColumnInListHashPruningManyValues, using 10 key columns and kMaxInListLength=50, old algorithm memory cost may reach 600MB, while new algorithm's memory cost can be ignored (it only need one objects and a few stacks for contexts). At the same time, new algorithm has a good speedup, some effect as below: combination_count: 555400692, old cost: 428238us, new cost: 713us, speedup: 600.6x combination_count: 89083783664568, old cost: 2764924us, new cost: 1145us, speedup: 2414.7x combination_count: 27194091724800, old cost: 1610475us, new cost: 1151us, speedup: 1399.2x combination_count: 7116622216704, old cost: 34544289us, new cost: 375us, speedup: 92118.1x combination_count: 37570734489600, old cost: 1733205us, new cost: 901us, speedup: 1923.6x Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 --- M src/kudu/common/partition_pruner-test.cc M src/kudu/common/partition_pruner.cc M src/kudu/common/partition_pruner.h 3 files changed, 238 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19794/10 -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 10 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [duplication] KUDU-3290 add thirdparty libraries
Yuqi Du has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19909 Change subject: [duplication] KUDU-3290 add thirdparty libraries .. [duplication] KUDU-3290 add thirdparty libraries This patch is the first patch for duplication, adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. As my plans, I will commit about 9 commits for duplication. 1. add thirdparty libraries 2. write to kafka 3. new raft role Duplicator and raft processing 4. registe kafka uri and validate and unregiste 5. recovering duplication when a leader switches 6. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication During this progressing, plans maybe adjust according to advices. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 7 files changed, 169 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/1 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 1 Gerrit-Owner: Yuqi Du
