Yuqi Du has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18350
Change subject: WIP KUDU-3290 support duplication to kafka ...................................................................... WIP KUDU-3290 support duplication to kafka Although the work is in progress, I also hope someone can review the scheme, and welcome your advisors. The issue: https://issues.apache.org/jira/browse/KUDU-3290 Design document: https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing Add a new MemberType: Duplicator. Add thirdparty for librdkafka and cppkafka. Add thread pool for duplicator, add kafka client for duplication. Support create table with duplictor, java and cpp Support alter table for enable/disable duplication, java and cpp Fix unit tests and add unit tests, Add itest for duplicate to kafka, Support kerberos for kafka client Conflicts: src/kudu/tablet/ops/write_op.cc Change-Id: I842476216f09150383d06c7a6f4c53bf7f2ec018 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/replica_controller-internal.cc M src/kudu/client/replica_controller-internal.h M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/client/table_creator-internal.cc M src/kudu/client/table_creator-internal.h M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/consensus_meta.cc M src/kudu/consensus/consensus_meta.h M src/kudu/consensus/metadata.proto M src/kudu/consensus/quorum_util.cc M src/kudu/consensus/quorum_util.h M src/kudu/consensus/raft_consensus.h A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/duplicator/kafka/mock_kafka_connector.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/duplication_with_kafka-itest.cc M src/kudu/integration-tests/external_mini_cluster-itest-base.cc M src/kudu/kserver/kserver.cc M src/kudu/kserver/kserver.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/master/sys_catalog.cc M src/kudu/mini-cluster/external_mini_cluster.cc M src/kudu/mini-cluster/external_mini_cluster.h M src/kudu/rebalance/cluster_status.h M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/local_tablet_writer.h M src/kudu/tablet/ops/op_driver.cc M src/kudu/tablet/ops/write_op.cc M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_bootstrap-test.cc M src/kudu/tablet/tablet_bootstrap.cc M src/kudu/tablet/tablet_replica-test-base.cc M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tools/ksck.cc M src/kudu/tools/ksck.h M src/kudu/tools/tool_action_perf.cc M src/kudu/transactions/txn_status_tablet.cc M src/kudu/tserver/tablet_copy_source_session-test.cc M src/kudu/tserver/tablet_service.cc M src/kudu/tserver/ts_tablet_manager.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh A thirdparty/kafka-simple-control.sh M thirdparty/vars.sh 68 files changed, 2,612 insertions(+), 75 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/18350/1 -- To view, visit http://gerrit.cloudera.org:8080/18350 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I842476216f09150383d06c7a6f4c53bf7f2ec018 Gerrit-Change-Number: 18350 Gerrit-PatchSet: 1 Gerrit-Owner: Yuqi Du <[email protected]>
