Hello Tidy Bot, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18350
to look at the new patch set (#12).
Change subject: WIP KUDU-3290 support duplication to kafka
......................................................................
WIP KUDU-3290 support duplication to kafka
The work is in progress, I hope someone
can review the scheme, and welcome your advisors.
Maybe new thirdparty deps: librdkafka, cppkafka lead to compile failed.
The issue: https://issues.apache.org/jira/browse/KUDU-3290
Design document:
https://docs.google.com/document/d/1ihqPFO1vulpYDYYKcHmhes0LCOXKQtdAf5ub0y2LiaM/edit?usp=sharing
Add a new MemberType: Duplicator, it can be thinked as leader's shadows(now
only one duplicator).
Add thirdparty for librdkafka and cppkafka.
Add thread pool for duplicator and duplicator replay wal thread pool, add kafka
client for duplication.
Support create table with duplictor, java and cpp
Support alter table for enable/disable duplication, java and cpp
Fix unit tests and add unit tests, Add itest for duplicate to kafka,
Support kerberos for kafka client
Conflicts:
src/kudu/tablet/ops/write_op.cc
Change-Id: I842476216f09150383d06c7a6f4c53bf7f2ec018
---
M CMakeLists.txt
A cmake_modules/FindCppKafka.cmake
A cmake_modules/FindRdKafka.cmake
M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java
M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java
M java/kudu-proto/build.gradle
M src/kudu/client/client.cc
M src/kudu/client/client.h
M src/kudu/client/replica-internal.cc
M src/kudu/client/replica-internal.h
M src/kudu/client/replica_controller-internal.cc
M src/kudu/client/replica_controller-internal.h
M src/kudu/client/scan_token-internal.cc
M src/kudu/client/table_alterer-internal.cc
M src/kudu/client/table_alterer-internal.h
M src/kudu/client/table_creator-internal.cc
M src/kudu/client/table_creator-internal.h
M src/kudu/common/types-test.cc
M src/kudu/common/types.cc
M src/kudu/common/types.h
M src/kudu/consensus/consensus.proto
M src/kudu/consensus/consensus_meta.cc
M src/kudu/consensus/consensus_meta.h
M src/kudu/consensus/consensus_meta_manager.cc
M src/kudu/consensus/consensus_meta_manager.h
M src/kudu/consensus/consensus_queue.cc
M src/kudu/consensus/metadata.proto
M src/kudu/consensus/peer_manager.cc
M src/kudu/consensus/quorum_util.cc
M src/kudu/consensus/quorum_util.h
M src/kudu/consensus/raft_consensus.cc
M src/kudu/consensus/raft_consensus.h
A src/kudu/duplicator/CMakeLists.txt
A src/kudu/duplicator/connector.h
A src/kudu/duplicator/duplication_replay-test.cc
A src/kudu/duplicator/duplication_replay.cc
A src/kudu/duplicator/duplication_replay.h
A src/kudu/duplicator/duplicator.cc
A src/kudu/duplicator/duplicator.h
A src/kudu/duplicator/kafka/kafka.proto
A src/kudu/duplicator/kafka/kafka_connector-test.cc
A src/kudu/duplicator/kafka/kafka_connector.cc
A src/kudu/duplicator/kafka/kafka_connector.h
A src/kudu/duplicator/wal_file_reader.h
M src/kudu/integration-tests/CMakeLists.txt
A src/kudu/integration-tests/duplication_with_kafka-itest.cc
M src/kudu/kserver/kserver.cc
M src/kudu/kserver/kserver.h
M src/kudu/master/catalog_manager.cc
M src/kudu/master/master.proto
M src/kudu/master/sys_catalog.cc
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/mini-cluster/external_mini_cluster.h
M src/kudu/rebalance/cluster_status.h
M src/kudu/tablet/CMakeLists.txt
M src/kudu/tablet/local_tablet_writer.h
M src/kudu/tablet/ops/alter_schema_op.h
A src/kudu/tablet/ops/duplication_op.cc
A src/kudu/tablet/ops/duplication_op.h
M src/kudu/tablet/ops/op.h
M src/kudu/tablet/ops/op_driver.cc
M src/kudu/tablet/ops/op_driver.h
M src/kudu/tablet/ops/op_tracker-test.cc
M src/kudu/tablet/ops/op_tracker.cc
M src/kudu/tablet/ops/participant_op.h
M src/kudu/tablet/ops/write_op.cc
M src/kudu/tablet/ops/write_op.h
M src/kudu/tablet/tablet-harness.h
M src/kudu/tablet/tablet.cc
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_bootstrap-test.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_replica-test-base.cc
M src/kudu/tablet/tablet_replica-test.cc
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tools/ksck.cc
M src/kudu/tools/ksck.h
M src/kudu/tools/tool_action_perf.cc
M src/kudu/transactions/txn_status_tablet.cc
M src/kudu/tserver/tablet_copy_source_session-test.cc
M src/kudu/tserver/tablet_server.cc
M src/kudu/tserver/tablet_server.h
M src/kudu/tserver/tablet_server_runner.cc
M src/kudu/tserver/tablet_service.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
M src/kudu/util/blocking_queue.h
M thirdparty/build-definitions.sh
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
A thirdparty/kafka-simple-control.sh
M thirdparty/vars.sh
93 files changed, 4,228 insertions(+), 161 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/18350/12
--
To view, visit http://gerrit.cloudera.org:8080/18350
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I842476216f09150383d06c7a6f4c53bf7f2ec018
Gerrit-Change-Number: 18350
Gerrit-PatchSet: 12
Gerrit-Owner: Yuqi Du <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yuqi Du <[email protected]>
