[
https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054500#comment-16054500
]
Mike Wingert commented on HBASE-15320:
--------------------------------------
Thanks for the feedback!
> Please start patches against the master branch. We can figure out what
> release lines a new feature can get into once things get hammered out more.
Sorry about that, I'll re-work the patch.
> What do we gain by running edits through the proxy servers rather than just
> putting directly into the kafka broker directly from the replication system?
I wanted to isolate the kafka logic out of the region server completely, in
case something happened on the kafka side. I also wanted to use the
replication peer system as a way to buffer things in case kafka slows down.
> How about some docs?
Docs are on the way =-)
> HBase connector for Kafka Connect
> ---------------------------------
>
> Key: HBASE-15320
> URL: https://issues.apache.org/jira/browse/HBASE-15320
> Project: HBase
> Issue Type: New Feature
> Components: Replication
> Reporter: Andrew Purtell
> Assignee: Mike Wingert
> Labels: beginner
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-15320.txt
>
>
> Implement an HBase connector with source and sink tasks for the Connect
> framework (http://docs.confluent.io/2.0.0/connect/index.html) available in
> Kafka 0.9 and later.
> See also:
> http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines
> An HBase source
> (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task)
> could be implemented as a replication endpoint or WALObserver, publishing
> cluster wide change streams from the WAL to one or more topics, with
> configurable mapping and partitioning of table changes to topics.
> An HBase sink task
> (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would
> persist, with optional transformation (JSON? Avro?, map fields to native
> schema?), Kafka SinkRecords into HBase tables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
