[
https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568672#comment-16568672
]
Mike Wingert commented on HBASE-15320:
--------------------------------------
[~stack], thanks for the feedback.
* When it starts the region server, I'm passing in these parameters:
hbase.cluster.distributed=true
zookeeper.znode.parent=/kafkaproxy
hbase.regionserver.port=17020
hbase.regionserver.info.port=17010
hbase.client.connection.impl=org.apache.hadoop.hbase.kafka.KafkaBridgeConnection
hbase.regionserver.admin.service=false
hbase.regionserver.client.service=false
hbase.wal.provider=org.apache.hadoop.hbase.wal.DisabledWALProvider
hbase.regionserver.workers=false
hfile.block.cache.size=0.0001
hbase.mob.file.cache.size=0
hbase.masterless=true
hbase.regionserver.metahandler.count=1
hbase.regionserver.replication.handler.count=1
hbase.regionserver.handler.count=1
hbase.ipc.server.read.threadpool.size=3
Are there other values I need to set?
* it allows you to specify various rules to route the replication messages or
drop them
Yes, it's read from the file in the conf dir (or specified via command line).
In my patch for hbase-21002 I'll add a readme to describe how to use the proxy.
> HBase connector for Kafka Connect
> ---------------------------------
>
> Key: HBASE-15320
> URL: https://issues.apache.org/jira/browse/HBASE-15320
> Project: HBase
> Issue Type: New Feature
> Components: Replication
> Reporter: Andrew Purtell
> Assignee: Mike Wingert
> Priority: Major
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: 15320.master.16.patch, 15320.master.16.patch,
> HBASE-15320.master.1.patch, HBASE-15320.master.10.patch,
> HBASE-15320.master.11.patch, HBASE-15320.master.12.patch,
> HBASE-15320.master.14.patch, HBASE-15320.master.15.patch,
> HBASE-15320.master.2.patch, HBASE-15320.master.3.patch,
> HBASE-15320.master.4.patch, HBASE-15320.master.5.patch,
> HBASE-15320.master.6.patch, HBASE-15320.master.7.patch,
> HBASE-15320.master.8.patch, HBASE-15320.master.8.patch,
> HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf
>
>
> Implement an HBase connector with source and sink tasks for the Connect
> framework (http://docs.confluent.io/2.0.0/connect/index.html) available in
> Kafka 0.9 and later.
> See also:
> http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines
> An HBase source
> (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task)
> could be implemented as a replication endpoint or WALObserver, publishing
> cluster wide change streams from the WAL to one or more topics, with
> configurable mapping and partitioning of table changes to topics.
> An HBase sink task
> (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would
> persist, with optional transformation (JSON? Avro?, map fields to native
> schema?), Kafka SinkRecords into HBase tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
