[
https://issues.apache.org/jira/browse/BEAM-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Mingmin updated BEAM-1407:
-----------------------------
Description:
enhance KafkaIO to work with Kafka client 0.9 and 0.10, (maybe 0.8 as well ?),
1). to fully leverage new features in each version, like external
authentication, timestamp in 0.10;
2). hide kafka API changed, to support steamless switch between different
cluster versions;
Scope of change:
1). add an abstract API for Kafka Consumer in existing KafkaIO. By default it's
kafka 0.9 to keep compatible, with an option for extension;
2). Wrapped API for Kafka 0.10, with Kafka message timestamp support;
3). wrapped API for Kafka 0.8 (TBD);
was:
enhance KafkaIO to work with Kafka client 0.9 and 0.10, to fully leverage new
features in 0.10: external authentication, timestamp.
Scope of change:
1). Add an adapter layer to hide the API differences between Kafka0.9 and 0.10;
2). use Kafka message timestamp as default in mode 0.10;
> Support multiple Kafka client in KakaIO
> ---------------------------------------
>
> Key: BEAM-1407
> URL: https://issues.apache.org/jira/browse/BEAM-1407
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Reporter: Xu Mingmin
> Assignee: Xu Mingmin
>
> enhance KafkaIO to work with Kafka client 0.9 and 0.10, (maybe 0.8 as well
> ?),
> 1). to fully leverage new features in each version, like external
> authentication, timestamp in 0.10;
> 2). hide kafka API changed, to support steamless switch between different
> cluster versions;
> Scope of change:
> 1). add an abstract API for Kafka Consumer in existing KafkaIO. By default
> it's kafka 0.9 to keep compatible, with an option for extension;
> 2). Wrapped API for Kafka 0.10, with Kafka message timestamp support;
> 3). wrapped API for Kafka 0.8 (TBD);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
