[
https://issues.apache.org/jira/browse/ROCKETMQ-82?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16342083#comment-16342083
]
ASF GitHub Bot commented on ROCKETMQ-82:
----------------------------------------
vesense opened a new pull request #45: ROCKETMQ-82: RocketMQ-Flink Integration
URL: https://github.com/apache/rocketmq-externals/pull/45
# What is the purpose of the change
RocketMQ integration for [Apache Flink](https://flink.apache.org/). This
module includes the RocketMQ source and sink that allows a flink job to either
write messages into a topic or read from topics in a flink job.
https://issues.apache.org/jira/browse/ROCKETMQ-82
# Brief changelog
* RocketMQSource - The RocketMQSource is based on RocketMQ pull consumer
mode, and provides exactly once reliability guarantees when checkpoints are
enabled.
Otherwise, the source doesn't provide any reliability guarantees.
* RocketMQSink - The RocketMQSink provides at-least-once reliability
guarantees when checkpoints are enabled and `withBatchFlushOnCheckpoint(true)`
is set.
Otherwise, the sink reliability guarantees depends on rocketmq producer's
retry policy, for this case, the messages sending way is sync by default,
but you can change it by invoking `withAsync(true)`.
* KeyValueDeserializationSchema - The main API for deserializing topic and
tags is the
`org.apache.rocketmq.flink.common.serialization.KeyValueDeserializationSchema`
interface.
`rocketmq-flink` includes general purpose `KeyValueDeserializationSchema`
implementations called `SimpleKeyValueDeserializationSchema`.
* KeyValueSerializationSchema - The main API for serializing topic and tags
is the
`org.apache.rocketmq.flink.common.serialization.KeyValueSerializationSchema`
interface.
`rocketmq-flink` includes general purpose `KeyValueSerializationSchema`
implementations called `SimpleKeyValueSerializationSchema`.
* TopicSelector - The main API for selecting topic and tags is the
`org.apache.rocketmq.flink.common.selector.TopicSelector` interface.
`rocketmq-flink` includes general purpose `TopicSelector` implementations
called `DefaultTopicSelector` and `SimpleTopicSelector`.
* RocketMQFlinkExample - which receive messages from RocketMQ brokers and
send messages to broker after processing.
# Verifying this change
local tests check passed ✅
apache-rat check passed ✅
checkstyle check passed ✅
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add the RocketMQ plugin for the Apache Flink
> --------------------------------------------
>
> Key: ROCKETMQ-82
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-82
> Project: Apache RocketMQ
> Issue Type: Wish
> Components: rocketmq-externals
> Reporter: Longda Feng
> Assignee: Xin Wang
> Priority: Minor
> Labels: gsoc2018
>
> A few basic data sources and sinks are built into Flink and are always
> available. The [predefined data
> sources|https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html#data-sources]
> include reading from files, directories, and sockets, and ingesting data
> from collections and iterators. The [predefined data
> sinks|https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html#data-sinks]
> support writing to files, to stdout and stderr, and to sockets.
> The connector is the integration plugin for RocketMQ and Flink, some details
> in here,
> 1.
> [https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/index.html]
> 2. [https://github.com/apache/rocketmq-externals/tree/master/rocketmq-flink]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
