[
https://issues.apache.org/jira/browse/FLINK-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070319#comment-16070319
]
ASF GitHub Bot commented on FLINK-6988:
---------------------------------------
GitHub user pnowojski opened a pull request:
https://github.com/apache/flink/pull/4239
[FLINK-6988] Initial flink-connector-kafka-0.11 with at-least-once semantic
Couple of first commits are from other PRs #4206 #4209 #4213
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pnowojski/flink kafka011
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4239.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4239
----
commit 5191d5b4b78620cfc5ecfc9088afba0d611eaacb
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-06-26T09:28:51Z
[FLINK-6996] Refactor and automaticall inherit KafkaProducer integration
tests
commit 1c7d349ce425ec0213059e062f10c90773cc780d
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-06-26T10:20:36Z
[FLINK-6996] Fix formatting in KafkaConsumerTestBase and
KafkaProducerTestBase
commit 5b849f98191439e69ca2357a4767f47957ee0250
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-06-23T11:41:55Z
[FLINK-7030] Build with scala-2.11 by default
commit 3f62aecb57cea9d43611ecfa24e2233a63197341
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-06-26T10:36:40Z
[FLINK-6996] Fix at-least-once semantic for FlinkKafkaProducer010
Add tests coverage for Kafka 0.10 and 0.9
commit 4b78626df474a8d49a406714a7142ad44d8a8faf
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-06-28T18:30:08Z
[FLINK-7032] Overwrite inherited values of compiler version from parent pom
Default values were 1.6 and were causing Intellij to constantly switch
language
level to 1.6, which in turn was causing compilation errors. It worked fine
for compiling from console using maven, because those values are separetly
set
in maven-compiler-plugin configuration.
commit 2c2556e72dd73c5e470e5afd6dab4a11cb41772d
Author: Piotr Nowojski <piotr.nowoj...@gmail.com>
Date: 2017-06-23T07:14:28Z
[FLINK-6988] Initial flink-connector-kafka-0.11 with at-least-once semantic
Code of 0.11 connector is based on 0.10 version
----
> Add Apache Kafka 0.11 connector
> -------------------------------
>
> Key: FLINK-6988
> URL: https://issues.apache.org/jira/browse/FLINK-6988
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Affects Versions: 1.3.1
> Reporter: Piotr Nowojski
> Assignee: Piotr Nowojski
>
> Kafka 0.11 (it will be released very soon) add supports for transactions.
> Thanks to that, Flink might be able to implement Kafka sink supporting
> "exactly-once" semantic. API changes and whole transactions support is
> described in
> [KIP-98|https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging].
> The goal is to mimic implementation of existing BucketingSink. New
> FlinkKafkaProducer011 would
> * upon creation begin transaction, store transaction identifiers into the
> state and would write all incoming data to an output Kafka topic using that
> transaction
> * on `snapshotState` call, it would flush the data and write in state
> information that current transaction is pending to be committed
> * on `notifyCheckpointComplete` we would commit this pending transaction
> * in case of crash between `snapshotState` and `notifyCheckpointComplete` we
> either abort this pending transaction (if not every participant successfully
> saved the snapshot) or restore and commit it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
