[
https://issues.apache.org/jira/browse/FLINK-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940029#comment-14940029
]
ASF GitHub Bot commented on FLINK-2727:
---------------------------------------
Github user mxm commented on a diff in the pull request:
https://github.com/apache/flink/pull/1163#discussion_r40933422
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/MessageAcknowledingSourceBase.java
---
@@ -0,0 +1,173 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.functions.source;
+
+import org.apache.flink.api.common.ExecutionConfig;
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.api.common.typeutils.TypeSerializer;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.api.java.typeutils.TypeExtractor;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.checkpoint.CheckpointNotifier;
+import org.apache.flink.streaming.api.checkpoint.Checkpointed;
+import org.apache.flink.streaming.api.functions.source.RichSourceFunction;
+import org.apache.flink.streaming.api.state.SerializedCheckpointData;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+
+/**
+ * Abstract base class for data sources that receive elements from a
message queue and
+ * acknowledge them back by IDs.
+ * <p>
+ * The mechanism for this source assumes that messages are identified by a
unique ID.
+ * When messages are taken from the message queue, the message must not be
dropped immediately,
+ * but must be retained until acknowledged. Messages that are not
acknowledged within a certain
+ * time interval will be served again (to a different connection,
established by the recovered source).
+ * <p>
+ * Note that this source can give no guarantees about message order in teh
case of failures,
--- End diff --
Typo: teh
> Add a base class for MessageQueue-with-acknowledgement sources
> --------------------------------------------------------------
>
> Key: FLINK-2727
> URL: https://issues.apache.org/jira/browse/FLINK-2727
> Project: Flink
> Issue Type: New Feature
> Components: Streaming
> Affects Versions: 0.10
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
>
> Several message queues (RabbitMQ, Amazon SQS) have the pattern that you
> retrieve messages and acknowledge them back by ID.
> We can create a simple base non-parallel source that provides tooling for:
> - Collecting the IDs of elements emitted between two checkpoints
> - Persisting them with the checkpoint, respecting proper serialization
> - Acknowledging them when a checkpoint is notified of completion.
> This assumes that the Message Queues retain unacknowledged messages and
> re-emit them after the acknowledgement period expired.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
