AHeise commented on a change in pull request #16701:
URL: https://github.com/apache/flink/pull/16701#discussion_r683292583
##########
File path:
flink-core/src/main/java/org/apache/flink/api/connector/sink/Sink.java
##########
@@ -46,44 +54,81 @@
public interface Sink<InputT, CommT, WriterStateT, GlobalCommT> extends
Serializable {
/**
- * Create a {@link SinkWriter}.
+ * Create a {@link SinkWriter}. If the application is resumed from a
checkpoint or savepoint and
+ * the sink is stateful, it will receive the corresponding state obtained
with {@link
+ * SinkWriter#snapshotState()} and serialized with {@link
#getWriterStateSerializer()}. If no
+ * state exists, the first existing, compatible state specified in {@link
+ * #getCompatibleStateNames()} will be loaded and passed.
*
* @param context the runtime context.
- * @param states the writer's state.
+ * @param states the writer's previous state.
* @return A sink writer.
- * @throws IOException if fail to create a writer.
+ * @throws IOException for any failure during creation.
+ * @see SinkWriter#snapshotState()
+ * @see #getWriterStateSerializer()
+ * @see #getCompatibleStateNames()
*/
SinkWriter<InputT, CommT, WriterStateT> createWriter(
InitContext context, List<WriterStateT> states) throws IOException;
/**
- * Creates a {@link Committer}.
+ * Any stateful sink needs to provide this state serializer and implement
{@link
+ * SinkWriter#snapshotState()} properly. The respective state is used in
{@link
+ * #createWriter(InitContext, List)} on recovery.
+ *
+ * @return the serializer of the writer's state type.
+ */
+ Optional<SimpleVersionedSerializer<WriterStateT>>
getWriterStateSerializer();
+
+ /**
+ * Creates a {@link Committer} which is part of a 2-phase-commit protocol.
The {@link
+ * SinkWriter} creates committables through {@link
SinkWriter#prepareCommit(boolean)} in the
+ * first phase. The committables are then passed to this committer and
persisted with {@link
+ * Committer#commit(List)}. If a committer is returned, the sink must also
return a {@link
+ * #getCommittableSerializer()}.
*
- * @return A committer.
- * @throws IOException if fail to create a committer.
+ * @return A committer for the 2-phase-commit protocol.
+ * @throws IOException for any failure during creation.
*/
Optional<Committer<CommT>> createCommitter() throws IOException;
/**
- * Creates a {@link GlobalCommitter}.
+ * Creates a {@link GlobalCommitter} which is part of a 2-phase-commit
protocol. The {@link
+ * SinkWriter} creates committables through {@link
SinkWriter#prepareCommit(boolean)} in the
+ * first phase. The committables are then passed to the Committer and
persisted with {@link
+ * Committer#commit(List)} which also can return an aggregated
committable. This aggregated
+ * committables are passed to this {@link GlobalCommitter} of which only a
single instance
+ * exists. If a global committer is returned, the sink must also return a
{@link
+ * #getCommittableSerializer()} and {@link
#getGlobalCommittableSerializer()}.
*
- * @return A global committer.
- * @throws IOException if fail to create a global committer.
+ * @return A global committer for the 2-phase-commit protocol.
+ * @throws IOException for any failure during creation.
*/
Optional<GlobalCommitter<CommT, GlobalCommT>> createGlobalCommitter()
throws IOException;
- /** Returns the serializer of the committable type. */
+ /**
+ * Returns the serializer of the committable type. The serializer is
required iff the sink has a
+ * {@link Committer} or {@link GlobalCommitter}.
+ */
Optional<SimpleVersionedSerializer<CommT>> getCommittableSerializer();
- /** Returns the serializer of the aggregated committable type. */
+ /**
+ * Returns the serializer of the aggregated committable type. The
serializer is required iff the
+ * sink has a {@link GlobalCommitter}.
+ */
Optional<SimpleVersionedSerializer<GlobalCommT>>
getGlobalCommittableSerializer();
- /** Return the serializer of the writer's state type. */
- Optional<SimpleVersionedSerializer<WriterStateT>>
getWriterStateSerializer();
+ /**
+ * A list of state names of sinks from which the state can be restored.
For example, the new
+ * file sink can resume from the state of an old streaming file sink as a
drop-in replacement
Review comment:
I can't exactly use a JavaDoc link because of circular dependency
issues. But I have used the proper class names in the code tag to make it more
obvious.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
