xintongsong commented on code in PR #25226:
URL: https://github.com/apache/flink/pull/25226#discussion_r1764718950
##########
flink-core/src/main/java/org/apache/flink/core/fs/IFileSystem.java:
##########
@@ -237,6 +238,26 @@ default RecoverableWriter createRecoverableWriter() throws
IOException {
"This file system does not support recoverable writers.");
}
+ /**
+ * Creates a new {@link RecoverableWriter}. A recoverable writer creates
streams that can
+ * persist and recover their intermediate state. Persisting and recovering
intermediate state is
+ * a core building block for writing to files that span multiple
checkpoints.
+ *
+ * <p>The returned object can act as a shared factory to open and recover
multiple streams.
+ *
+ * <p>This method is optional on file systems and various file system
implementations may not
+ * support this method, throwing an {@code UnsupportedOperationException}.
+ *
+ * @param conf Map contains a flag to indicate whether the writer should
not write to local
+ * storage. and can provide more information to instantiate the writer.
+ * @return A RecoverableWriter for this file system.
+ * @throws IOException Thrown, if the recoverable writer cannot be
instantiated.
+ */
+ default RecoverableWriter createRecoverableWriter(Map<String, String>
conf) throws IOException {
+ throw new UnsupportedOperationException(
+ "This file system does not support recoverable writers that
does not write on local machine.");
+ }
Review Comment:
I think there's no need to throw an exception on this. With the map, the
caller can pass in arbitrary k-v pairs. FileSystem implementations only need to
pick up configs that they know about, and can simply ignore the others. If the
FileSystem implementation doesn't want to ready any custom configs from the
map, it can simply ignore all of them. From this perspective, I think the
default behavior should be just ignore the map and fallback to the non-argument
`createRecoverableWriter`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
