[
https://issues.apache.org/jira/browse/FLINK-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104573#comment-16104573
]
ASF GitHub Bot commented on FLINK-7265:
---------------------------------------
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4397#discussion_r130027082
--- Diff:
flink-core/src/main/java/org/apache/flink/core/fs/ConsistencyLevel.java ---
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.core.fs;
+
+import org.apache.flink.annotation.PublicEvolving;
+
+/**
+ * An enumeration describing the level of consistency offered by a {@link
FileSystem}.
+ *
+ * <p>The consistency levels described here make statements about the
visibility
+ * of file existence and file contents in the presence of <i>new file
creation</i>,
+ * <i>file deletion</i>, <i>file renaming</i>, and <i>directory
listing</i>.
+ *
+ * <p>An operation is defined as consistent if the following holds: After
the function
+ * call triggering the operation returns, its result is immediately
reflected in
+ * in the view presented to any other party calling a file system function.
+ *
+ * <p>Please note that these levels do not make any statements about the
effects or visibility of
+ * file content modification or file appends. In fact, content
modification or appending are
+ * not supported in various file systems.
+ *
+ * <p>Some of these consistency levels indicate that the storage system
does not actually
+ * qualify to be called a FileSystem, but rather a blob-/object store.
+ */
+@PublicEvolving
+public enum ConsistencyLevel {
--- End diff --
Have you thought about making the individual properties fields of the enum,
i.e. have fields for `newFileCreation`, `fileDeletion`, `fileRenaming`,
`directoryListing`. Then `READ_AFTER_CREATE` would become, for example,
`READ_AFTER_CREATE(CONSISTENT, NOT_CONSISTENT, NOT_CONSISTENT,
NOT_CONSISTENT)`, and so on. I'm not sure whether the fields would be booleans
or another enum themselves. It seems there is at least `CONSISTENT`,
`NOT_CONSISTENT`, and `ATOMIC`.
That's just something to consider when merging this on master. I think we
can merge as is to fix the problem people are having with checkpointing to S3.
> FileSystems should describe their kind and consistency level
> ------------------------------------------------------------
>
> Key: FLINK-7265
> URL: https://issues.apache.org/jira/browse/FLINK-7265
> Project: Flink
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.3.1
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Fix For: 1.4.0, 1.3.2
>
>
> Currently, all {{FileSystem}} types look the same to Flink.
> However, certain operations should only be executed on certain kinds of file
> systems.
> For example, it makes no sense to attempt to delete an empty parent directory
> on S3, because there are no such thinks as directories, only hierarchical
> naming in the keys (file names).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
