[
https://issues.apache.org/jira/browse/FLINK-37439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17933726#comment-17933726
]
Zakelly Lan commented on FLINK-37439:
-------------------------------------
Merge into master: 67b02e48 ... c7aa1748
2.0: f2b5e28b ... 9baec0bc
> Missing file creation with overwrite semantic in ForSt wrapped file system
> --------------------------------------------------------------------------
>
> Key: FLINK-37439
> URL: https://issues.apache.org/jira/browse/FLINK-37439
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 2.0.0
> Reporter: Zakelly Lan
> Assignee: Zakelly Lan
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.0.0
>
>
> ForSt may create LOG files twice with overwrite, however, the ForSt wrapped
> file system does not support overwriting files. Exception thrown in this
> case:
> {code:java}
> Caused by: java.lang.IllegalStateException: Try to add a file that is already
> in mappingTable, but with inconsistent entry. Key:
> oss://xxxxxxxxxxxxxxxxxx/ba3bcc9061a3858257e3119f0a6493ab/shared/op_AsyncKeyedProcessOperator_7261c9d38047b376863833deae26476f__4_8__attempt_0/db/LOG,
> source:
> FileBackedSource{filePath=/tmp/ba3bcc9061a3858257e3119f0a6493ab/op_AsyncKeyedProcessOperator_7261c9d38047b376863833deae26476f__4_8__attempt_0/db/1b72a4ee-9d4f-4d37-b1d4-781774c4da78},
> fileOwnership: PRIVATE_OWNED_BY_DB. Entry in table:
> MappingEntry{source=FileBackedSource{filePath=/tmp/ba3bcc9061a3858257e3119f0a6493ab/op_AsyncKeyedProcessOperator_7261c9d38047b376863833deae26476f__4_8__attempt_0/db/f60883d6-355e-491b-82d9-1875e5cb151e},
> fileOwnership=PRIVATE_OWNED_BY_DB, isDirectory= false}
> at
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:193)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.fs.filemapping.FileMappingManager.getExistingMappingEntry(FileMappingManager.java:107)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.fs.filemapping.FileMappingManager.addFileToMappingTable(FileMappingManager.java:97)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.fs.filemapping.FileMappingManager.createNewFile(FileMappingManager.java:70)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.fs.ForStFlinkFileSystem.create(ForStFlinkFileSystem.java:189)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.fs.ForStFlinkFileSystem.create(ForStFlinkFileSystem.java:182)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.fs.StringifiedForStFileSystem.create(StringifiedForStFileSystem.java:74)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.forstdb.RocksDB.write0(Native Method)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.forstdb.RocksDB.write(RocksDB.java:1843)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.ForStDBWriteBatchWrapper.flush(ForStDBWriteBatchWrapper.java:136)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at
> org.apache.flink.state.forst.ForStWriteBatchOperation.lambda$process$0(ForStWriteBatchOperation.java:72)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> ... 29 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
