[
https://issues.apache.org/jira/browse/FLINK-23918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17443630#comment-17443630
]
Till Rohrmann commented on FLINK-23918:
---------------------------------------
cc [~twalthr]
> FileNotFoundException when open auto-compact in filesystem
> ----------------------------------------------------------
>
> Key: FLINK-23918
> URL: https://issues.apache.org/jira/browse/FLINK-23918
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem, FileSystems, Table SQL /
> Ecosystem
> Affects Versions: 1.12.2
> Environment: Flink 1.12.2
> HDFS 3.2.1
> config:
> flink parallelism is 8
> task manager slot is 4
> task manager memory is 8g
> checkpoint min interval is 60s
> state backend use: rocksdb
> state ttl: 3 days
> Reporter: li wei
> Priority: Blocker
> Attachments: image-2021-08-23-18-09-17-674.png,
> image-2021-08-23-18-14-20-389.png, image-2021-08-23-18-14-35-275.png
>
>
> After I config auto-compaction=true in filesystem connector, sometimes it
> will restart and show me exception like this "Caused by:
> java.io.FileNotFoundException: File does not exist:
> hdfs://xxxxxx/tmp/xxxxx/dt=20210823/.uncompacted-part-ef59d89c-36fb-49c5-9b74-dd6a7882ea9f-0-241".
> And when I use this filename as keyword to search, I found that it first show
> me an exception, then I got 2 compact success logs.
> I think it's not right here, First, flink should not throw this exception;
> Second, this file should not be merged into 2 files.
> !image-2021-08-23-18-14-20-389.png!
> !image-2021-08-23-18-14-35-275.png!
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
