zncleon opened a new issue, #15487:
URL: https://github.com/apache/iceberg/issues/15487
### Apache Iceberg version
1.10.1 (latest release)
### Query engine
Flink
### Please describe the bug 🐞
bug report is:
```
INFO IncrementalFileCleanup:68 - Expired snapshot: BaseSnapshot
...
WARN Tasks:459 - Retrying task after failure: sleepTimeMs=1087 Failed to
open input stream for file:xxx
org.apache.iceberg.exceptions.NotFoundException: Failed to open input stream
for file:xxx
at
org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:185)
~[iceberg-flink-runtime-1.20-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:102)
~[iceberg-flink-runtime-1.20-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.avro.AvroIterable.iterator(AvroIterable.java:77)
~[iceberg-flink-runtime-1.20-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.IncrementalFileCleanup.lambda$cleanFiles$3(IncrementalFileCleanup.java:209)
~[iceberg-flink-runtime-1.20-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
[iceberg-flink-runtime-1.20-1.11.0-SNAPSHOT.jar:?]
at org.apache.iceberg.util.Tasks$Builder$1.run(Tasks.java:315)
[iceberg-flink-runtime-1.20-1.11.0-SNAPSHOT.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
...
WARN IncrementalFileCleanup:281 - Failed to get deleted files: this may
cause orphaned data files
```
and my Flink tablemaintenance configuration is:
```
TriggerLockFactory lockFactory = new JdbcLockFactory
...
builder.uidSuffix("iceberg-table-maintenance-job")
.rateLimit(Duration.ofSeconds(60))
.lockCheckDelay(Duration.ofSeconds(30))
.parallelism(8)
.add(ExpireSnapshots.builder()
.scheduleOnInterval(Duration.ofHours(1))
.maxSnapshotAge(Duration.ofHours(3)))
.add(DeleteOrphanFiles.builder()
.scheduleOnInterval(Duration.ofHours(1))
.minAge(Duration.ofHours(1)))
.append();
```
I set the `ExpireSnapshots` and `DeleteOrphanFIles`. And I use jdbc lock to
manage the concurreny of this two procedures. Is my configuration do work? and
what introduce this error?
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
