pvary commented on code in PR #15713:
URL: https://github.com/apache/iceberg/pull/15713#discussion_r3783114969
##########
core/src/test/java/org/apache/iceberg/TestDeleteFiles.java:
##########
@@ -738,7 +744,120 @@ public void
removingDataFilesWhenTruncatingAlsoRemovesDVs() {
statuses(Status.DELETED, Status.DELETED));
}
+ @TestTemplate
+ public void testFilterManifestsWithLimitedIOPool() {
+ TrackingFileIO io = new TrackingFileIO(FILE_IO);
+ TestTables.TestTableOperations ops =
+ new TestTables.TestTableOperations("limited-test", tableDir, io);
+ Table testTable =
+ TestTables.create(
+ tableDir, "limited-test", SCHEMA, SPEC, SortOrder.unsorted(),
formatVersion, ops);
+ commit(testTable, testTable.newFastAppend().appendFile(FILE_A), branch);
+ Snapshot deleteSnap = commit(testTable,
testTable.newDelete().deleteFile(FILE_A), branch);
+
assertThat(deleteSnap.summary()).containsEntry(SnapshotSummary.DELETED_FILES_PROP,
"1");
+ assertThat(io.peakCount()).isEqualTo(1);
+ }
+
private static ByteBuffer longToBuffer(long value) {
return ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putLong(0,
value);
}
+
+ /** A {@link FileIO} that tracks input streams. */
+ private static class TrackingFileIO implements FileIO {
Review Comment:
We could save a few lines of code if we use `Mockito` to spy on
`LocalFileIO`.
Maybe something like this:
```
/** Returns a {@link FileIO} that records the peak number of concurrently
open input streams. */
private static FileIO trackingIO(AtomicInteger peakOpenStreams) {
AtomicInteger openStreams = new AtomicInteger(0);
FileIO io = Mockito.spy(new TestTables.LocalFileIO());
Mockito.doAnswer(
newInputFile -> {
InputFile file = Mockito.spy((InputFile)
newInputFile.callRealMethod());
Mockito.doAnswer(
newStream -> {
peakOpenStreams.accumulateAndGet(openStreams.incrementAndGet(), Math::max);
SeekableInputStream stream =
Mockito.spy((SeekableInputStream)
newStream.callRealMethod());
Mockito.doAnswer(
close -> {
openStreams.decrementAndGet();
return close.callRealMethod();
})
.when(stream)
.close();
return stream;
})
.when(file)
.newStream();
return file;
})
.when(io)
.newInputFile(Mockito.anyString());
return io;
}
```
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
