Reo-LEI commented on a change in pull request #3120:
URL: https://github.com/apache/iceberg/pull/3120#discussion_r710026824
##########
File path: data/src/main/java/org/apache/iceberg/data/DeleteFilter.java
##########
@@ -189,7 +192,11 @@ protected boolean shouldKeep(T item) {
if (posDeletes.stream().mapToLong(DeleteFile::recordCount).sum() <
setFilterThreshold) {
return Deletes.filter(
records, this::pos,
- Deletes.toPositionSet(dataFile.path(),
CloseableIterable.concat(deletes)));
+ Deletes.toPositionSet(
+ dataFile.path(),
+ CloseableIterable.concat(deletes, ThreadPools.getWorkerPool(),
ThreadPools.getPoolParallelism())
Review comment:
I add an optional method to set the read pool and config the read
parallelism by code. @rdblue
And I think we could config the read parallelism by
'iceberg.worker.num-threads' rather than table property, because I don't want
to spread the table property widely. @jackye1995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
