BryceKan3 commented on PR #15428:
URL: https://github.com/apache/lucene/pull/15428#issuecomment-3549309780
> Would it make sense to use virtual threads and create a local executor
internally, instead of making the callers pass one? Maybe create a helper
function that opens and returns a `SegmentReader[]` array from a provided
`SegmentInfos`...
>
> how about something like –
>
> ```java
> // pass a null array when there are no old readers?
> private static SegmentReader[] createSegmentReaders(SegmentInfos sis,
SegmentReader[] oldReaders) {
> final SegmentReader[] readers = new SegmentReader[sis.size()];
> try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
> List<Future<SegmentReader>> futures = new ArrayList<>();
> for (int i = sis.size() - 1; i >= 0; i--) {
> // TODO: add check for cases where old reader should be reused?
> final int index = i;
> futures.add(
> (executor)
> .submit(
> () ->
> new SegmentReader(
> sis.info(index),
> sis.getIndexCreatedVersionMajor(),
> IOContext.DEFAULT)));
> }
> RuntimeException firstException = null;
> for (int i = 0; i < futures.size(); i++) {
> try {
> readers[sis.size() - 1 - i] = futures.get(i).get();
> } catch (ExecutionException | InterruptedException e) {
> // If there is an exception creating the reader we still process
> // the rest of the completed futures to allow us to close created
readers
> if (firstException == null) firstException = new
RuntimeException(e);
> }
> }
> if (firstException != null) throw firstException;
> } finally {
> return readers;
> }
> }
> ```
I just did some testing with passing in
Executors.newVirtualThreadPerTaskExecutor() as the executor service and was
still able to see the benefits. I think this is a really good call out -
reduces the complexity involved and customers will get the benefit without
having to update their code to pass in an executor. I will work on implementing
this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
