Daniel Lowe created SOLR-17433:
----------------------------------
Summary: SolrStream by default creates a Http2SolrClient that can
only stream for 60 seconds
Key: SOLR-17433
URL: https://issues.apache.org/jira/browse/SOLR-17433
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: SolrJ, streaming expressions
Affects Versions: 9.6, 9.5, 9.4
Reporter: Daniel Lowe
This behaviour changed in SolrJ 9.4, in earlier versions a {{HttpSolrClient
}}was directly instantiated in SolrStream's open() method, while from SolrJ 9.4
a client is instantiated via a {{{}SolrClientCache{}}}.
I believe the expected behaviour is that the client that the SolrClientCache
instantiates should have appropriate timeout settings for streaming usage. I
think the 60 second timeout arises from {{{}SolrClientCache{}}}'s MIN_TIMEOUT
of 60 seconds. I note that that this is inconsistent with
HttpClientUtil.DEFAULT_SO_TIMEOUT (600 seconds) which is used by default if you
manually instantiate a {{Http2SolrClient }}or {{HttpJdkSolrClient.}}
Given what I think the typical use case is for SolrStream ideally I don't think
there should even be a timeout assuming the stream is still returning content.
I think you can restore the original behaviour by using:
{code:java}
context.setSolrClientCache(new SolrClientCache(new
HttpSolrClient.Builder(url).build().getHttpClient())); {code}
Example code:
{code:java}
ModifiableSolrParams params = new ModifiableSolrParams();
params.set("expr", "search(<your_collection_name>,qt=\"/export\",q=\"" +
someLongRunningQuery + "\", fl=\"id\", sort=\"id asc\")");
params.set("qt", "/stream");
try (TupleStream solrStream = new SolrStream(url, params)) {
StreamContext context = new StreamContext();
solrStream.setStreamContext(context);
solrStream.open();
Tuple tuple;
while (!(tuple = solrStream.read()).EOF) {
}
}
{code}
{code:java}
Caused by: java.util.concurrent.TimeoutException: Total timeout 60000 ms
elapsed at
org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:342)
at
org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:322)
at
org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:110)
at
org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:197)
at org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:294)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
