Timeout causes DIH to finish with error message. So, If I check DIH response to be sure that DIH session have finished without any mistakes.... it causes some trouble :) I haven't check yet whether all records successfully imported to solr. Supposed that after timeout shard does not accept records from DIH. Am I wrong? -- Vadim
> -----Original Message----- > From: Mikhail Khludnev [mailto:m...@apache.org] > Sent: Monday, September 02, 2019 12:23 PM > To: Vadim Ivanov; solr-user > Subject: Re: Idle Timeout while DIH indexing and implicit sharding in 7.4 > > It seems like reasonable behavior. SolrWriter hangs while import is > running, it holds DistributedZkUpdateProcessor, which holds > SolrCmdDistributor, which keep client connection to shards open, which > causes timeout. > It might be worked around by supplying custom SolrWriter which finishes > UpdateProcessor's chain from time to time and recreate it. However, that > exception shouldn't cause any problem or it does? > Also, it's worth to track as a jira, or mentioned in the ticket regarding > adjusting DIH for Cloud. > > On Mon, Sep 2, 2019 at 9:44 AM Vadim Ivanov < > vadim.iva...@spb.ntk-intourist.ru> wrote: > > > I’ve raised that timeout from 5 min to 40 min. > > > > It somehow mitigated the issue in my use case. > > > > Problem occurs when some updates goes to one shard in the beginning of > > long DIH session, then all updates goes to other shards for more than 300 > > sec. > > > > And when ( after 300 sec) some updates goes again to the first shard it > > faces closed connection problem. > > > > > > > > Either it’s right behavior or not – it’s hard to say…. > > > > > > > > > > > > -- > > > > Vadim > > > > *From:* Mikhail Khludnev [mailto:m...@apache.org] > > *Sent:* Monday, September 02, 2019 1:31 AM > > *To:* solr-user > > *Cc:* vadim.iva...@spb.ntk-intourist.ru > > *Subject:* Re: Idle Timeout while DIH indexing and implicit sharding in > > 7.4 > > > > > > > > Giving that > > > > org.apache.solr.common.util.FastInputStream.peek(FastInputStream.java:60) > > at > > > > > org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoad > er.java:107) > > > > > > > > > > JavabinLoader hangs on Stream.peek(), awaiting -1, and hit timeout. I > > guess it's might be related with "closing sockets". I've looked through > > CHANGES after 6.5.1, here's what might be examined for impacting this: > > > > > > > > * SOLR-10779: JavaBinCodec should use close consistently > > > > * SOLR-12290: Do not close any servlet streams and improve our servlet > > stream > > > > Description Resource Path Location > > * SOLR-12477: An update would return a client error(400) if it hit a > > AlreadyClos > > > > * SOLR-12897: Introduce AlreadyClosedException to clean up silly close / > > shutd > > > > > > > > Have nothing more than that so far. > > > > > > > > On Sun, Sep 1, 2019 at 8:50 PM swapna.minnaka...@copart.com < > > swapna.minnaka...@copart.com> wrote: > > > > I am facing same exact issue. We never had any issue with 6.5.1 when doing > > full index (initial bulk load) > > After upgrading to 7.5.0, getting below exception and indexing is taking a > > very long time > > > > 2019-09-01 10:11:27.436 ERROR (qtp1650813924-22) [c:c_collection s:shard1 > > r:core_node3 x:c_collection_shard1_replica_n1] o.a.s.h.RequestHandlerBase > > java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout > > expired: 300000/300000 ms > > at > > > > > org.eclipse.jetty.server.HttpInput$ErrorState.noContent(HttpInput.java:1080) > > at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:313) > > at > > > > > org.apache.solr.servlet.ServletInputStreamWrapper.read(ServletInputStream > Wrapper.java:74) > > at > > > > > org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java: > 100) > > at > > > > > org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputS > tream.java:79) > > at > > org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:88) > > at > > org.apache.solr.common.util.FastInputStream.peek(FastInputStream.java:60) > > at > > > > > org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoad > er.java:107) > > at > > org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55) > > at > > > > > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandl > er.java:97) > > at > > > > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Cont > entStreamHandlerBase.java:68) > > at > > > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandler > Base.java:199) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541) > > at > > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709) > > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515) > > at > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) > > at > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) > > at > > > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.j > ava:1634) > > at > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:14 > 6) > > at > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java > :132) > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.jav > a:257) > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java: > 1595) > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.jav > a:255) > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.jav > a:1317) > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java > :203) > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1 > 564) > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java > :201) > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java: > 1219) > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:14 > 4) > > at > > > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHan > dlerCollection.java:219) > > at > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.ja > va:126) > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java > :132) > > at > > > > > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java: > 335) > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java > :132) > > at org.eclipse.jetty.server.Server.handle(Server.java:531) > > at > > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) > > at > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) > > at > > org.eclipse.jetty.io > > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) > > at org.eclipse.jetty.io > > .FillInterest.fillable(FillInterest.java:102) > > at > > org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.j > ava:333) > > > > > > > > -- > > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html > > > > > > > > > > -- > > > > Sincerely yours > > Mikhail Khludnev > > > > > -- > Sincerely yours > Mikhail Khludnev
