Hello. We have a Solr 3.5 master randomly hanging during delta import. It does not happen very often.
When it does, the output of dataimport?command=status is shown below. After each page refresh, the only thing that changes is the "Time Elapsed". I have also done a thread dump (see below). It shows that the DIH is hanging on java.net.SocketInputStream.socketRead As the default DIH DataSource connectionTimeout and readTimeout seem to have no effect, I have started having a look at setting the following system properties: - sun.net.client.defaultConnectTimeout - sun.net.client.defaultReadTimeout or Oracle own - oracle.net.READ_TIMEOUT - oracle.jdbc.ReadTimeout Please, is there any better way of solving this issue? Thank you very much. Arcadius. DIH Status page: <str name="command">status</str> <str name="status">busy</str> <str name="importResponse">A command is still running...</str> <lst name="statusMessages"> <str name="Time Elapsed">48:31:22.998</str> <str name="Total Requests made to DataSource">173708</str> <str name="Total Rows Fetched">1385636</str> <str name="Total Documents Skipped">0</str> <str name="Delta Dump started">2012-11-10 16:35:36</str> <str name="Identifying Delta">2012-11-10 16:35:36</str> <str name="Deltas Obtained">2012-11-10 16:38:29</str> <str name="Building documents">2012-11-10 16:38:29</str> <str name="Total Changed Documents">1026622</str></lst> Thread Dump: > No deadlocks found. > > Thread 16873: (state = IN_NATIVE) > > - java.net.SocketInputStream.socketRead0(java.io.FileDescriptor, byte[], > int, int, int) @bci=0 (Compiled frame; information may be imprecise) > > - java.net.SocketInputStream.read(byte[], int, int) @bci=84, line=129 > (Compiled frame) > > - oracle.net.ns.Packet.receive() @bci=31, line=240 (Compiled frame) > > - oracle.net.ns.DataPacket.receive() @bci=1, line=92 (Compiled frame) > > - oracle.net.ns.NetInputStream.getNextPacket() @bci=48, line=172 > (Compiled frame) > > - oracle.net.ns.NetInputStream.read(byte[], int, int) @bci=33, line=117 > (Compiled frame) > > - oracle.jdbc.driver.T4CMAREngine.getNBytes(byte[], int, int) @bci=15, > line=1395 (Compiled frame) > > - oracle.jdbc.driver.T4CMAREngine.unmarshalNBytes(byte[], int, int) @bci=21, > line=1368 (Compiled frame) > > - oracle.jdbc.driver.T4CMAREngine.unmarshalBuffer(byte[], int, int) @bci=50, > line=1906 (Compiled frame) > > - oracle.jdbc.driver.T4CMAREngine.unmarshalCLR(byte[], int, int[], int, > int) @bci=178, line=1672 (Compiled frame) > > - oracle.jdbc.driver.T4CMAREngine.unmarshalCLR(byte[], int, int[], int) > @bci=7, line=1589 (Compiled frame) > > - oracle.jdbc.driver.T4CMAREngine.unmarshalCLR(byte[], int, int[]) @bci=6, > line=1582 (Compiled frame) > > - oracle.jdbc.driver.T4CVarcharAccessor.unmarshalOneRow() @bci=214, > line=206 (Compiled frame) > > - oracle.jdbc.driver.T4CTTIrxd.unmarshal(oracle.jdbc.driver.Accessor[], > int, int) @bci=167, line=895 (Compiled frame) > > - oracle.jdbc.driver.T4C8Oall.receive() @bci=514, line=713 (Compiled > frame) > > - oracle.jdbc.driver.T4CStatement.doOall8(boolean, boolean, boolean, > boolean) @bci=653, line=183 (Compiled frame) > > - oracle.jdbc.driver.T4CStatement.executeForDescribe() @bci=39, line=780 > (Compiled frame) > > - oracle.jdbc.driver.T4CStatement.executeMaybeDescribe() @bci=104, > line=855 (Interpreted frame) > > - oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout() @bci=139, > line=1186 (Interpreted frame) > > - oracle.jdbc.driver.OracleStatement.executeInternal(java.lang.String) > @bci=62, line=1770 (Interpreted frame) > > - oracle.jdbc.driver.OracleStatement.execute(java.lang.String) @bci=13, > line=1739 (Interpreted frame) > > - oracle.jdbc.driver.OracleStatementWrapper.execute(java.lang.String) > @bci=5, line=299 (Interpreted frame) > > - org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator. > <init>(org.apache.solr.handler.dataimport.JdbcDataSource, > java.lang.String) @bci=98, line=246 (Interpreted frame) > > - > org.apache.solr.handler.dataimport.JdbcDataSource.getData(java.lang.String) > @bci=6, line=210 (Interpreted frame) > > - > org.apache.solr.handler.dataimport.JdbcDataSource.getData(java.lang.String) > @bci=2, line=39 (Interpreted frame) > > - > org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(java.lang.String) > @bci=19, line=59 (Compiled frame) > > - org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow() @bci=21, > line=73 (Compiled frame) > > - org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow() > @bci=18, line=238 (Compiled frame) > > - > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(org.apache.solr.handler.dataimport.VariableResolverImpl, > org.apache.solr.handler.dataimport.DocBuilder$DocWrapper, java.util.Map, > org.apache.solr.handler.dataimport.DataConfig$Entity, boolean, > org.apache.solr.handler.dataimport.ContextImpl) @bci=297, line=596 > (Compiled frame) > > - org.apache.solr.handler.dataimport.DocBuilder.doDelta() @bci=196, > line=303 (Compiled frame) > > - org.apache.solr.handler.dataimport.DocBuilder.execute() @bci=363, > line=179 (Interpreted frame) > > - > org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(org.apache.solr.handler.dataimport.SolrWriter, > org.apache.solr.handler.dataimport.DataImporter$RequestParams) @bci=51, > line=390 (Interpreted frame) > > - > org.apache.solr.handler.dataimport.DataImporter.runCmd(org.apache.solr.handler.dataimport.DataImporter$RequestParams, > org.apache.solr.handler.dataimport.SolrWriter) @bci=89, line=429 > (Interpreted frame) > > - org.apache.solr.handler.dataimport.DataImporter$1.run() @bci=12, > line=408 (Interpreted frame) > > >