Hi Glen, Thank you very much for the quick response, I would like to try increasing the netTimoutForStreamingResults , is that something I can do it in the MySQL side? or in the solr side?
Giri On Tue, Jun 8, 2010 at 6:17 PM, Glen Newton <glen.new...@gmail.com> wrote: > As the index gets larger, the underlying housekeeping of the Lucene > index sometimes causes pauses in the indexing. The JDBC connection > (and/or the underlying socket) to the MySql database can time out > during these pauses. > > - If it is not set, you should add this to your JCBD url: > autoreconnect=true > - "Increase the netTimoutForStreamingResults value" from > > http://lucene.grantingersoll.com/2008/07/16/mysql-solr-and-communications-link-failure/ > > See also: > http://lucene.472066.n3.nabble.com/Recommended-MySQL-JDBC-driver-td817458.html > > -Glen Newton > http://zzzoot.blogspot.com/ > > On 09/06/2010, Giri <giriprak...@gmail.com> wrote: > > Hi Group, > > > > I have been trying index about 70 million records in the solr index, the > > data is coming from the MySQL database, and I am using the > DataImportHandler > > with batchSize set to -1. When I perform a full-import, it indexes about > 27 > > million records then throws the following exception: > > > > Any help will be really appreciated! > > > > thanks! > > > > Giri > > > > ------------------------------------------------------- > > > > WARNING: Error reading data > > com.mysql.jdbc.CommunicationsException: Communications link failure due > to > > underlying exception: > > > > ** BEGIN NESTED EXCEPTION ** > > > > java.io.EOFException > > > > STACKTRACE: > > > > java.io.EOFException > > at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1934) > > at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2433) > > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2909) > > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:798) > > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1316) > > at > com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:370) > > at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:360) > > at com.mysql.jdbc.ResultSet.next(ResultSet.java:5897) > > at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:265) > > at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$500(JdbcDataSource.java:161) > > at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:196) > > at > > > org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) > > at > > > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:77) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:178) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:136) > > at > > > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) > > at > > > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:386) > > at > > > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377) > > > > > > ** END NESTED EXCEPTION ** > > > > > > > > Last packet sent to the server was 5359471 ms ago. > > at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2592) > > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2909) > > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:798) > > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1316) > > at > com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:370) > > at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:360) > > at com.mysql.jdbc.ResultSet.next(ResultSet.java:5897) > > at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:265) > > at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$500(JdbcDataSource.java:161) > > at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:196) > > at > > > org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) > > at > > > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:77) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:178) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:136) > > at > > > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) > > at > > > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:386) > > at > > > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377) > > > > > -- > > - >
