I cannot see what is the problem. If you can do a thread dump and paste it here I may be able to suggest something
On Mon, Nov 10, 2008 at 9:44 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > > Hi Shalin > > I don't get Ive executed : > http://....com:8180/solr/video/dataimport?command=full-import > Then when I Checked : > http://solr-test.adm.video.com:8180/solr/video/dataimport > It looks like document are created : > <lst name="initArgs"> > − > <lst name="defaults"> > <str name="config">data-config.xml</str> > </lst> > </lst> > <str name="status">busy</str> > <str name="importResponse">A command is still running...</str> > − > <lst name="statusMessages"> > <str name="Time Elapsed">0:0:46.331</str> > <str name="Total Requests made to DataSource">37347</str> > <str name="Total Rows Fetched">75108</str> > <str name="Total Documents Processed">6224</str> > <str name="Total Documents Skipped">0</str> > <str name="Full Dump Started">2008-11-10 17:02:24</str> > </lst> > > but then after a while, time keep increasing but just time no documents ;.. > and it can be like that for hours. > > <str name="status">busy</str> > <str name="importResponse">A command is still running...</str> > − > <lst name="statusMessages"> > <str name="Time Elapsed">0:4:41.295</str> > <str name="Total Requests made to DataSource">84103</str> > <str name="Total Rows Fetched">113842</str> > <str name="Total Documents Processed">14016</str> > <str name="Total Documents Skipped">0</str> > <str name="Full Dump Started">2008-11-10 17:02:24</str> > </lst> > > and no exception : http://www.nabble.com/file/p20422919/syslog syslog > > Thanks for your help, > > > > Shalin Shekhar Mangar wrote: >> >> Hi, >> >> I'm confused by the log you have posted. There is no import command to be >> seen to /dataimport. Only one request is in the log to /dataimport at the >> very end. The logs say that Solr started up and some warming queries were >> executed and that's it. >> >> What do you mean by it looks stuck? Does the count of processed documents >> increase? Any exceptions during import would cause DataImportHandler to >> abort. The only case that DIH will not catch is when a error like out of >> memory happened. But still Solr or Tomcat should catch and log that >> exception (or it may just die). >> >> On Mon, Nov 10, 2008 at 8:33 PM, sunnyfr <[EMAIL PROTECTED]> wrote: >> >>> >>> Hi, >>> >>> I can't get what's going on ?? >>> If somebody could help me it would be very nice >>> thanks >>> >>> >>> sunnyfr wrote: >>> > >>> > Hi, >>> > >>> > I don't get what's happening >>> > I tried to make a full import with a limit inside between to id ... >>> > everything works fine then the time keep running but it looks stuck and >>> > Idon't have any error in my logs: >>> > >>> > Like if a process couldn't be executed, I didn't turn on snapshooter or >>> > autocommit is properly setteled I think : >>> > <autoCommit> >>> > <maxDocs>10000</maxDocs> >>> > <maxTime>1000</maxTime> >>> > </autoCommit> >>> > >>> > >>> > Data-Config: >>> > >>> > <str name="config">data-config.xml</str> >>> > </lst> >>> > </lst> >>> > <str name="status">busy</str> >>> > <str name="importResponse">A command is still running...</str> >>> > − >>> > <lst name="statusMessages"> >>> > <str name="Time Elapsed">0:16:32.47</str> ... <str name="Time >>> > Elapsed">0:19:18.160</str> >>> > <str name="Total Requests made to DataSource">661141</str> >>> > <str name="Total Rows Fetched">1647265</str> >>> > <str name="Total Documents Processed">110189</str> >>> > <str name="Total Documents Skipped">0</str> >>> > <str name="Full Dump Started">2008-11-06 16:50:17</str> >>> > </lst> >>> > − >>> > <str name="WARNING"> >>> > This response format is experimental. It is likely to change in the >>> > future. >>> > </str> >>> > </response> >>> > >>> > My stats : >>> > name: /dataimport >>> > class: org.apache.solr.handler.dataimport.DataImportHandler >>> > version: 1.0 >>> > description: Manage data import from databases to Solr >>> > stats: Status : RUNNING_FULL_DUMP >>> > Documents Processed : 110189 >>> > Requests made to DataSource : 661141 >>> > Rows Fetched : 1647265 >>> > Documents Deleted : 0 >>> > Documents Skipped : 0 >>> > Total Documents Processed : 0 >>> > Total Requests made to DataSource : 0 >>> > Total Rows Fetched : 0 >>> > Total Documents Deleted : 0 >>> > Total Documents Skipped : 0 >>> > handlerStart : 1225986600254 >>> > requests : 86 >>> > errors : 0 >>> > timeouts : 0 >>> > totalTime : 0 >>> > avgTimePerRequest : 0.0 >>> > avgRequestsPerSecond : 0.05853414 >>> > >>> > searcherName : [EMAIL PROTECTED] main >>> > caching : true >>> > numDocs : 109686 >>> > maxDoc : 109686 >>> > readerImpl : MultiSegmentReader >>> > readerDir : >>> > org.apache.lucene.store.FSDirectory@/data/solr/video/data/index >>> > indexVersion : 1225986600336 >>> > openedAt : Thu Nov 06 16:57:37 CET 2008 >>> > registeredAt : Thu Nov 06 16:57:37 CET 2008 >>> > warmupTime : 0 >>> > >>> > >>> > Logs: >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result >>> for >>> > [EMAIL PROTECTED] main >>> > >>> ^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming >>> > [EMAIL PROTECTED] main from [EMAIL PROTECTED] main >>> > >>> ^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.SolrDeletionPolicy onInit INFO: >>> > SolrDeletionPolicy.onInit: commits:num=1 >>> > >>> ^Icommit{dir=/data/solr/video/data/index,segFN=segments_6g,version=1225986600336,generation=232,filenames=[segments_6g, >>> > _33.fnm, _33.frq, _33.prx, _33.tis, _33.tii, _33.nrm, _33.tvx, _33.tvf, >>> > _33.tvd, _33.fdx, _33.fdt, _66.fnm, _66.frq, _66.prx, _66.tis, _66.tii, >>> > _66.nrm, _66.tvx, _66.tvf, _66.tvd, _66.fdx, _66.fdt, _6g.fnm, _6g.frq, >>> > _6g.prx, _6g.tis, _6g.tii, _6g.nrm, _6g.tvx, _6g.tvf, _6g.tvd, _6g.fdx, >>> > _6g.fdt, _6r.fnm, _6r.frq, _6r.prx, _6r.tis, _6r.tii, _6r.nrm, _6r.tvx, >>> > _6r.tvf, _6r.tvd, _6r.fdx, _6r.fdt, _72.fnm, _72.frq, _72.prx, _72.tis, >>> > _72.tii, _72.nrm, _72.tvx, _72.tvf, _72.tvd, _72.fdx, _72.fdt, _73.fnm, >>> > _73.frq, _73.prx, _73.tis, _73.tii, _73.nrm, _73.tvx, _73.tvf, _73.tvd, >>> > _73.fdx, _73.fdt] >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit >>> = >>> > 1225986600336 >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result >>> for >>> > [EMAIL PROTECTED] main >>> > >>> ^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming >>> > [EMAIL PROTECTED] main from [EMAIL PROTECTED] main >>> > >>> ^IdocumentCache{lookups=0,hits=0,hitratio=0.00,inserts=20,evictions=0,size=20,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result >>> for >>> > [EMAIL PROTECTED] main >>> > >>> ^IdocumentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.QuerySenderListener newSearcher INFO: >>> > QuerySenderListener sending requests to [EMAIL PROTECTED] main >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=null >>> path=null >>> > params={start=0&q=solr&rows=10} hits=0 status=0 QTime=0 >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=null >>> path=null >>> > params={start=0&q=rocks&rows=10} hits=960 status=0 QTime=0 >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=null >>> path=null >>> > params={q=static+newSearcher+warming+query+from+solrconfig.xml} >>> hits=1149 >>> > status=0 QTime=0 >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.QuerySenderListener newSearcher INFO: >>> > QuerySenderListener done. >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.core.SolrCore registerSearcher INFO: [video] Registered >>> > new searcher [EMAIL PROTECTED] main >>> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >>> > org.apache.solr.search.SolrIndexSearcher close INFO: Closing >>> > [EMAIL PROTECTED] main >>> > >>> ^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > >>> ^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > >>> ^IdocumentCache{lookups=0,hits=0,hitratio=0.00,inserts=20,evictions=0,size=20,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >>> > Nov 6 16:57:59 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:59 PM >>> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=/solr >>> > path=/dataimport params={} status=0 QTime=0 >>> > >>> > >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/solr-1.3---Problem-Full-Import-tp20364250p20421643.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > -- > View this message in context: > http://www.nabble.com/solr-1.3---Problem-Full-Import-tp20364250p20422919.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- --Noble Paul
