Hi Shalin I don't get Ive executed : http://....com:8180/solr/video/dataimport?command=full-import Then when I Checked : http://solr-test.adm.dailymotion.com:8180/solr/video/dataimport It looks like document are created : <lst name="initArgs"> − <lst name="defaults"> <str name="config">data-config.xml</str> </lst> </lst> <str name="status">busy</str> <str name="importResponse">A command is still running...</str> − <lst name="statusMessages"> <str name="Time Elapsed">0:0:46.331</str> <str name="Total Requests made to DataSource">37347</str> <str name="Total Rows Fetched">75108</str> <str name="Total Documents Processed">6224</str> <str name="Total Documents Skipped">0</str> <str name="Full Dump Started">2008-11-10 17:02:24</str> </lst>
but then after a while, time keep increasing but just time no documents ;.. and it can be like that for hours. <str name="status">busy</str> <str name="importResponse">A command is still running...</str> − <lst name="statusMessages"> <str name="Time Elapsed">0:4:41.295</str> <str name="Total Requests made to DataSource">84103</str> <str name="Total Rows Fetched">113842</str> <str name="Total Documents Processed">14016</str> <str name="Total Documents Skipped">0</str> <str name="Full Dump Started">2008-11-10 17:02:24</str> </lst> and no exception : http://www.nabble.com/file/p20422854/syslog syslog Thanks for your help, Shalin Shekhar Mangar wrote: > > Hi, > > I'm confused by the log you have posted. There is no import command to be > seen to /dataimport. Only one request is in the log to /dataimport at the > very end. The logs say that Solr started up and some warming queries were > executed and that's it. > > What do you mean by it looks stuck? Does the count of processed documents > increase? Any exceptions during import would cause DataImportHandler to > abort. The only case that DIH will not catch is when a error like out of > memory happened. But still Solr or Tomcat should catch and log that > exception (or it may just die). > > On Mon, Nov 10, 2008 at 8:33 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > >> >> Hi, >> >> I can't get what's going on ?? >> If somebody could help me it would be very nice >> thanks >> >> >> sunnyfr wrote: >> > >> > Hi, >> > >> > I don't get what's happening >> > I tried to make a full import with a limit inside between to id ... >> > everything works fine then the time keep running but it looks stuck and >> > Idon't have any error in my logs: >> > >> > Like if a process couldn't be executed, I didn't turn on snapshooter or >> > autocommit is properly setteled I think : >> > <autoCommit> >> > <maxDocs>10000</maxDocs> >> > <maxTime>1000</maxTime> >> > </autoCommit> >> > >> > >> > Data-Config: >> > >> > <str name="config">data-config.xml</str> >> > </lst> >> > </lst> >> > <str name="status">busy</str> >> > <str name="importResponse">A command is still running...</str> >> > − >> > <lst name="statusMessages"> >> > <str name="Time Elapsed">0:16:32.47</str> ... <str name="Time >> > Elapsed">0:19:18.160</str> >> > <str name="Total Requests made to DataSource">661141</str> >> > <str name="Total Rows Fetched">1647265</str> >> > <str name="Total Documents Processed">110189</str> >> > <str name="Total Documents Skipped">0</str> >> > <str name="Full Dump Started">2008-11-06 16:50:17</str> >> > </lst> >> > − >> > <str name="WARNING"> >> > This response format is experimental. It is likely to change in the >> > future. >> > </str> >> > </response> >> > >> > My stats : >> > name: /dataimport >> > class: org.apache.solr.handler.dataimport.DataImportHandler >> > version: 1.0 >> > description: Manage data import from databases to Solr >> > stats: Status : RUNNING_FULL_DUMP >> > Documents Processed : 110189 >> > Requests made to DataSource : 661141 >> > Rows Fetched : 1647265 >> > Documents Deleted : 0 >> > Documents Skipped : 0 >> > Total Documents Processed : 0 >> > Total Requests made to DataSource : 0 >> > Total Rows Fetched : 0 >> > Total Documents Deleted : 0 >> > Total Documents Skipped : 0 >> > handlerStart : 1225986600254 >> > requests : 86 >> > errors : 0 >> > timeouts : 0 >> > totalTime : 0 >> > avgTimePerRequest : 0.0 >> > avgRequestsPerSecond : 0.05853414 >> > >> > searcherName : [EMAIL PROTECTED] main >> > caching : true >> > numDocs : 109686 >> > maxDoc : 109686 >> > readerImpl : MultiSegmentReader >> > readerDir : >> > org.apache.lucene.store.FSDirectory@/data/solr/video/data/index >> > indexVersion : 1225986600336 >> > openedAt : Thu Nov 06 16:57:37 CET 2008 >> > registeredAt : Thu Nov 06 16:57:37 CET 2008 >> > warmupTime : 0 >> > >> > >> > Logs: >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result >> for >> > [EMAIL PROTECTED] main >> > >> ^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming >> > [EMAIL PROTECTED] main from [EMAIL PROTECTED] main >> > >> ^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.SolrDeletionPolicy onInit INFO: >> > SolrDeletionPolicy.onInit: commits:num=1 >> > >> ^Icommit{dir=/data/solr/video/data/index,segFN=segments_6g,version=1225986600336,generation=232,filenames=[segments_6g, >> > _33.fnm, _33.frq, _33.prx, _33.tis, _33.tii, _33.nrm, _33.tvx, _33.tvf, >> > _33.tvd, _33.fdx, _33.fdt, _66.fnm, _66.frq, _66.prx, _66.tis, _66.tii, >> > _66.nrm, _66.tvx, _66.tvf, _66.tvd, _66.fdx, _66.fdt, _6g.fnm, _6g.frq, >> > _6g.prx, _6g.tis, _6g.tii, _6g.nrm, _6g.tvx, _6g.tvf, _6g.tvd, _6g.fdx, >> > _6g.fdt, _6r.fnm, _6r.frq, _6r.prx, _6r.tis, _6r.tii, _6r.nrm, _6r.tvx, >> > _6r.tvf, _6r.tvd, _6r.fdx, _6r.fdt, _72.fnm, _72.frq, _72.prx, _72.tis, >> > _72.tii, _72.nrm, _72.tvx, _72.tvf, _72.tvd, _72.fdx, _72.fdt, _73.fnm, >> > _73.frq, _73.prx, _73.tis, _73.tii, _73.nrm, _73.tvx, _73.tvf, _73.tvd, >> > _73.fdx, _73.fdt] >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit >> = >> > 1225986600336 >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result >> for >> > [EMAIL PROTECTED] main >> > >> ^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming >> > [EMAIL PROTECTED] main from [EMAIL PROTECTED] main >> > >> ^IdocumentCache{lookups=0,hits=0,hitratio=0.00,inserts=20,evictions=0,size=20,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result >> for >> > [EMAIL PROTECTED] main >> > >> ^IdocumentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.QuerySenderListener newSearcher INFO: >> > QuerySenderListener sending requests to [EMAIL PROTECTED] main >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=null >> path=null >> > params={start=0&q=solr&rows=10} hits=0 status=0 QTime=0 >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=null >> path=null >> > params={start=0&q=rocks&rows=10} hits=960 status=0 QTime=0 >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=null >> path=null >> > params={q=static+newSearcher+warming+query+from+solrconfig.xml} >> hits=1149 >> > status=0 QTime=0 >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.QuerySenderListener newSearcher INFO: >> > QuerySenderListener done. >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.core.SolrCore registerSearcher INFO: [video] Registered >> > new searcher [EMAIL PROTECTED] main >> > Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM >> > org.apache.solr.search.SolrIndexSearcher close INFO: Closing >> > [EMAIL PROTECTED] main >> > >> ^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > >> ^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > >> ^IdocumentCache{lookups=0,hits=0,hitratio=0.00,inserts=20,evictions=0,size=20,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} >> > Nov 6 16:57:59 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:59 PM >> > org.apache.solr.core.SolrCore execute INFO: [video] webapp=/solr >> > path=/dataimport params={} status=0 QTime=0 >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/solr-1.3---Problem-Full-Import-tp20364250p20421643.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/solr-1.3---Problem-Full-Import-tp20364250p20422854.html Sent from the Solr - User mailing list archive at Nabble.com.
