Re: Solr 3.5 DIH delta-import replicating full index or Admin UI problem?

2012-07-17 Thread Erick Erickson
What you're seeing is the replication of the changed segments (new
segments actually). Replication only moves new or merged
segments and they will be a varying portion of the total index. If you
optimized, you'd see the entire index be moved (but you don't
need to do that!).

You should be able to search on the slave and find the documents you've
just added. If you can do that there's no problem.

Best
Erick

On Mon, Jul 16, 2012 at 3:10 PM, Arcadius Ahouansou
arcad...@menelic.com wrote:
 Hello.

 We are running Solr 3.5 multicore in master-slave mode.


 -Our delta-import looks like:
 /solr/core01/dataimport?command=delta-import*optimize=false*

 The size of the index in 1.18GB

 When delta-import is going on, on the slave admin UI
  8983/solr/core01/admin/replication/index.jsp
 I can see the following output:
 
 Master http://solrmaster01.somedomain.com:8983/solr/core01/replication
 Latest Index Version:null, Generation: null
 Replicatable Index Version:1342183977587, Generation: 33
 Poll Interval 00:00:60
 Local Index Index Version: 1342183977585, Generation: 32
 Location: /var/somedomain/solr/solrhome/core01/data/index
 Size: 1.18 GB
 Times Replicated Since Startup: 32
 Previous Replication Done At: Mon Jul 16 17:08:58 GMT 2012
 Config Files Replicated At: null
 Config Files Replicated: null
 Times Config Files Replicated Since Startup: null
 Next Replication Cycle At: Mon Jul 16 17:09:58 GMT 2012
 Current Replication Status Start Time: Mon Jul 16 17:08:58 GMT 2012
 Files Downloaded: 12 / 95
 *Downloaded: 4.33 KB / 1.18 GB [0.0%]*
 *Downloading File: _1o.fdt, Downloaded: 510 bytes / 510 bytes [100.0%]*
 Time Elapsed: 22s, Estimated Time Remaining: 6266208s, Speed: 201 bytes/s
 -


 - Does Downloaded: 4.33 KB / *1.18 GB [0.0%] *means that the solr slave
 is going to download the whole 1.18GB?

 -I have been monitoring this and the replications takes less that a minute.
 And checking the files in the index directory on the slave, the timestamps
 are quite different, so apparently, the slave is not downloading the full
 index all the time.

 -Please, has anyone else seen the whole index size being shown as
 denominator of the Downloaded fraction?

 -Anything I may be doing wrong?

 -Also notice the Files Downloaded: 12 / 95.  That bit never increase
 to 95 / 95


 Our solrconfig looks like this:

 --
 requestHandler name=/replication class=solr.ReplicationHandler 
 lst name=master
  str name=enable${enable.master:false}/str
  str name=replicateAftercommit/str
  str name=replicateAfterstartup/str
  str
 name=confFilessolrconfig.xml,synonyms.txt,schema.xml,stopwords.txt,data-config.xml/str
 /lst
 lst name=slave
  str name=enable${enable.slave:false}/str
  str name=masterUrlsome-master-full-url/str
  str name=pollInterval00:00:60/str
 /lst
 /requestHandler
 --


 Thanks.

 Arcadius.

 *
 *


Solr 3.5 DIH delta-import replicating full index or Admin UI problem?

2012-07-16 Thread Arcadius Ahouansou
Hello.

We are running Solr 3.5 multicore in master-slave mode.


-Our delta-import looks like:
/solr/core01/dataimport?command=delta-import*optimize=false*

The size of the index in 1.18GB

When delta-import is going on, on the slave admin UI
 8983/solr/core01/admin/replication/index.jsp
I can see the following output:

Master http://solrmaster01.somedomain.com:8983/solr/core01/replication
Latest Index Version:null, Generation: null
Replicatable Index Version:1342183977587, Generation: 33
Poll Interval 00:00:60
Local Index Index Version: 1342183977585, Generation: 32
Location: /var/somedomain/solr/solrhome/core01/data/index
Size: 1.18 GB
Times Replicated Since Startup: 32
Previous Replication Done At: Mon Jul 16 17:08:58 GMT 2012
Config Files Replicated At: null
Config Files Replicated: null
Times Config Files Replicated Since Startup: null
Next Replication Cycle At: Mon Jul 16 17:09:58 GMT 2012
Current Replication Status Start Time: Mon Jul 16 17:08:58 GMT 2012
Files Downloaded: 12 / 95
*Downloaded: 4.33 KB / 1.18 GB [0.0%]*
*Downloading File: _1o.fdt, Downloaded: 510 bytes / 510 bytes [100.0%]*
Time Elapsed: 22s, Estimated Time Remaining: 6266208s, Speed: 201 bytes/s
-


- Does Downloaded: 4.33 KB / *1.18 GB [0.0%] *means that the solr slave
is going to download the whole 1.18GB?

-I have been monitoring this and the replications takes less that a minute.
And checking the files in the index directory on the slave, the timestamps
are quite different, so apparently, the slave is not downloading the full
index all the time.

-Please, has anyone else seen the whole index size being shown as
denominator of the Downloaded fraction?

-Anything I may be doing wrong?

-Also notice the Files Downloaded: 12 / 95.  That bit never increase
to 95 / 95


Our solrconfig looks like this:

--
requestHandler name=/replication class=solr.ReplicationHandler 
lst name=master
 str name=enable${enable.master:false}/str
 str name=replicateAftercommit/str
 str name=replicateAfterstartup/str
 str
name=confFilessolrconfig.xml,synonyms.txt,schema.xml,stopwords.txt,data-config.xml/str
/lst
lst name=slave
 str name=enable${enable.slave:false}/str
 str name=masterUrlsome-master-full-url/str
 str name=pollInterval00:00:60/str
/lst
/requestHandler
--


Thanks.

Arcadius.

*
*