[
https://issues.apache.org/jira/browse/SOLR-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759290#action_12759290
]
Artem Russakovskii edited comment on SOLR-1458 at 9/24/09 3:27 PM:
-------------------------------------------------------------------
Also, after an optimize is issued to the master, the problem goes away. For a
while, and then starts again. I do optimizes every hour, and commits are
ongoing every minute.
For instance, after an optimize on the master:
{code}
<response>
<lst name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst><arr name="filelist"><lst><str
name="name">_1bx2.frq</str>
<long name="lastmodified">1253826921000</long><long
name="size">67855561</long></lst><lst><str name="name">_1bx2.nrm</str><long
name="lastmodified">1253826921000</long><long
name="size">2515184</long></lst><lst><str name="name">_1bx2.tii</str><long
name="lastmodified">1253826921000</long><long
name="size">581824</long></lst><lst><str name="name">_1bx2.fnm</str><long
name="lastmodified">1253826906000</long><long
name="size">132</long></lst><lst><str name="name">_1bx2.fdt</str><long
name="lastmodified">1253826906000</long><long
name="size">7805294</long></lst><lst><str name="name">_1bx2.tis</str><long
name="lastmodified">1253826921000</long><long
name="size">43326001</long></lst><lst><str name="name">_1bx2.fdx</str><long
name="lastmodified">1253826906000</long><long
name="size">4024292</long></lst><lst><str name="name">_1bx2.prx</str><long
name="lastmodified">1253826921000</long><long
name="size">47213429</long></lst><lst><str name="name">segments_19jy</str><long
name="lastmodified">1253826922000</long><long
name="size">287</long></lst></arr>
</response>
{code}
And another thing worth mentioning, I updated the solr.war from 8/25/09 nightly
to 9/22/09 nightly but I hope that doesn't cause problems.
was (Author: archon810):
Also, after an optimize is issued to the master, the problem goes away. For
a while, and then starts again. I do optimizes every hour, and commits are
ongoing every minute.
For instance, after an optimize on the master:
{code}
<response>
<lst name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst><arr name="filelist"><lst><str
name="name">_1bx2.frq</str><long name="lastmodified">1253826921000</long><long
name="size">67855561</long></lst><lst><str name="name">_1bx2.nrm</str><long
name="lastmodified">1253826921000</long><long
name="size">2515184</long></lst><lst><str name="name">_1bx2.tii</str><long
name="lastmodified">1253826921000</long><long
name="size">581824</long></lst><lst><str name="name">_1bx2.fnm</str><long
name="lastmodified">1253826906000</long><long
name="size">132</long></lst><lst><str name="name">_1bx2.fdt</str><long
name="lastmodified">1253826906000</long><long
name="size">7805294</long></lst><lst><str name="name">_1bx2.tis</str><long
name="lastmodified">1253826921000</long><long
name="size">43326001</long></lst><lst><str name="name">_1bx2.fdx</str><long
name="lastmodified">1253826906000</long><long
name="size">4024292</long></lst><lst><str name="name">_1bx2.prx</str><long
name="lastmodified">1253826921000</long><long
name="size">47213429</long></lst><lst><str name="name">segments_19jy</str><long
name="lastmodified">1253826922000</long><long
name="size">287</long></lst></arr>
</response>
{code}
> Java Replication error: NullPointerException SEVERE: SnapPull failed on
> 2009-09-22 nightly
> ------------------------------------------------------------------------------------------
>
> Key: SOLR-1458
> URL: https://issues.apache.org/jira/browse/SOLR-1458
> Project: Solr
> Issue Type: Bug
> Components: replication (java)
> Affects Versions: 1.4
> Environment: CentOS x64
> 8GB RAM
> Tomcat, running with 7G max memory; memory usage is <2GB, so it's not the
> problem
> Host a: master
> Host b: slave
> Multiple single core Solr instances, using JNDI.
> Java replication
> Reporter: Artem Russakovskii
> Assignee: Noble Paul
> Fix For: 1.4
>
> Attachments: SOLR-1458.patch, SOLR-1458.patch
>
>
> After finally figuring out the new Java based replication, we have started
> both the slave and the master and issued optimize to all master Solr
> instances. This triggered some replication to go through just fine, but it
> looks like some of it is failing.
> Here's what I'm getting in the slave logs, repeatedly for each shard:
> {code}
> SEVERE: SnapPull failed
> java.lang.NullPointerException
> at
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:271)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:258)
> at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> If I issue an optimize again on the master to one of the shards, it then
> triggers a replication and replicates OK. I have a feeling that these
> SnapPull failures appear later on but right now I don't have enough to form a
> pattern.
> Here's replication.properties on one of the failed slave instances.
> {code}
> cat data/replication.properties
> #Replication details
> #Wed Sep 23 19:35:30 PDT 2009
> replicationFailedAtList=1253759730020,1253759700018,1253759670019,1253759640018,1253759610018,1253759580022,1253759550019,1253759520016,1253759490026,1253759460016
> previousCycleTimeInSeconds=0
> timesFailed=113
> indexReplicatedAtList=1253759730020,1253759700018,1253759670019,1253759640018,1253759610018,1253759580022,1253759550019,1253759520016,1253759490026,1253759460016
> indexReplicatedAt=1253759730020
> replicationFailedAt=1253759730020
> lastCycleBytesDownloaded=0
> timesIndexReplicated=113
> {code}
> and another
> {code}
> cat data/replication.properties
> #Replication details
> #Wed Sep 23 18:42:01 PDT 2009
> replicationFailedAtList=1253756490034,1253756460169
> previousCycleTimeInSeconds=1
> timesFailed=2
> indexReplicatedAtList=1253756521284,1253756490034,1253756460169
> indexReplicatedAt=1253756521284
> replicationFailedAt=1253756490034
> lastCycleBytesDownloaded=22932293
> timesIndexReplicated=3
> {code}
> Some relevant configs:
> In solrconfig.xml:
> {code}
> <!-- For docs see http://wiki.apache.org/solr/SolrReplication -->
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="master">
> <str name="enable">${enable.master:false}</str>
> <str name="replicateAfter">optimize</str>
> <str name="backupAfter">optimize</str>
> <str name="commitReserveDuration">00:00:20</str>
> </lst>
> <lst name="slave">
> <str name="enable">${enable.slave:false}</str>
> <!-- url of master, from properties file -->
> <str name="masterUrl">${master.url}</str>
> <!-- how often to check master -->
> <str name="pollInterval">00:00:30</str>
> </lst>
> </requestHandler>
> {code}
> The slave then has this in solrcore.properties:
> {code}
> enable.slave=true
> master.url=URLOFMASTER/replication
> {code}
> and the master has
> {code}
> enable.master=true
> {code}
> I'd be glad to provide more details but I'm not sure what else I can do.
> SOLR-926 may be relevant.
> Thanks.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
