[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893461#comment-13893461
]
Hoss Man commented on SOLR-5658:
mark: since this issue was recorded as fixed in 4.6.1 changes, re-opening now
to address the problem it may have caused seems like a bad idea from an
accountability standpoint -- since if/when it's fixed, it will be confusing to
users if it gets re-recorded in CHANGES under 4.7 (or whatever)
Suggest you re-resolve this, and open a new linked (Broken By) issue for the
newly discovered problem in 4.6.1.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893483#comment-13893483
]
Mark Miller commented on SOLR-5658:
---
Reopen is just so its not lost until we figure what if anything we do.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891608#comment-13891608
]
Jessica Cheng commented on SOLR-5658:
-
Hi Mark,
Is the change of serialization of docMap from Map to List necessary? Looks like
the unmarshaled docMap isn't used anymore either, but the casting there is
causing a ClassCastException: java.util.LinkedHashMap cannot be cast to
java.util.List when interacting with an older solrj client.
Thanks,
Jessica
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1]
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891613#comment-13891613
]
Yonik Seeley commented on SOLR-5658:
Ah, this explains the issue our partner was having testing out HDS 4.6.1 - he
didn't upgrade the entire cluster from 4.6.0 at once and got an exception in
JavaBinCodec complaining about Unknown type 19
Not sure if there is much we can do about it now given that 4.6.1 has been
released.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891636#comment-13891636
]
Mark Miller commented on SOLR-5658:
---
Oh darn, that's no good. Need to fix that for 4.7.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891637#comment-13891637
]
Mark Miller commented on SOLR-5658:
---
bq. Is the change of serialization of docMap from Map to List necessary?
It's part of supporting the iterator/streaming case. It's needed because of how
things currently work. However, we shouldn't break with a class case when using
an older client - we should have the same old bad behavior.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891643#comment-13891643
]
Mark Miller commented on SOLR-5658:
---
We should probably add the ability to configure what format distributed updates
will use internally so that you can tmp flip to xml or something for this type
of issue.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891647#comment-13891647
]
Jessica Cheng commented on SOLR-5658:
-
Thanks for looking at it Mark!
If you don't mind me asking, the one thing I didn't understand is why docMap is
needed. The line in unmarshal
quotedocMap = (ListEntrySolrInputDocument,MapObject,Object)
namedList[0].get(docsMap);/quote
loads docMap from the named list but the docMap variable doesn't seem to be
used anywhere. Also, a text search of docsMap seems to indicate that
JavaBinUpdateRequestCodec is the only class using it. What am I missing?
Thanks,
Jessica
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler –
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891649#comment-13891649
]
Mark Miller commented on SOLR-5658:
---
bq. exception in JavaBinCodec complaining about Unknown type 19
I hadn't considered going the other way - new client to old.
That's a bummer. Sucks this was implemented wrong first.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 4.6.1, 5.0, 4.7
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887802#comment-13887802
]
Mark Miller commented on SOLR-5658:
---
Thanks Shalin! Your call on the new issue or not.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886604#comment-13886604
]
ASF subversion and git services commented on SOLR-5658:
---
Commit 1562836 from sha...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1562836 ]
SOLR-5658: Removing System.out.println in JavaBinUpdatedRequestCodec added for
debugging
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886608#comment-13886608
]
Shalin Shekhar Mangar commented on SOLR-5658:
-
Perhaps I should remove this println as another issue because this has already
been released?
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886676#comment-13886676
]
ASF subversion and git services commented on SOLR-5658:
---
Commit 1562860 from sha...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1562860 ]
SOLR-5658: Removing System.out.println in JavaBinUpdatedRequestCodec added for
debugging
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882849#comment-13882849
]
Erik Hatcher commented on SOLR-5658:
[~markmil...@gmail.com] Is this ticket complete as of Solr 4.6.1? Just
wondering if it can be closed. Thanks!
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879844#comment-13879844
]
Shalin Shekhar Mangar commented on SOLR-5658:
-
Thanks for reporting this Varun.
The thing to note between the 4.5 logs and the 4.6.x logs is that in 4.5, there
are two commit statements per node (1 soft commit with openSearcher=true and
another hard commit with openSearcher=false) whereas in 4.6.x there is only one
commit (hard commit with openSearcher=false). We need to find out why.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879849#comment-13879849
]
Shalin Shekhar Mangar commented on SOLR-5658:
-
Also reported by a user (thanks Varun for pointing it out to me privately):
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879998#comment-13879998
]
Mark Miller commented on SOLR-5658:
---
This is probably related to what params we pass on and which we filter out in
DistributedUpdateProcessor.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@36137e08
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880013#comment-13880013
]
Mark Miller commented on SOLR-5658:
---
Patch adds a test and forwards on the commitWithin param.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@36137e08
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880073#comment-13880073
]
Shalin Shekhar Mangar commented on SOLR-5658:
-
Thanks Mark. Should we re-spin 4.6.1 for this?
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@36137e08
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880086#comment-13880086
]
Mark Miller commented on SOLR-5658:
---
I don't know. If there is support for it, I'm certainly willing to do it, but
I'm not sure it's worth an RC4 myself. I think this issue has been around a
very long time, and it's just random luck something seemed like it worked on
4.5.1 or older in any simple testing.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880114#comment-13880114
]
Daniel Collins commented on SOLR-5658:
--
Mark, what's the impact of this issue? Are you saying that soft commits were
never distributed (which seems quite a big deal!), or is it more subtle than
that?
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880117#comment-13880117
]
Mark Miller commented on SOLR-5658:
---
Right - it was not distributed. Not since we started filtering most parameters,
which was a very long time ago.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880139#comment-13880139
]
Daniel Collins commented on SOLR-5658:
--
Ok, was just puzzled about how our system is working then (4.4.0), we
consistently see softCommits running on the replicas, maybe it is autoCommit
firing instead...
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1]
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880226#comment-13880226
]
Shalin Shekhar Mangar commented on SOLR-5658:
-
bq. I think this issue has been around a very long time, and it's just random
luck something seemed like it worked on 4.5.1 or older in any simple testing.
I'm not sure about that. The commitWithin is also set in the AddUpdateCommand
in addition to the request params. I ran your test against 4.5 without the fix
for 5 times and it didn't fail. But it never passes with trunk (without the
fix) so I think there may be another bug introduced with the streaming changes.
I'll look at this again tomorrow my time.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880244#comment-13880244
]
Mark Miller commented on SOLR-5658:
---
I think it just depends on if it's a request wide or document level
commitWithin? If it was request level, the only way it would have worked
previously is if there was some code that looked for the commitWithin param
explicitly and set the commitWithin on the AddUpdateCommand - I don't recall
something like that.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880253#comment-13880253
]
Mark Miller commented on SOLR-5658:
---
Okay, outside of the distributed code, Solr does setup the cmd object with a
request level commitWithin. That is how this used to work, that is why we
didn't have to propagate the param.
Perhaps the commitWithin is being lost when parsing the javabin.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880256#comment-13880256
]
Mark Miller commented on SOLR-5658:
---
I see the bug.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@36137e08
main{StandardDirectoryReader(segments_2:7:nrt
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880298#comment-13880298
]
Mark Miller commented on SOLR-5658:
---
I'm still look at the details, but the cause in the change of behavior is the
switch from xml to binary update requests.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880472#comment-13880472
]
Mark Miller commented on SOLR-5658:
---
Kind of ugly - the real fix will take a bit of work. The patch might be a
decent partial fix / workaround for 4.6.1 if we respin.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880569#comment-13880569
]
ASF subversion and git services commented on SOLR-5658:
---
Commit 1560859 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1560859 ]
SOLR-5658: commitWithin and overwrite are not being distributed to replicas now
that SolrCloud uses javabin to distribute updates.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880573#comment-13880573
]
ASF subversion and git services commented on SOLR-5658:
---
Commit 1560860 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1560860 ]
SOLR-5658: commitWithin and overwrite are not being distributed to replicas now
that SolrCloud uses javabin to distribute updates.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Shalin Shekhar Mangar
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher –
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880588#comment-13880588
]
ASF subversion and git services commented on SOLR-5658:
---
Commit 1560866 from [~markrmil...@gmail.com] in branch
'dev/branches/lucene_solr_4_6'
[ https://svn.apache.org/r1560866 ]
SOLR-5658: commitWithin and overwrite are not being distributed to replicas now
that SolrCloud uses javabin to distribute updates.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880692#comment-13880692
]
Mark Miller commented on SOLR-5658:
---
As a separate issue, it probably makes sense to send request level commitWithin
as a param rather than setting it per doc - that would mean less repeated data
in the request. We still need to properly support per doc like this as well
though, because that is the level cmd objects support and we are distributing
cmd objects.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880716#comment-13880716
]
Shalin Shekhar Mangar commented on SOLR-5658:
-
This was nasty. Thanks for fixing and back-porting this, Mark!
I opened SOLR-5660 to send request level commitWithin as a param.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler –
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the new documents added
[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880726#comment-13880726
]
Mark Miller commented on SOLR-5658:
---
Another issue this fixed is that the documents were being serialized and sent
twice - though they were not processed twice, so just wasteful and not
functionally problematic.
commitWithin does not reflect the new documents added
-
Key: SOLR-5658
URL: https://issues.apache.org/jira/browse/SOLR-5658
Project: Solr
Issue Type: Bug
Affects Versions: 4.6, 5.0
Reporter: Varun Thacker
Assignee: Mark Miller
Priority: Critical
Fix For: 5.0, 4.7, 4.6.1
Attachments: SOLR-5658.patch, SOLR-5658.patch
I start 4 nodes using the setup mentioned on -
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
I added a document using -
curl http://localhost:8983/solr/update?commitWithin=1 -H Content-Type:
text/xml --data-binary 'adddocfield
name=idtestdoc/field/doc/add'
In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with
openSearcher=false
So even after 10 seconds queries on none of the shards reflect the added
document.
This was also reported on the solr-user list (
http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
)
Here are the relevant logs
Logs from Solr 4.5.1
Node 1:
{code}
420021 [qtp619011445-12] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={commitWithin=1} {add=[testdoc]} 0 45
{code}
Node 2:
{code}
119896 [qtp1608701025-10] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:8983/solr/collection1/update.distrib=TOLEADERwt=javabinversion=2}
{add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@e174f70
main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO
org.apache.solr.update.UpdateHandler – end_commit_flush
{code}
Node 3:
Node 4:
{code}
374545 [qtp1608701025-16] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update
params={distrib.from=http://192.168.1.103:7574/solr/collection1/update.distrib=FROMLEADERwt=javabinversion=2}
{add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO
org.apache.solr.update.UpdateHandler – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO
