[
https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880226#comment-13880226
]
Shalin Shekhar Mangar commented on SOLR-5658:
---------------------------------------------
bq. I think this issue has been around a very long time, and it's just random
luck something seemed like it worked on 4.5.1 or older in any simple testing.
I'm not sure about that. The commitWithin is also set in the AddUpdateCommand
in addition to the request params. I ran your test against 4.5 without the fix
for 5 times and it didn't fail. But it never passes with trunk (without the
fix) so I think there may be another bug introduced with the streaming changes.
I'll look at this again tomorrow my time.
> commitWithin does not reflect the new documents added
> -----------------------------------------------------
>
> Key: SOLR-5658
> URL: https://issues.apache.org/jira/browse/SOLR-5658
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.6, 5.0
> Reporter: Varun Thacker
> Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-5658.patch
>
>
> I start 4 nodes using the setup mentioned on -
> https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
>
> I added a document using -
> curl http://localhost:8983/solr/update?commitWithin=10000 -H "Content-Type:
> text/xml" --data-binary '<add><doc><field
> name="id">testdoc</field></doc></add>'
> In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit
> with openSearcher=false
> In Solr 4.6.x there is there is only one commit hard commit with
> openSearcher=false
>
> So even after 10 seconds queries on none of the shards reflect the added
> document.
> This was also reported on the solr-user list (
> http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html
> )
> Here are the relevant logs
> Logs from Solr 4.5.1
> Node 1:
> {code}
> 420021 [qtp619011445-12] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 45
> {code}
>
> Node 2:
> {code}
> 119896 [qtp1608701025-10] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update
> params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2}
> {add=[testdoc (1458003295513608192)]} 0 348
> 129648 [commitScheduler-8-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> 129679 [commitScheduler-8-thread-1] INFO
> org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
> 129680 [commitScheduler-8-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
> 129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
> QuerySenderListener sending requests to Searcher@e174f70
> main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
> 129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
> QuerySenderListener done.
> 129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
> [collection1] Registered new searcher Searcher@e174f70
> main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
> 134648 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> SolrDeletionPolicy.onCommit: commits: num=2
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
> 134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> newest commit generation = 4
> 134660 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
>
> Node 3:
>
> Node 4:
> {code}
> 374545 [qtp1608701025-16] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update
> params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2}
> {add=[testdoc (1458002133233172480)]} 0 20
> 384545 [commitScheduler-8-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> 384552 [commitScheduler-8-thread-1] INFO
> org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
> 384553 [commitScheduler-8-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
> 384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
> QuerySenderListener sending requests to Searcher@36137e08
> main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
> 384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
> QuerySenderListener done.
> 384554 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore –
> [collection1] Registered new searcher Searcher@36137e08
> main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
> 389545 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 389549 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> SolrDeletionPolicy.onCommit: commits: num=2
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
> 389550 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> newest commit generation = 3
> 389551 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
>
>
> Using Solr 4.6
>
> Node 1:
> {code}
> 124513 [qtp1314570047-13] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 348
> {code}
> Node 2:
> {code}
> 101586 [qtp1608701025-13] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update
> params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2}
> {add=[testdoc (1458003613357965312)]} 0 217
> 116407 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 116429 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> SolrDeletionPolicy.onCommit: commits: num=2
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> 116430 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> newest commit generation = 2
> 116444 [commitScheduler-7-thread-1] INFO
> org.apache.solr.search.SolrIndexSearcher – Opening Searcher@75e32318 realtime
> 116445 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
>
> Node 3:
>
>
> Node 4:
> {code}
> 68183 [qtp1338008566-14] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update
> params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2}
> {add=[testdoc (1458003613357965312)]} 0 43
> 83183 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler
> – start
> commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 83207 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> SolrDeletionPolicy.onCommit: commits: num=2
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> 83208 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> newest commit generation = 2
> 83220 [commitScheduler-7-thread-1] INFO
> org.apache.solr.search.SolrIndexSearcher – Opening Searcher@326f944c realtime
> 83220 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler
> – end_commit_flush
> {code}
>
> Using Solr 4.6.1
>
> Node 1:
> {code}
> 301363 [qtp619011445-15] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 32
> {code}
> Node 2:
> {code}
> 207000 [qtp619011445-17] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update
> params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2}
> {add=[testdoc (1458004563169640448)]} 0 28
> 221974 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 221987 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> SolrDeletionPolicy.onCommit: commits: num=2
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
> 221987 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> newest commit generation = 3
> 221989 [commitScheduler-7-thread-1] INFO
> org.apache.solr.search.SolrIndexSearcher – Opening Searcher@132713fa realtime
> 221990 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
> Node 3:
>
> Node 4:
> {code}
> 193133 [qtp1608701025-16] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
> webapp=/solr path=/update
> params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2}
> {add=[testdoc (1458004563169640448)]} 0 23
> 208133 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 208141 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> SolrDeletionPolicy.onCommit: commits: num=2
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
>
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
> 208141 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore –
> newest commit generation = 3
> 208144 [commitScheduler-7-thread-1] INFO
> org.apache.solr.search.SolrIndexSearcher – Opening Searcher@3171c7df realtime
> 208146 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
