[
https://issues.apache.org/jira/browse/SOLR-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ishan Chattopadhyaya updated SOLR-8687:
---------------------------------------
Description:
I am facing a problem with stress testing SOLR-5944, even though I think this
problem persists in Solr even without my changes.
The symptom is that during a stress test (similar to TestStressReorder), RTG
gets a document which is older version than that of the last acknowledged write.
Possible reason:
{code}
(DUH2's commit())
...
1: if (cmd.softCommit) {
2: // ulog.preSoftCommit();
3: synchronized (solrCoreState.getUpdateLock()) {
4: if (ulog != null) ulog.preSoftCommit(cmd);
5: core.getSearcher(true, false, waitSearcher, true);
6: if (ulog != null) ulog.postSoftCommit(cmd);
7: }
8: callPostSoftCommitCallbacks();
9: }
...
{code}
* Before line 1, there was an update (say id=2) which was in ulog's map. Maps
are, say, map=\{2=LogPtr(1234)\} , prevMap=\{...\} , prevMap2=\{...\}
* Due to line 4 (ulog.preSoftCommit()), the maps were rotated. Now, the id=2 is
in prevMap: map={}, prevMap=\{2=LogPtr(1234)\}, prevMap2=\{...\} . Till now RTG
for id=2 will work.
* Due to line 5, a new searcher is due to be opened. But this is asynchronous,
and lets assume this doesn't complete before few more lines are executed.
* Due to line 6 (ulog.postSoftCommit()), the previous maps are cleared out. Now
the maps are: map={}, prevMap=null, prevMap2=null
* If there's an RTG for id=2, it will not work from the ulog's maps, so it will
fall through to be searched using the last searcher. But, the searcher due to
be opened in line 5 hasn't yet been opened. In this case, the returned document
will be whatever version of id=2 that was present in the previous searcher.
Can someone please confirm if this is a potential problem? If so, any
suggestions for a fix, please? I tried opening a ulog.openRealtimeSearcher() in
the above synchronized block, but the problem still persists, but I haven't
looked into why that could be.
was:
I am facing a problem with stress testing SOLR-5944, even though I think this
problem persists in Solr even without my changes.
The symptom is that during a stress test (similar to TestStressReorder), RTG
gets a document which is older version than that of the last acknowledged write.
Possible reason:
{code}
(DUH2's commit())
...
1: if (cmd.softCommit) {
2: // ulog.preSoftCommit();
3: synchronized (solrCoreState.getUpdateLock()) {
4: if (ulog != null) ulog.preSoftCommit(cmd);
5: core.getSearcher(true, false, waitSearcher, true);
6: if (ulog != null) ulog.postSoftCommit(cmd);
7: }
8: callPostSoftCommitCallbacks();
9: }
...
{code}
* Before line 1, there was an update (say id=2) which was in ulog's map. Maps
are, say, `map={2=LogPtr(1234)}, prevMap={...}, prevMap2={...}`
* Due to line 4 (ulog.preSoftCommit()), the maps were rotated. Now, the id=2 is
in prevMap: `map={}, prevMap={2=LogPtr(1234)}, prevMap2={...}`. Till now RTG
for id=2 will work.
* Due to line 5, a new searcher is due to be opened. But this is asynchronous,
and lets assume this doesn't complete before few more lines are executed.
* Due to line 6 (ulog.postSoftCommit()), the previous maps are cleared out. Now
the maps are: `map={}, prevMap=null, prevMap2=null`
* If there's an RTG for id=2, it will not work from the ulog's maps, so it will
fall through to be searched using the last searcher. But, the searcher due to
be opened in line 5 hasn't yet been opened. In this case, the returned document
will be whatever version of id=2 that was present in the previous searcher.
Can someone please confirm if this is a potential problem? If so, any
suggestions for a fix, please? I tried opening a ulog.openRealtimeSearcher() in
the above synchronized block, but the problem still persists, but I haven't
looked into why that could be.
> Race condition with RTGs during soft commit
> -------------------------------------------
>
> Key: SOLR-8687
> URL: https://issues.apache.org/jira/browse/SOLR-8687
> Project: Solr
> Issue Type: Bug
> Reporter: Ishan Chattopadhyaya
>
> I am facing a problem with stress testing SOLR-5944, even though I think this
> problem persists in Solr even without my changes.
> The symptom is that during a stress test (similar to TestStressReorder), RTG
> gets a document which is older version than that of the last acknowledged
> write.
> Possible reason:
> {code}
> (DUH2's commit())
> ...
> 1: if (cmd.softCommit) {
> 2: // ulog.preSoftCommit();
> 3: synchronized (solrCoreState.getUpdateLock()) {
> 4: if (ulog != null) ulog.preSoftCommit(cmd);
> 5: core.getSearcher(true, false, waitSearcher, true);
> 6: if (ulog != null) ulog.postSoftCommit(cmd);
> 7: }
> 8: callPostSoftCommitCallbacks();
> 9: }
> ...
> {code}
> * Before line 1, there was an update (say id=2) which was in ulog's map. Maps
> are, say, map=\{2=LogPtr(1234)\} , prevMap=\{...\} , prevMap2=\{...\}
> * Due to line 4 (ulog.preSoftCommit()), the maps were rotated. Now, the id=2
> is in prevMap: map={}, prevMap=\{2=LogPtr(1234)\}, prevMap2=\{...\} . Till
> now RTG for id=2 will work.
> * Due to line 5, a new searcher is due to be opened. But this is
> asynchronous, and lets assume this doesn't complete before few more lines are
> executed.
> * Due to line 6 (ulog.postSoftCommit()), the previous maps are cleared out.
> Now the maps are: map={}, prevMap=null, prevMap2=null
> * If there's an RTG for id=2, it will not work from the ulog's maps, so it
> will fall through to be searched using the last searcher. But, the searcher
> due to be opened in line 5 hasn't yet been opened. In this case, the returned
> document will be whatever version of id=2 that was present in the previous
> searcher.
> Can someone please confirm if this is a potential problem? If so, any
> suggestions for a fix, please? I tried opening a ulog.openRealtimeSearcher()
> in the above synchronized block, but the problem still persists, but I
> haven't looked into why that could be.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
