[jira] [Comment Edited] (SOLR-9330) Race condition between core reload and statistics request
[
https://issues.apache.org/jira/browse/SOLR-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507657#comment-15507657
]
Andrey Kudryavtsev edited comment on SOLR-9330 at 9/20/16 8:16 PM:
---
If we are talking about root cause of the race - it looks like things are a
little bit more easily than I thought. There is just another thread executor
in CoreAdminHandler. If there will be API to avoid it, reload operation will
become more synchronous (patch is attached).
I thought that it will require to do something like too_sync.patch, but not
this time probably.
was (Author: werder):
If we are talking about root cause of the race - it looks like things are a
little bit more easily than I thought. There is just another thread executor
in CoreAdminHandler. If there will be API to avoid it, reload operation will
become more synchronous (patch is attached).
It thought that it will require to do something like too_sync.patch, but not
this time probably.
> Race condition between core reload and statistics request
> -
>
> Key: SOLR-9330
> URL: https://issues.apache.org/jira/browse/SOLR-9330
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 5.5
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9330.patch, SOLR-9390.patch, SOLR-9390.patch,
> SOLR-9390.patch, too_sync.patch
>
>
> Things happened that we execute this two requests consecutively in Solr 5.5:
> * Core reload: /admin/cores?action=RELOAD=_coreName_
> * Check core statistics: /_coreName_/admin/mbeans?stats=true
> And sometimes second request ends with this error:
> {code}
> ERROR org.apache.solr.servlet.HttpSolrCall -
> null:org.apache.lucene.store.AlreadyClosedException: this IndexReader is
> closed
> at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:274)
> at
> org.apache.lucene.index.StandardDirectoryReader.getVersion(StandardDirectoryReader.java:331)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.solr.search.SolrIndexSearcher.getStatistics(SolrIndexSearcher.java:2404)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.addMBean(SolrInfoMBeanHandler.java:164)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.getMBeanInfo(SolrInfoMBeanHandler.java:134)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.handleRequestBody(SolrInfoMBeanHandler.java:65)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2082)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:670)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:458)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)
> {code}
> If my understanding of SolrCore internals is correct, it happens because of
> async nature of reload request:
> * New searcher is "registered" in separate thread
> * Old searcher is closed in same separate thread and only after new one is
> registered
> * When old searcher is closing, it removes itself from map with MBeans
> * If statistic requests happens before old searcher is completely removed
> from everywhere - exception can happen.
> What do you think if we will introduce new parameter for reload request which
> makes it fully synchronized? Basically it will force it to call {code}
> SolrCore#getSearcher(boolean forceNew, boolean returnSearcher, final Future[]
> waitSearcher, boolean updateHandlerReopens) {code} with waitSearcher!= null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9330) Race condition between core reload and statistics request
[
https://issues.apache.org/jira/browse/SOLR-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507657#comment-15507657
]
Andrey Kudryavtsev edited comment on SOLR-9330 at 9/20/16 8:09 PM:
---
If we are talking about root cause of the race - it looks like things are a
little bit more easily than I thought. There is just another thread executor
in CoreAdminHandler. If there will be API to avoid it, reload operation will
become more synchronous (patch is attached).
It thought that it will require to do something like too_sync.patch, but not
this time probably.
was (Author: werder):
If we are talking about root cause of the race - it looks like things are a
little bit more easily than I thought. There is just another thread executor
in CoreAdminHandler. If there will be API to avoid it, reload operation will
become more synchronous (patch is attached).
It thought that it will require to do something like SOLR-9390toosync.patch,
but not this time probably.
> Race condition between core reload and statistics request
> -
>
> Key: SOLR-9330
> URL: https://issues.apache.org/jira/browse/SOLR-9330
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 5.5
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9330.patch, SOLR-9390.patch, SOLR-9390.patch,
> SOLR-9390.patch, too_sync.patch
>
>
> Things happened that we execute this two requests consecutively in Solr 5.5:
> * Core reload: /admin/cores?action=RELOAD=_coreName_
> * Check core statistics: /_coreName_/admin/mbeans?stats=true
> And sometimes second request ends with this error:
> {code}
> ERROR org.apache.solr.servlet.HttpSolrCall -
> null:org.apache.lucene.store.AlreadyClosedException: this IndexReader is
> closed
> at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:274)
> at
> org.apache.lucene.index.StandardDirectoryReader.getVersion(StandardDirectoryReader.java:331)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.solr.search.SolrIndexSearcher.getStatistics(SolrIndexSearcher.java:2404)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.addMBean(SolrInfoMBeanHandler.java:164)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.getMBeanInfo(SolrInfoMBeanHandler.java:134)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.handleRequestBody(SolrInfoMBeanHandler.java:65)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2082)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:670)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:458)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)
> {code}
> If my understanding of SolrCore internals is correct, it happens because of
> async nature of reload request:
> * New searcher is "registered" in separate thread
> * Old searcher is closed in same separate thread and only after new one is
> registered
> * When old searcher is closing, it removes itself from map with MBeans
> * If statistic requests happens before old searcher is completely removed
> from everywhere - exception can happen.
> What do you think if we will introduce new parameter for reload request which
> makes it fully synchronized? Basically it will force it to call {code}
> SolrCore#getSearcher(boolean forceNew, boolean returnSearcher, final Future[]
> waitSearcher, boolean updateHandlerReopens) {code} with waitSearcher!= null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9330) Race condition between core reload and statistics request
[
https://issues.apache.org/jira/browse/SOLR-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507657#comment-15507657
]
Andrey Kudryavtsev edited comment on SOLR-9330 at 9/20/16 8:09 PM:
---
If we are talking about root cause of the race - it looks like things are a
little bit more easily than I thought. There is just another thread executor
in CoreAdminHandler. If there will be API to avoid it, reload operation will
become more synchronous (patch is attached).
It thought that it will require to do something like SOLR-9390toosync.patch,
but not this time probably.
was (Author: werder):
If we are talking about root cause of the race - it looks like things are a
little bit more easily than I thought. There is just another thread executor
in CoreAdminHandler. If there will be API to avoid it, reload operation will
become more synchronous (patch is attached).
It thought that it will require to do something like SOLR-9390_too_sync.patch,
but not this time probably.
> Race condition between core reload and statistics request
> -
>
> Key: SOLR-9330
> URL: https://issues.apache.org/jira/browse/SOLR-9330
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 5.5
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9330.patch, SOLR-9390.patch, SOLR-9390.patch,
> SOLR-9390.patch, SOLR-9390_too_sync.patch
>
>
> Things happened that we execute this two requests consecutively in Solr 5.5:
> * Core reload: /admin/cores?action=RELOAD=_coreName_
> * Check core statistics: /_coreName_/admin/mbeans?stats=true
> And sometimes second request ends with this error:
> {code}
> ERROR org.apache.solr.servlet.HttpSolrCall -
> null:org.apache.lucene.store.AlreadyClosedException: this IndexReader is
> closed
> at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:274)
> at
> org.apache.lucene.index.StandardDirectoryReader.getVersion(StandardDirectoryReader.java:331)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.solr.search.SolrIndexSearcher.getStatistics(SolrIndexSearcher.java:2404)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.addMBean(SolrInfoMBeanHandler.java:164)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.getMBeanInfo(SolrInfoMBeanHandler.java:134)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.handleRequestBody(SolrInfoMBeanHandler.java:65)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2082)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:670)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:458)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)
> {code}
> If my understanding of SolrCore internals is correct, it happens because of
> async nature of reload request:
> * New searcher is "registered" in separate thread
> * Old searcher is closed in same separate thread and only after new one is
> registered
> * When old searcher is closing, it removes itself from map with MBeans
> * If statistic requests happens before old searcher is completely removed
> from everywhere - exception can happen.
> What do you think if we will introduce new parameter for reload request which
> makes it fully synchronized? Basically it will force it to call {code}
> SolrCore#getSearcher(boolean forceNew, boolean returnSearcher, final Future[]
> waitSearcher, boolean updateHandlerReopens) {code} with waitSearcher!= null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9330) Race condition between core reload and statistics request
[
https://issues.apache.org/jira/browse/SOLR-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389833#comment-15389833
]
Joel Bernstein edited comment on SOLR-9330 at 7/22/16 5:04 PM:
---
We are currently looking at exactly this issue with Alfresco's Solr
integration. So we'll be happy to help find the right fix for this.
was (Author: joel.bernstein):
We are currently looking at exactly this issue at Alfresco. So we'll be happy
to help find the right fix for this.
> Race condition between core reload and statistics request
> -
>
> Key: SOLR-9330
> URL: https://issues.apache.org/jira/browse/SOLR-9330
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 5.5
>Reporter: Andrey Kudryavtsev
>
> Things happened that we execute this two requests consequentially in Solr 5.5:
> * Core reload: /admin/cores?action=RELOAD=_coreName_
> * Check core statistics: /_coreName_/admin/mbeans?stats=true
> And sometimes second request ends with this error:
> {code}
> ERROR org.apache.solr.servlet.HttpSolrCall -
> null:org.apache.lucene.store.AlreadyClosedException: this IndexReader is
> closed
> at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:274)
> at
> org.apache.lucene.index.StandardDirectoryReader.getVersion(StandardDirectoryReader.java:331)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
> at
> org.apache.solr.search.SolrIndexSearcher.getStatistics(SolrIndexSearcher.java:2404)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.addMBean(SolrInfoMBeanHandler.java:164)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.getMBeanInfo(SolrInfoMBeanHandler.java:134)
> at
> org.apache.solr.handler.admin.SolrInfoMBeanHandler.handleRequestBody(SolrInfoMBeanHandler.java:65)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2082)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:670)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:458)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)
> {code}
> If my understanding of SolrCore internals is correct, it happens because of
> async nature of reload request:
> * New searcher is "registered" in separate thread
> * Old searcher is closed in same separate thread and only after new one is
> registered
> * When old searcher is closing, it removes itself from map with MBeans
> * If statistic requests happens before old searcher is completely removed
> from everywhere - exception can happen.
> What do you think if we will introduce new parameter for reload request which
> make it fully synchronized? Basically it will force it to call {code}
> SolrCore#getSearcher(boolean forceNew, boolean returnSearcher, final Future[]
> waitSearcher, boolean updateHandlerReopens) {code} with waitSearcher!= null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
