[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2017-08-03 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113230#comment-16113230
 ] 

David Smiley commented on LUCENE-7188:
--

The change is in the 5x series, just not back in 5.1 in particular.  You could 
back-port this yourself, in a fork.  Alternatively, you could probably fork 
NRTCachingDirectoryFactory with the related classes to fix yourself locally, 
then plug into solrconfig.

> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>Assignee: Yonik Seeley
> Fix For: 6.1, 6.0.1, 5.5.1, 7.0
>
> Attachments: LUCENE-7188.patch
>
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, 
> _9v_Lucene50_0.doc, _9v_Lucene50_0.pos, 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2017-08-03 Thread Muktesh Mehta (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112334#comment-16112334
 ] 

Muktesh Mehta commented on LUCENE-7188:
---

[~yo...@apache.org] [~dsmi...@mac.com] We are using Solr 5.1 and facing same 
issue. Is the only way to solve this issue upgrading Solr version? Can 
[https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b31c398] fix be 
backported  to 5.1?

> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>Assignee: Yonik Seeley
> Fix For: 6.1, 6.0.1, 5.5.1, 7.0
>
> Attachments: LUCENE-7188.patch
>
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, 
> _9v_Lucene50_0.doc, _9v_Lucene50_0.pos, 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-05-19 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291315#comment-15291315
 ] 

ASF subversion and git services commented on LUCENE-7188:
-

Commit b31c39812ec0e6017afe20568bb21d44aef2fa78 in lucene-solr's branch 
refs/heads/branch_6_0 from [~yo...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b31c398 ]

LUCENE-7188: remove incorrect sanity check in NRTCachingDirectory.listAll() 
that throws IllegalStateException


> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>Assignee: Yonik Seeley
> Fix For: 6.1, 6.0.1, 5.5.1, master (7.0)
>
> Attachments: LUCENE-7188.patch
>
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-11 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236633#comment-15236633
 ] 

David Smiley commented on LUCENE-7188:
--

I think you're right Yonik.  At first I was about to disagree to write that we 
should fix this by having listAll hold both locks, but I now think there's no 
point to it.  We know exactly why listAll can momentarily find a file in both 
places, and it's okay. It's a race condition of no consequence.  The file is 
still there to return from listAll; we know what to do.  There are other race 
conditions of no consequences too that have comments {{"// Another thread beat 
us..."}}.

listAll can use Collections.addAll like so to replace two for loops:
{code:java}
Collections.addAll(files, cache.listAll());
Collections.addAll(files, in.listAll());
{code}

I'm unsure why listAll needs any synchronization at all, for that matter.  It's 
an omission that the Directory API says nothing about thread-safety; 
nevertheless I assume instances should be thread-safe.  

Everything below could be its own JIRA issue:

Can we remove _all_ synchronization on "this" (not substituting it with 
anything else either) and change methods to not use/depend on calling 
{{cache.fileNameExists()}}?  Using that method as a guard is, I think, what led 
to synchronizing on "this" in the first place.  For example, fileLength could 
be rewritten to this:
{code:java}
public long fileLength(String name) throws IOException {
try {
  return cache.fileLength(name);
} catch (FileNotFoundException | NoSuchFileException e) {
  return in.fileLength(name);
}
  }
{code}
 -- and likewise for deleteFile and openInput.  Then the synchronized(this) 
inside unCache can be removed too, leaving none left.

unCache calls {{cache.fileNameExists()}} too but I think it's okay since it 
merely exits early.   

unCache's use of uncacheLock could probably be changed to be a per-file name 
lock instead of one shared lock for the directory instance since it only needs 
to lock per file name it's un-caching.  Although I'm not sure it's worth the 
complexity bothering.

> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>Assignee: Yonik Seeley
> Fix For: master, 6.1, 5.5.1
>
> Attachments: LUCENE-7188.patch
>
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-11 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236549#comment-15236549
 ] 

ASF subversion and git services commented on LUCENE-7188:
-

Commit ace13b5728df97664ae21e617de9e6ae3706657e in lucene-solr's branch 
refs/heads/branch_6x from [~yo...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ace13b5 ]

LUCENE-7188: remove incorrect sanity check in NRTCachingDirectory.listAll() 
that throws IllegalStateException


> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>Assignee: Yonik Seeley
> Fix For: 6.1
>
> Attachments: LUCENE-7188.patch
>
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-11 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236550#comment-15236550
 ] 

ASF subversion and git services commented on LUCENE-7188:
-

Commit a49b748b58b31af51d06c1d360777d679285e8fd in lucene-solr's branch 
refs/heads/branch_5_5 from [~yo...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a49b748 ]

LUCENE-7188: remove incorrect sanity check in NRTCachingDirectory.listAll() 
that throws IllegalStateException


> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>Assignee: Yonik Seeley
> Fix For: 6.1
>
> Attachments: LUCENE-7188.patch
>
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-11 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236548#comment-15236548
 ] 

ASF subversion and git services commented on LUCENE-7188:
-

Commit c1a70f31a605ac254c4c5d556444659aaa3201e5 in lucene-solr's branch 
refs/heads/master from [~yo...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c1a70f3 ]

LUCENE-7188: remove incorrect sanity check in NRTCachingDirectory.listAll() 
that throws IllegalStateException


> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>Assignee: Yonik Seeley
> Fix For: 6.1
>
> Attachments: LUCENE-7188.patch
>
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-11 Thread Yonik Seeley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235981#comment-15235981
 ] 

Yonik Seeley commented on LUCENE-7188:
--

I've also seen this exception/bug crop up when running chaos monkey tests.

David, you are correct.  LUCENE-5945 caused this bug by adding the exception 
without additional needed synchronization.  It can be fixed by either removing 
the exception or adding the additional synchronization.


> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, 
> _9v_Lucene50_0.doc, _9v_Lucene50_0.pos, _9v_Lucene50_0.tim, 
> _9v_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, 
> 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-07 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230471#comment-15230471
 ] 

David Smiley commented on LUCENE-7188:
--

Wow. The F-that part was uncalled for; I wish you could keep your cool and 
respectively disagree with me.  What do you think of my analysis of the bug?

> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, 
> _9v_Lucene50_0.doc, _9v_Lucene50_0.pos, _9v_Lucene50_0.tim, 
> _9v_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, 
> _9w_Lucene50_0.doc, _9w_Lucene50_0.pos, _9w_Lucene50_0.tim, 
> _9w_Lucene50_0.tip, _9x.fdt, _9x.fdx, _9x.fnm, _9x.nvd, _9x.nvm, _9x.si, 
> 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-07 Thread Robert Muir (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230413#comment-15230413
 ] 

Robert Muir commented on LUCENE-7188:
-

fuck that. This is the correct change: since the leniency around directory 
creation was removed in that very issue, it was the correct place to remove the 
leniency.

> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm, _9t.nvd, _9t.nvm, _9t.si, _9t_6.liv, 
> _9t_Lucene50_0.doc, _9t_Lucene50_0.pos, _9t_Lucene50_0.tim, 
> _9t_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, 
> _9u_Lucene50_0.doc, _9u_Lucene50_0.pos, _9u_Lucene50_0.tim, 
> _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, 
> _9v_Lucene50_0.doc, _9v_Lucene50_0.pos, _9v_Lucene50_0.tim, 
> _9v_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, 
> _9w_Lucene50_0.doc, _9w_Lucene50_0.pos, _9w_Lucene50_0.tim, 
> _9w_Lucene50_0.tip, _9x.fdt, _9x.fdx, _9x.fnm, _9x.nvd, 

[jira] [Commented] (LUCENE-7188) IllegalStateException in NRTCachingDirectory.listAll

2016-04-07 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230391#comment-15230391
 ] 

David Smiley commented on LUCENE-7188:
--

(I converted this to a a Lucene issue in JIRA)

bq. Stuff is broken if NRTCachingDirectory thinks a file is both cached in ram 
and on disk, the file is "in two places"

Makes sense, though the circumstance seems benign for listAll().  It appears 
there is a race condition between listAll() and unCache().  Notice listAll() 
syncs on "this".  unCache only synchronizes on "this" to delete the file form 
the cache _after_ it has already written to the real backing directory.  

bq.  it was no mistake or "WIP"

Next time, unless it's a trivial change (e.g. formatting, docs, ...) please 
create separate issues for changes that aren't within the scope of the issue 
you're including it on.  Possibly as a required-by if there is a dependency.


> IllegalStateException in NRTCachingDirectory.listAll
> 
>
> Key: LUCENE-7188
> URL: https://issues.apache.org/jira/browse/LUCENE-7188
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.2.1
> Environment: Production, QA
>Reporter: Semion Mc Alice
>
> Hey,
> we are getting IllegalStateException in 2 different circumstances. The first 
> one is on Status calls:
> {noformat}
> ERROR - 2016-02-01 22:32:43.164; [   ] org.apache.solr.common.SolrException; 
> org.apache.solr.common.SolrException: Error handling 'status' action 
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:748)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:228)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:193)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at 
> org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalStateException: file: 
> MMapDirectory@D:\Solr\server\solr\Prod_Core1_shard1_replica2\data\index 
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d307e5 appears both 
> in delegate and in cache: cache=[_a0.fdt, _9t_7.liv, _a0.fdx, 
> _a0_Lucene50_0.tip, _a0.nvm, _a0_Lucene50_0.doc, _a0_Lucene50_0.tim, _a0.fnm, 
> _a0_Lucene50_0.pos, _a0.si],delegate=[pending_segments_93, segments_92, 
> write.lock, _9t.fdt, _9t.fdx, _9t.fnm,