date:20150514


[ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543736#comment-14543736
 ] 

Arcadius Ahouansou commented on LUCENE-6464:


Thanks [~mikemccand] for the review.
Will do those minor changes today.

Please, have you also noticed the TODO and comments about the usage of 
{code}MatchAllDocsQuery(){code} that I have introduced?

Thanks.


 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch, LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7543) Create GraphQuery that allows graph traversal as a query operator.

2015-05-14 Thread Kevin Watters (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kevin Watters updated SOLR-7543:

Description:
I have a GraphQuery that I implemented a long time back that allows a user to
specify a startQuery to identify which documents to start graph traversal
from. It then gathers up the edge ids for those documents , optionally applies
an additional filter. The query is then re-executed continually until no new
edge ids are identified. I am currently hosting this code up at
https://github.com/kwatters/solrgraph and I would like to work with the
community to get some feedback and ultimately get it committed back in as a
lucene query.

Here's a bit more of a description of the parameters for the query / graph
traversal:

q - the initial start query that identifies the universe of documents to start
traversal from.
fromField - the field name that contains the node id
toField - the name of the field that contains the edge id(s).
traversalFilter - this is an additional query that can be supplied to limit the
scope of graph traversal to just the edges that satisfy the traversalFilter
query.
maxDepth - integer specifying how deep the breadth first search should go.
returnStartNodes - boolean to determine if the documents that matched the
original q should be returned as part of the graph.
onlyLeafNodes - boolean that filters the graph query to only return
documents/nodes that have no edges.

We identify a set of documents with q as any arbitrary lucene query. It will
collect the values in the fromField, create an OR query with those values ,
optionally apply an additional constraint from the traversalFilter and walk
the result set until no new edges are detected. Traversal can also be stopped
at N hops away as defined with the maxDepth. This is a BFS (Breadth First
Search) algorithm. Cycle detection is done by not revisiting the same document
for edge extraction.

This query operator does not keep track of how you arrived at the document, but
only that the traversal did arrive at the document.

was:I have a GraphQuery that I implemented a long time back that allows a
user to specify a seedQuery to identify which documents to start graph
traversal from. It then gathers up the edge ids for those documents ,
optionally applies an additional filter. The query is then re-executed
continually until no new edge ids are identified. I am currently hosting this
code up at https://github.com/kwatters/solrgraph and I would like to work with
the community to get some feedback and ultimately get it committed back in as a
lucene query.

Create GraphQuery that allows graph traversal as a query operator.
--

Key: SOLR-7543
URL: https://issues.apache.org/jira/browse/SOLR-7543
Project: Solr
Issue Type: New Feature
Components: search
Reporter: Kevin Watters
Priority: Minor

I have a GraphQuery that I implemented a long time back that allows a user to
specify a startQuery to identify which documents to start graph traversal
from. It then gathers up the edge ids for those documents , optionally
applies an additional filter. The query is then re-executed continually
until no new edge ids are identified. I am currently hosting this code up at
https://github.com/kwatters/solrgraph and I would like to work with the
community to get some feedback and ultimately get it committed back in as a
lucene query.
Here's a bit more of a description of the parameters for the query / graph
traversal:
q - the initial start query that identifies the universe of documents to
start traversal from.
fromField - the field name that contains the node id
toField - the name of the field that contains the edge id(s).
traversalFilter - this is an additional query that can be supplied to limit
the scope of graph traversal to just the edges that satisfy the
traversalFilter query.
maxDepth - integer specifying how deep the breadth first search should go.
returnStartNodes - boolean to determine if the documents that matched the
original q should be returned as part of the graph.
onlyLeafNodes - boolean that filters the graph query to only return
documents/nodes that have no edges.
We identify a set of documents with q as any arbitrary lucene query. It
will collect the values in the fromField, create an OR query with those
values , optionally apply an additional constraint from the traversalFilter
and walk the result set until no new edges are detected. Traversal can also
be stopped at N hops away as defined with the maxDepth. This is a BFS
(Breadth First Search) algorithm. Cycle detection is done by not revisiting
the same document for edge extraction.
This query operator does not keep track of how you arrived

[jira] [Assigned] (SOLR-7517) When you can't persist a core.properties file we simply log an error and continue.

2015-05-14 Thread Mark Miller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-7517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller reassigned SOLR-7517:
-

Assignee: Mark Miller

 When you can't persist a core.properties file we simply log an error and 
 continue.
 --

 Key: SOLR-7517
 URL: https://issues.apache.org/jira/browse/SOLR-7517
 Project: Solr
  Issue Type: Bug
Reporter: Mark Miller
Assignee: Mark Miller
Priority: Critical

 This means a user usually won't know none of his cores are persisted. He runs 
 and indexes for days. Then restarts. All the cores are gone.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_45) - Build # 12675 - Failure!

2015-05-14 Thread ASF subversion and git services (JIRA)

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/12675/
Java: 32bit/jdk1.8.0_45 -server -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 1179 lines...]
   [junit4] JVM J0: stdout was not empty, see: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/test/temp/junit4-J0-20150514_132109_530.sysout
   [junit4]  JVM J0: stdout (verbatim) 
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  SIGSEGV (0xb) at pc=0xf6ea4481, pid=19502, tid=3862952768
   [junit4] #
   [junit4] # JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 
1.8.0_45-b14)
   [junit4] # Java VM: Java HotSpot(TM) Server VM (25.45-b02 mixed mode 
linux-x86 )
   [junit4] # Problematic frame:
   [junit4] # V  [libjvm.so+0x72c481]  
ObjArrayKlass::oop_oop_iterate_nv(oopDesc*, ParScanWithoutBarrierClosure*)+0x61
   [junit4] #
   [junit4] # Failed to write core dump. Core dumps have been disabled. To 
enable core dumping, try ulimit -c unlimited before starting Java again
   [junit4] #
   [junit4] # An error report file with more information is saved as:
   [junit4] # 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/test/J0/hs_err_pid19502.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
   [junit4] #
   [junit4]  JVM J0: EOF 

[...truncated 521 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: 
/home/jenkins/tools/java/32bit/jdk1.8.0_45/jre/bin/java -server 
-XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/heapdumps -ea 
-esa -Dtests.prefix=tests -Dtests.seed=9FEB405654808AD8 -Xmx512M -Dtests.iters= 
-Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random 
-Dtests.postingsformat=random -Dtests.docvaluesformat=random 
-Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.0.0 
-Dtests.cleanthreads=perMethod 
-Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/logging.properties
 -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false 
-Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp 
-Djava.io.tmpdir=./temp 
-Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/core/test/temp
 -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene 
-Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/clover/db
 
-Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/junit4/tests.policy
 -Dtests.LUCENE_VERSION=6.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory 
-Djava.awt.headless=true -Djdk.map.althashing.threshold=0 
-Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true 
-Djava.security.manager=org.apache.lucene.util.TestSecurityManager 
-Dfile.encoding=ISO-8859-1 -classpath

[jira] [Commented] (SOLR-7524) Make Streaming Expressions Java 7 Compatible


[ 
https://issues.apache.org/jira/browse/SOLR-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543705#comment-14543705
 ] 

ASF subversion and git services commented on SOLR-7524:
---

Commit 1679376 from [~joel.bernstein] in branch 'dev/trunk'
[ https://svn.apache.org/r1679376 ]

SOLR-7377,SOLR-7524:Make Streaming Expressions Java 7 Compatible

 Make Streaming Expressions Java 7 Compatible
 

 Key: SOLR-7524
 URL: https://issues.apache.org/jira/browse/SOLR-7524
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Affects Versions: Trunk
Reporter: Dennis Gove
Priority: Trivial
 Fix For: 5.2

 Attachments: SOLR-7524.patch


 SOLR-7377 added Streaming Expressions to trunk. It uses, by choice and not 
 necessity, some features of Java 8. This patch is to make minor changes to 
 three files to make Streaming Expressions compatible with Java 7 and therefor 
 able to be included in version 5.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7377) SOLR Streaming Expressions

2015-05-14 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543704#comment-14543704
 ] 

ASF subversion and git services commented on SOLR-7377:
---

Commit 1679376 from [~joel.bernstein] in branch 'dev/trunk'
[ https://svn.apache.org/r1679376 ]

SOLR-7377,SOLR-7524:Make Streaming Expressions Java 7 Compatible

 SOLR Streaming Expressions
 --

 Key: SOLR-7377
 URL: https://issues.apache.org/jira/browse/SOLR-7377
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Reporter: Dennis Gove
Priority: Minor
 Fix For: Trunk, 5.2

 Attachments: SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, 
 SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, 
 SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch


 It would be beneficial to add an expression-based interface to Streaming API 
 described in SOLR-7082. Right now that API requires streaming requests to 
 come in from clients as serialized bytecode of the streaming classes. The 
 suggestion here is to support string expressions which describe the streaming 
 operations the client wishes to perform. 
 {code:java}
 search(collection1, q=*:*, fl=id,fieldA,fieldB, sort=fieldA asc)
 {code}
 With this syntax in mind, one can now express arbitrarily complex stream 
 queries with a single string.
 {code:java}
 // merge two distinct searches together on common fields
 merge(
   search(collection1, q=id:(0 3 4), fl=id,a_s,a_i,a_f, sort=a_f asc, a_s 
 asc),
   search(collection2, q=id:(1 2), fl=id,a_s,a_i,a_f, sort=a_f asc, a_s 
 asc),
   on=a_f asc, a_s asc)
 // find top 20 unique records of a search
 top(
   n=20,
   unique(
 search(collection1, q=*:*, fl=id,a_s,a_i,a_f, sort=a_f desc),
 over=a_f desc),
   sort=a_f desc)
 {code}
 The syntax would support
 1. Configurable expression names (eg. via solrconfig.xml one can map unique 
 to a class implementing a Unique stream class) This allows users to build 
 their own streams and use as they wish.
 2. Named parameters (of both simple and expression types)
 3. Unnamed, type-matched parameters (to support requiring N streams as 
 arguments to another stream)
 4. Positional parameters
 The main goal here is to make streaming as accessible as possible and define 
 a syntax for running complex queries across large distributed systems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7547) SDF should short circuit for static content request

2015-05-14 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544974#comment-14544974
 ] 

ASF subversion and git services commented on SOLR-7547:
---

Commit 1679496 from [~anshumg] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679496 ]

SOLR-7547: Short circuit SolrDispatchFilter for static content request. Right 
now it creates a new HttpSolrCall object and tries to process it.(merge from 
trunk)

 SDF should short circuit for static content request
 ---

 Key: SOLR-7547
 URL: https://issues.apache.org/jira/browse/SOLR-7547
 Project: Solr
  Issue Type: Bug
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Fix For: 5.2

 Attachments: SOLR-7547.patch


 As of now, when we request the Admin UI page, I see those requests coming 
 into SDF and creating the HttpSolrCall object. This shouldn't happen and 
 requests for those paths should just short circuit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7547) SDF should short circuit for static content request

2015-05-14 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-7547:
---
Fix Version/s: 5.2

 SDF should short circuit for static content request
 ---

 Key: SOLR-7547
 URL: https://issues.apache.org/jira/browse/SOLR-7547
 Project: Solr
  Issue Type: Bug
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Fix For: 5.2

 Attachments: SOLR-7547.patch


 As of now, when we request the Admin UI page, I see those requests coming 
 into SDF and creating the HttpSolrCall object. This shouldn't happen and 
 requests for those paths should just short circuit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7547) SDF should short circuit for static content request


[ 
https://issues.apache.org/jira/browse/SOLR-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544966#comment-14544966
 ] 

ASF subversion and git services commented on SOLR-7547:
---

Commit 1679493 from [~anshumg] in branch 'dev/trunk'
[ https://svn.apache.org/r1679493 ]

SOLR-7547: Short circuit SolrDispatchFilter for static content request. Right 
now it creates a new HttpSolrCall object and tries to process it.

 SDF should short circuit for static content request
 ---

 Key: SOLR-7547
 URL: https://issues.apache.org/jira/browse/SOLR-7547
 Project: Solr
  Issue Type: Bug
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Attachments: SOLR-7547.patch


 As of now, when we request the Admin UI page, I see those requests coming 
 into SDF and creating the HttpSolrCall object. This shouldn't happen and 
 requests for those paths should just short circuit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7508) SolrParams.toMultiMap() does not handle arrays

2015-05-14 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544968#comment-14544968
 ] 

ASF subversion and git services commented on SOLR-7508:
---

Commit 1679494 from [~noble.paul] in branch 'dev/trunk'
[ https://svn.apache.org/r1679494 ]

SOLR-7508: SolrParams.toMultiMap() does not handle arrays

 SolrParams.toMultiMap() does not handle arrays
 --

 Key: SOLR-7508
 URL: https://issues.apache.org/jira/browse/SOLR-7508
 Project: Solr
  Issue Type: Bug
  Components: SolrJ
Affects Versions: 5.0, 5.1
Reporter: Thomas Scheffler
  Labels: easyfix, easytest
 Attachments: SOLRJ-7508.patch


 Following JUnit test to show what I mean:
 {code}
 ModifiableSolrParams params = new ModifiableSolrParams();
 String[] paramValues = new String[] { title:junit, author:john };
 String paramName = fq;
 params.add(paramName, paramValues);
 NamedListObject namedList = params.toNamedList();
 assertEquals(parameter values are not equal, paramValues, 
 namedList.get(paramName));
 MapString, String[] multiMap = SolrParams.toMultiMap(namedList);
 assertEquals(Expected  + paramValues.length +  values, 
 paramValues.length, multiMap.get(paramName).length);
 {code}
 The first {{assertEquals()}} will run fine, while the last one triggers the 
 error. Suddenly the length of the array is 1 and it's value of {{fq}} is 
 like {{[Ljava.lang.String;@6f09c9c0}}. Looking into the code I see that the 
 toMultiMap() method does not even look for arrays.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7508) SolrParams.toMultiMap() does not handle arrays


 [ 
https://issues.apache.org/jira/browse/SOLR-7508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-7508:
-
Priority: Trivial  (was: Major)

 SolrParams.toMultiMap() does not handle arrays
 --

 Key: SOLR-7508
 URL: https://issues.apache.org/jira/browse/SOLR-7508
 Project: Solr
  Issue Type: Bug
  Components: SolrJ
Affects Versions: 5.0, 5.1
Reporter: Thomas Scheffler
Priority: Trivial
  Labels: easyfix, easytest
 Attachments: SOLRJ-7508.patch


 Following JUnit test to show what I mean:
 {code}
 ModifiableSolrParams params = new ModifiableSolrParams();
 String[] paramValues = new String[] { title:junit, author:john };
 String paramName = fq;
 params.add(paramName, paramValues);
 NamedListObject namedList = params.toNamedList();
 assertEquals(parameter values are not equal, paramValues, 
 namedList.get(paramName));
 MapString, String[] multiMap = SolrParams.toMultiMap(namedList);
 assertEquals(Expected  + paramValues.length +  values, 
 paramValues.length, multiMap.get(paramName).length);
 {code}
 The first {{assertEquals()}} will run fine, while the last one triggers the 
 error. Suddenly the length of the array is 1 and it's value of {{fq}} is 
 like {{[Ljava.lang.String;@6f09c9c0}}. Looking into the code I see that the 
 toMultiMap() method does not even look for arrays.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-7544) Refactor and cleanup CollectionsHandler

Noble Paul created SOLR-7544:


 Summary: Refactor and cleanup CollectionsHandler
 Key: SOLR-7544
 URL: https://issues.apache.org/jira/browse/SOLR-7544
 Project: Solr
  Issue Type: Task
Reporter: Noble Paul
Assignee: Noble Paul






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7544) Refactor and cleanup CollectionsHandler


 [ 
https://issues.apache.org/jira/browse/SOLR-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-7544:
-
Attachment: SOLR-7544.patch

 Refactor and cleanup CollectionsHandler
 ---

 Key: SOLR-7544
 URL: https://issues.apache.org/jira/browse/SOLR-7544
 Project: Solr
  Issue Type: Task
Reporter: Noble Paul
Assignee: Noble Paul
 Attachments: SOLR-7544.patch, SOLR-7544.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7543) Create GraphQuery that allows graph traversal as a query operator.

2015-05-14 Thread Kevin Watters (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543982#comment-14543982
]

Kevin Watters commented on SOLR-7543:
-

Hi Yonik, thanks for chiming in! Yup, you can think of this as a multi-step
join. In fact, I use the graph operator with a maxDepth of 1 to implement an
inner join.
I like things to be consistent (it's easier for others to grok that way), we
can rename the fromField and the toField to be from and to. When it comes
to the GraphQueryParser(Plugin), I'm open to whatever the community likes and
whatever is consistent with the other parsers out there.
(I've always been a bit thrown by the !parser_name syntax, which is why I also
have a client side object model so that I programmatically build up an
expression, I serialize that expression over to my custom parser that
deserializes and converts into the appropriate lucene query objects. ). I
suppose I just want to make sure that the v=my_start_query can be any
arbitrary lucene query.
I also still need to work up some richer examples and test cases as part of
this ticket.

Create GraphQuery that allows graph traversal as a query operator.
--

Key: SOLR-7543
URL: https://issues.apache.org/jira/browse/SOLR-7543
Project: Solr
Issue Type: New Feature
Components: search
Reporter: Kevin Watters
Priority: Minor

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


 [ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan updated LUCENE-6482:

Description: 
This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
4.9.1), with many threads seeming deadlocked but RUNNABLE:
{noformat}
elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
   java.lang.Thread.State: RUNNABLE
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
at 
org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
at org.elasticsearch.index.store.Store.access$300(Store.java:76)
at 
org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
at 
org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
at 
org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
at 
org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
at 
org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
at 
org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
at 
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
at 
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:268)
at 
org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}

It didn't really make sense to see RUNNABLE threads in Object.wait(), but this 
seems to be symptomatic of deadlocks in static initialization 
(http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).

I found LUCENE-5573 as an instance of this having come up with Lucene code 
before.

I'm not sure what exactly is going on, but the deadlock in this case seems to 
involve these threads:

{noformat}
elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon prio=5 
os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() [0x7f79daed8000]
   java.lang.Thread.State: RUNNABLE
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at java.lang.Class.newInstance(Class.java:433)
at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:67)
- locked 0x00061fef4968 (a org.apache.lucene.util.NamedSPILoader)
at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:47)
at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:37)
at 
org.apache.lucene.codecs.PostingsFormat.clinit(PostingsFormat.java:44)
at 
org.elasticsearch.index.codec.postingsformat.PostingFormats.clinit(PostingFormats.java:67)
at 
org.elasticsearch.index.codec.CodecModule.configurePostingsFormats(CodecModule.java:126)
at 
org.elasticsearch.index.codec.CodecModule.configure(CodecModule.java:178)
at 
org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
- locked 0x00061fef49e8 (a 
org.elasticsearch.index.codec.CodecModule)
at 
org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:204)
at 
org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:85)
at 
org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:130)
at 
org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:99)
- locked 0x00061fef4c10 (a 
org.elasticsearch.common.inject.InheritingState)
at

[jira] [Updated] (LUCENE-6459) [suggest] Query Interface for suggest API


 [ 
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated LUCENE-6459:
-
Attachment: LUCENE-6459.patch

 [suggest] Query Interface for suggest API
 -

 Key: LUCENE-6459
 URL: https://issues.apache.org/jira/browse/LUCENE-6459
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/search
Affects Versions: 5.1
Reporter: Areek Zillur
Assignee: Areek Zillur
 Fix For: Trunk, 5.x, 5.1

 Attachments: LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, 
 LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch


 This patch factors out common indexing/search API used by the recently 
 introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
 The motivation is to provide a query interface for FST-based fields 
 (*SuggestField* and *ContextSuggestField*) 
 for enabling suggestion scoring and more powerful automaton queries. 
 Previously, only prefix ‘queries’ with index-time weights were supported but 
 we can also support:
 * Prefix queries expressed as regular expressions:  get suggestions that 
 match multiple prefixes
   ** *Example:* _star\[wa\|tr\]_ matches _starwars_ and _startrek_
 * Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions 
 scored by how close they are to the query prefix
 ** *Example:* querying for _seper_ will score _separate_ higher then 
 _superstitious_
 * Context Queries: get suggestions boosted and/or filtered based on their 
 indexed contexts (meta data)
 ** *Boost example:* get typo tolerant suggestions on song names with 
 prefix _like a roling_ boosting songs with 
 genre _rock_ and _indie_
 ** *Filter example:* get suggestion on all file names starting with 
 _finan_ only for _user1_ and _user2_
 h3. Suggest API
 {code}
 SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
 CompletionQuery query = ...
 TopSuggestDocs suggest = searcher.suggest(query, num);
 {code}
 h3. CompletionQuery
 *CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*. 
 A *CompletionQuery* produces a *CompletionWeight*, 
 which allows *CompletionQuery* implementations to pass in an automaton that 
 will be intersected with a FST and allows boosting and 
 meta data extraction from the intersected partial paths. A *CompletionWeight* 
 produces a *CompletionScorer*. A *CompletionScorer* 
 executes a Top N search against the FST with the provided automaton, scoring 
 and filtering all matched paths. 
 h4. PrefixCompletionQuery
 Return documents with values that match the prefix of an analyzed term text 
 Documents are sorted according to their suggest field weight. 
 {code}
 PrefixCompletionQuery(Analyzer analyzer, Term term)
 {code}
 h4. RegexCompletionQuery
 Return documents with values that match the prefix of a regular expression
 Documents are sorted according to their suggest field weight.
 {code}
 RegexCompletionQuery(Term term)
 {code}
 h4. FuzzyCompletionQuery
 Return documents with values that has prefixes within a specified edit 
 distance of an analyzed term text.
 Documents are ‘boosted’ by the number of matching prefix letters of the 
 suggestion with respect to the original term text.
 {code}
 FuzzyCompletionQuery(Analyzer analyzer, Term term)
 {code}
 h5. Scoring
 {{suggestion_weight + (global_maximum_weight * boost)}}
 where {{suggestion_weight}}, {{global_maximum_weight}} and {{boost}} are all 
 integers. 
 {{boost = # of prefix characters matched}}
 h4. ContextQuery
 Return documents that match a {{CompletionQuery}} filtered and/or boosted by 
 provided context(s). 
 {code}
 ContextQuery(CompletionQuery query)
 contextQuery.addContext(CharSequence context, int boost, boolean exact)
 {code}
 *NOTE:* {{ContextQuery}} should be used with {{ContextSuggestField}} to query 
 suggestions boosted and/or filtered by contexts.
 Running {{ContextQuery}} against a {{SuggestField}} will error out.
 h5. Scoring
 {{suggestion_weight + (global_maximum_weight * context_boost)}}
 where {{suggestion_weight}}, {{global_maximum_weight}} and {{context_boost}} 
 are all integers
 When used with {{FuzzyCompletionQuery}},
 {{suggestion_weight + (global_maximum_weight * (context_boost + 
 fuzzy_boost))}}
 h3. Context Suggest Field
 To use {{ContextQuery}}, use {{ContextSuggestField}} instead of 
 {{SuggestField}}. Any {{CompletionQuery}} can be used with 
 {{ContextSuggestField}}, the default behaviour is to return suggestions from 
 *all* contexts. {{Context}} for every completion hit 
 can be accessed through {{SuggestScoreDoc#context}}.
 {code}
 ContextSuggestField(String name, CollectionCharSequence contexts, String 
 value, int weight) 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LUCENE-6459) [suggest] Query Interface for suggest API


 [ 
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated LUCENE-6459:
-
Attachment: (was: LUCENE-6459.patch)

 [suggest] Query Interface for suggest API
 -

 Key: LUCENE-6459
 URL: https://issues.apache.org/jira/browse/LUCENE-6459
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/search
Affects Versions: 5.1
Reporter: Areek Zillur
Assignee: Areek Zillur
 Fix For: Trunk, 5.x, 5.1

 Attachments: LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, 
 LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch


 This patch factors out common indexing/search API used by the recently 
 introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
 The motivation is to provide a query interface for FST-based fields 
 (*SuggestField* and *ContextSuggestField*) 
 for enabling suggestion scoring and more powerful automaton queries. 
 Previously, only prefix ‘queries’ with index-time weights were supported but 
 we can also support:
 * Prefix queries expressed as regular expressions:  get suggestions that 
 match multiple prefixes
   ** *Example:* _star\[wa\|tr\]_ matches _starwars_ and _startrek_
 * Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions 
 scored by how close they are to the query prefix
 ** *Example:* querying for _seper_ will score _separate_ higher then 
 _superstitious_
 * Context Queries: get suggestions boosted and/or filtered based on their 
 indexed contexts (meta data)
 ** *Boost example:* get typo tolerant suggestions on song names with 
 prefix _like a roling_ boosting songs with 
 genre _rock_ and _indie_
 ** *Filter example:* get suggestion on all file names starting with 
 _finan_ only for _user1_ and _user2_
 h3. Suggest API
 {code}
 SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
 CompletionQuery query = ...
 TopSuggestDocs suggest = searcher.suggest(query, num);
 {code}
 h3. CompletionQuery
 *CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*. 
 A *CompletionQuery* produces a *CompletionWeight*, 
 which allows *CompletionQuery* implementations to pass in an automaton that 
 will be intersected with a FST and allows boosting and 
 meta data extraction from the intersected partial paths. A *CompletionWeight* 
 produces a *CompletionScorer*. A *CompletionScorer* 
 executes a Top N search against the FST with the provided automaton, scoring 
 and filtering all matched paths. 
 h4. PrefixCompletionQuery
 Return documents with values that match the prefix of an analyzed term text 
 Documents are sorted according to their suggest field weight. 
 {code}
 PrefixCompletionQuery(Analyzer analyzer, Term term)
 {code}
 h4. RegexCompletionQuery
 Return documents with values that match the prefix of a regular expression
 Documents are sorted according to their suggest field weight.
 {code}
 RegexCompletionQuery(Term term)
 {code}
 h4. FuzzyCompletionQuery
 Return documents with values that has prefixes within a specified edit 
 distance of an analyzed term text.
 Documents are ‘boosted’ by the number of matching prefix letters of the 
 suggestion with respect to the original term text.
 {code}
 FuzzyCompletionQuery(Analyzer analyzer, Term term)
 {code}
 h5. Scoring
 {{suggestion_weight + (global_maximum_weight * boost)}}
 where {{suggestion_weight}}, {{global_maximum_weight}} and {{boost}} are all 
 integers. 
 {{boost = # of prefix characters matched}}
 h4. ContextQuery
 Return documents that match a {{CompletionQuery}} filtered and/or boosted by 
 provided context(s). 
 {code}
 ContextQuery(CompletionQuery query)
 contextQuery.addContext(CharSequence context, int boost, boolean exact)
 {code}
 *NOTE:* {{ContextQuery}} should be used with {{ContextSuggestField}} to query 
 suggestions boosted and/or filtered by contexts.
 Running {{ContextQuery}} against a {{SuggestField}} will error out.
 h5. Scoring
 {{suggestion_weight + (global_maximum_weight * context_boost)}}
 where {{suggestion_weight}}, {{global_maximum_weight}} and {{context_boost}} 
 are all integers
 When used with {{FuzzyCompletionQuery}},
 {{suggestion_weight + (global_maximum_weight * (context_boost + 
 fuzzy_boost))}}
 h3. Context Suggest Field
 To use {{ContextQuery}}, use {{ContextSuggestField}} instead of 
 {{SuggestField}}. Any {{CompletionQuery}} can be used with 
 {{ContextSuggestField}}, the default behaviour is to return suggestions from 
 *all* contexts. {{Context}} for every completion hit 
 can be accessed through {{SuggestScoreDoc#context}}.
 {code}
 ContextSuggestField(String name, CollectionCharSequence contexts, String 
 value, int weight) 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (SOLR-7524) Make Streaming Expressions Java 7 Compatible


[ 
https://issues.apache.org/jira/browse/SOLR-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544050#comment-14544050
 ] 

Joel Bernstein commented on SOLR-7524:
--

Thanks Dennis!

 Make Streaming Expressions Java 7 Compatible
 

 Key: SOLR-7524
 URL: https://issues.apache.org/jira/browse/SOLR-7524
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Affects Versions: Trunk
Reporter: Dennis Gove
Priority: Trivial
 Fix For: 5.2

 Attachments: SOLR-7524.patch


 SOLR-7377 added Streaming Expressions to trunk. It uses, by choice and not 
 necessity, some features of Java 8. This patch is to make minor changes to 
 three files to make Streaming Expressions compatible with Java 7 and therefor 
 able to be included in version 5.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-7524) Make Streaming Expressions Java 7 Compatible


 [ 
https://issues.apache.org/jira/browse/SOLR-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein closed SOLR-7524.

Resolution: Fixed

 Make Streaming Expressions Java 7 Compatible
 

 Key: SOLR-7524
 URL: https://issues.apache.org/jira/browse/SOLR-7524
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Affects Versions: Trunk
Reporter: Dennis Gove
Priority: Trivial
 Fix For: 5.2

 Attachments: SOLR-7524.patch


 SOLR-7377 added Streaming Expressions to trunk. It uses, by choice and not 
 necessity, some features of Java 8. This patch is to make minor changes to 
 three files to make Streaming Expressions compatible with Java 7 and therefor 
 able to be included in version 5.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7377) SOLR Streaming Expressions


 [ 
https://issues.apache.org/jira/browse/SOLR-7377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-7377:
-
Attachment: (was: SOLR-7377.patch)

 SOLR Streaming Expressions
 --

 Key: SOLR-7377
 URL: https://issues.apache.org/jira/browse/SOLR-7377
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Reporter: Dennis Gove
Priority: Minor
 Fix For: Trunk, 5.2

 Attachments: SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, 
 SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, 
 SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch


 It would be beneficial to add an expression-based interface to Streaming API 
 described in SOLR-7082. Right now that API requires streaming requests to 
 come in from clients as serialized bytecode of the streaming classes. The 
 suggestion here is to support string expressions which describe the streaming 
 operations the client wishes to perform. 
 {code:java}
 search(collection1, q=*:*, fl=id,fieldA,fieldB, sort=fieldA asc)
 {code}
 With this syntax in mind, one can now express arbitrarily complex stream 
 queries with a single string.
 {code:java}
 // merge two distinct searches together on common fields
 merge(
   search(collection1, q=id:(0 3 4), fl=id,a_s,a_i,a_f, sort=a_f asc, a_s 
 asc),
   search(collection2, q=id:(1 2), fl=id,a_s,a_i,a_f, sort=a_f asc, a_s 
 asc),
   on=a_f asc, a_s asc)
 // find top 20 unique records of a search
 top(
   n=20,
   unique(
 search(collection1, q=*:*, fl=id,a_s,a_i,a_f, sort=a_f desc),
 over=a_f desc),
   sort=a_f desc)
 {code}
 The syntax would support
 1. Configurable expression names (eg. via solrconfig.xml one can map unique 
 to a class implementing a Unique stream class) This allows users to build 
 their own streams and use as they wish.
 2. Named parameters (of both simple and expression types)
 3. Unnamed, type-matched parameters (to support requiring N streams as 
 arguments to another stream)
 4. Positional parameters
 The main goal here is to make streaming as accessible as possible and define 
 a syntax for running complex queries across large distributed systems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-7546) SimplePostTool (and thus bin/post) cannot index files with unknown extensions

2015-05-14 Thread Erik Hatcher (JIRA)

Erik Hatcher created SOLR-7546:
--

 Summary: SimplePostTool (and thus bin/post) cannot index files 
with unknown extensions
 Key: SOLR-7546
 URL: https://issues.apache.org/jira/browse/SOLR-7546
 Project: Solr
  Issue Type: Bug
  Components: scripts and tools
Affects Versions: 5.1, 5.0
Reporter: Erik Hatcher
Assignee: Erik Hatcher
 Fix For: 5.2


With Solr 5.1.0, indexing a png file does not work:
{code}
$ bin/post -filetypes png -c test2 docs/images/quickstart-query-screen.png 
java -classpath /Users/erikhatcher/solr-5.1.0/dist/solr-core-5.1.0.jar 
-Dauto=yes -Dfiletypes=png -Dc=test2 -Ddata=files 
org.apache.solr.util.SimplePostTool docs/images/quickstart-query-screen.png
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/test2/update...
Entering auto mode. File endings considered are png
SimplePostTool: WARNING: Skipping quickstart-query-screen.png. Unsupported file 
type for auto mode.
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/test2/update...
Time spent: 0:00:00.039
{code}

png is not in SimplePostTool's mimeMap data structure and thus is considered 
unsupported.

Instead, when in auto mode, SimplePostTool should send anything it encounters 
(within the `filetypes` constraint) to Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


 [ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan updated LUCENE-6482:

Description: 
This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
4.9.1), with many threads seeming deadlocked but RUNNABLE:
{noformat}
elasticsearch[blabla-es0][clusterService#updateTask][T#1] #79 daemon prio=5 
os_prio=0 tid=0x7fd16988d000 nid=0x6e01 waiting on condition 
[0x7fd0bc279000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x000614a22508 (a 
org.elasticsearch.common.util.concurrent.BaseFuture$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at 
org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:274)
at 
org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:113)
at 
org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:45)
at 
org.elasticsearch.gateway.local.LocalGatewayAllocator.buildShardStores(LocalGatewayAllocator.java:443)
at 
org.elasticsearch.gateway.local.LocalGatewayAllocator.allocateUnassigned(LocalGatewayAllocator.java:281)
at 
org.elasticsearch.cluster.routing.allocation.allocator.ShardsAllocators.allocateUnassigned(ShardsAllocators.java:74)
at 
org.elasticsearch.cluster.routing.allocation.AllocationService.reroute(AllocationService.java:217)
at 
org.elasticsearch.cluster.routing.allocation.AllocationService.applyStartedShards(AllocationService.java:86)
at 
org.elasticsearch.cluster.action.shard.ShardStateAction$4.execute(ShardStateAction.java:278)
at 
org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:328)
at 
org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:153)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}

It didn't really make sense to see RUNNABLE threads in Object.wait(), but this 
seems to be symptomatic of deadlocks in static initialization 
(http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).

I found LUCENE-5573 as an instance of this having come up with Lucene code 
before.

I'm not sure what exactly is going on, but the deadlock in this case seems to 
involve these threads:

{noformat}
elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon prio=5 
os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() [0x7f79daed8000]
   java.lang.Thread.State: RUNNABLE
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at java.lang.Class.newInstance(Class.java:433)
at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:67)
- locked 0x00061fef4968 (a org.apache.lucene.util.NamedSPILoader)
at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:47)
at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:37)
at 
org.apache.lucene.codecs.PostingsFormat.clinit(PostingsFormat.java:44)
at 
org.elasticsearch.index.codec.postingsformat.PostingFormats.clinit(PostingFormats.java:67)
at 
org.elasticsearch.index.codec.CodecModule.configurePostingsFormats(CodecModule.java:126)
at 
org.elasticsearch.index.codec.CodecModule.configure(CodecModule.java:178)
at 
org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
- locked 0x00061fef49e8 (a 
org.elasticsearch.index.codec.CodecModule)
at 
org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:204)
at 
org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:85)
at 
org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:130)
at 
org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:99)
- locked 0x00061fef4c10 (a

[jira] [Updated] (SOLR-7546) SimplePostTool (and thus bin/post) cannot index files with unknown extensions

2015-05-14 Thread Erik Hatcher (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Erik Hatcher updated SOLR-7546:
---
Attachment: SOLR-7546.patch

patch to allow every file (in auto mode, that passes the `filetypes` check) to
be sent to Solr. Also removed suspicious code in reset() method that set the
file filter to default state, ignoring the filetypes setting.

SimplePostTool (and thus bin/post) cannot index files with unknown extensions
-

Key: SOLR-7546
URL: https://issues.apache.org/jira/browse/SOLR-7546
Project: Solr
Issue Type: Bug
Components: scripts and tools
Affects Versions: 5.0, 5.1
Reporter: Erik Hatcher
Assignee: Erik Hatcher
Fix For: 5.2

Attachments: SOLR-7546.patch

With Solr 5.1.0, indexing a png file does not work:
{code}
$ bin/post -filetypes png -c test2 docs/images/quickstart-query-screen.png
java -classpath /Users/erikhatcher/solr-5.1.0/dist/solr-core-5.1.0.jar
-Dauto=yes -Dfiletypes=png -Dc=test2 -Ddata=files
org.apache.solr.util.SimplePostTool docs/images/quickstart-query-screen.png
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/test2/update...
Entering auto mode. File endings considered are png
SimplePostTool: WARNING: Skipping quickstart-query-screen.png. Unsupported
file type for auto mode.
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/test2/update...
Time spent: 0:00:00.039
{code}
png is not in SimplePostTool's mimeMap data structure and thus is
considered unsupported.
Instead, when in auto mode, SimplePostTool should send anything it encounters
(within the `filetypes` constraint) to Solr.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms

2015-05-14 Thread Nicholas Knize (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-6481:
---
Attachment: LUCENE-6481.patch

Minor patch update that adds geodesic to geodetic projection / reprojection 
methods to GeoUtils.

 Improve GeoPointField type to only visit high precision boundary terms 
 ---

 Key: LUCENE-6481
 URL: https://issues.apache.org/jira/browse/LUCENE-6481
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/index
Reporter: Nicholas Knize
 Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, 
 LUCENE-6481_WIP.patch


 Current GeoPointField [LUCENE-6450 | 
 https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges 
 along the space-filling curve that represent a provided bounding box.  This 
 determines which terms to visit in the terms dictionary and which to skip. 
 This is suboptimal for large bounding boxes as we may end up visiting all 
 terms (which could be quite large). 
 This incremental improvement is to improve GeoPointField to only visit high 
 precision terms in boundary ranges and use the postings list for ranges that 
 are completely within the target bounding box.
 A separate improvement is to switch over to auto-prefix and build an 
 Automaton representing the bounding box.  That can be tracked in a separate 
 issue.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6480) Extend Simple GeoPointField Type to 3d

2015-05-14 Thread Nicholas Knize (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544116#comment-14544116
 ] 

Nicholas Knize commented on LUCENE-6480:


bq. The maximum error you can get, as I stated before, by using a sphere rather 
than a real earth model is a few meters.

[Here's | 
https://gis.stackexchange.com/questions/25494/how-accurate-is-approximating-the-earth-as-a-sphere/]
 a good discussion and analysis on the maximum error for a geo point using the 
sphere over an earth model. tldr: unit sphere can give upwards of 22km 
depending on the points.

The latest patch over at [LUCENE-6481 | 
https://issues.apache.org/jira/browse/LUCENE-6481] adds ecfToLLA and llaToECF 
methods using the WGS84 datum.  This version uses the non-iterative geodetic to 
geodesic conversion so it can give upwards of 5m horizontal error and 3m 
vertical error, but it can be significantly faster than the iterative approach 
depending on the location on the earth.

As a next phase improvement to geo3d it might be interesting to compare 
contrast performance/accuracy using both approaches?

 Extend Simple GeoPointField Type to 3d 
 ---

 Key: LUCENE-6480
 URL: https://issues.apache.org/jira/browse/LUCENE-6480
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/index
Reporter: Nicholas Knize

 [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] proposes a 
 simple GeoPointField type to lucene core. This field uses 64bit encoding of 2 
 dimensional points to construct sorted term representations of GeoPoints 
 (aka: GeoHashing).
 This feature investigates adding support for encoding 3 dimensional 
 GeoPoints, either by extending GeoPointField to a Geo3DPointField or adding 
 an additional 3d constructor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6459) [suggest] Query Interface for suggest API

2015-05-14 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated LUCENE-6459:
-
Attachment: LUCENE-6459.patch

Thanks [~mikemccand] for the feedback :)

{quote}
I thought we accept long (not int) as index-time weight? (But, I
think that really is overkill... maybe they should just floats, like
per-field boosting at index time)
{quote}
IMO, a suggestion weight is just an index-time boost for the
associated entry.

{quote}
bq. one possibility would be to use index-weight + (Int.MAX * boost) instead of 
using MaxWeight of suggestions

Sorry I don't understand the idea here?
{quote}

After a query automaton has been intersected with the FST in {{NRTSuggester}}, 
boosts and/or context is computed/extracted from each of the partial matched 
paths
by the {{CompletionWeight}} before performing a TopN search.
For example, {{FuzzyCompletionWeight}} would count the number of prefix 
characters
a matched input has w.r.t. the analyzed query prefix and set the boost for it 
to the
number of common prefix length.
Calculating a suggestion score of {{weight + (maxWeight * boost)}} makes sure 
that 
entries with a higher boost (longer common prefix w.r.t. query prefix) will 
always be 
scored higher regardless of the index-time weight of suggestion entries. The 
segment-level {{maxWeight}} is stored in {{CompletionPostingsFormat}} 
(CompletionIndex),
and the {{maxWeight}} is computed across all segments at query-time.
Since, the maximum weight for any suggestion entry will be = 
Integer.MAX_VALUE, we
can just replace the {{maxWeight}} for a suggestField with Integer.MAX_VALUE? 
One problem
might be the loss of precision when converting the long score to a float?

{quote}
It seems like we could detect this mis-use, since CompletionTerms seems to know 
whether the field was indexed with contexts or not? I.e, if I accidentally try 
to run a ContextQuery against a field indexed with only SuggestField, it seems 
like I should get an exception saying I screwed up ... (similar to trying to 
run a PhraseQuery on a field that did not index positions)? Maybe add a simple 
test case?
{quote}
I updated the patch to error out when using a {{ContextQuery}} against a 
{{SuggestField}} at rewrite with test.

{quote}
Can we rename {{TopSuggestDocsCollector.num()}} to maybe .getCountToCollect or 
something a bit more verbose?
{quote}

Changed {{TopSuggestDocsCollector.num()}} to {{getCountToCollect()}}

 [suggest] Query Interface for suggest API
 -

 Key: LUCENE-6459
 URL: https://issues.apache.org/jira/browse/LUCENE-6459
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/search
Affects Versions: 5.1
Reporter: Areek Zillur
Assignee: Areek Zillur
 Fix For: Trunk, 5.x, 5.1

 Attachments: LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, 
 LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch


 This patch factors out common indexing/search API used by the recently 
 introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
 The motivation is to provide a query interface for FST-based fields 
 (*SuggestField* and *ContextSuggestField*) for enabling suggestion scoring 
 and more powerful automaton queries. 
 Previously, only prefix ‘queries’ with index-time weights were supported but 
 we can also support:
 * Prefix queries expressed as regular expressions:  get suggestions that 
 match multiple prefixes
   ** Example: _star\[wa\|tr\]_ matches _starwars_ and _startrek_
 * Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions 
 scored by how close they are to the query prefix
 ** Example: querying for _seper_ will score _separate_ higher then 
 _superstitious_
 * Context Queries: get suggestions boosted and/or filtered based on their 
 indexed contexts (meta data)
 ** Example: get typo tolerant suggestions on song names with prefix _like 
 a roling_ boosting songs with genre _rock_ and _indie_
 ** Example: get suggestion on all file names starting with _finan_ only 
 for _user1_ and _user2_
 h3. Suggest API
 {code}
 SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
 CompletionQuery query = ...
 TopSuggestDocs suggest = searcher.suggest(query, num);
 {code}
 h3. CompletionQuery
 *CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*. 
 A *CompletionQuery* produces a *CompletionWeight*, which allows 
 *CompletionQuery* implementations to pass in an automaton that will be 
 intersected with a FST and allows boosting and meta data extraction from the 
 intersected partial paths. A *CompletionWeight* produces a 
 *CompletionScorer*. A *CompletionScorer* executes a Top N search against the 
 FST with the provided automaton, scoring and filtering all

[jira] [Commented] (LUCENE-6480) Extend Simple GeoPointField Type to 3d

2015-05-14 Thread Karl Wright (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544180#comment-14544180
 ] 

Karl Wright commented on LUCENE-6480:
-

bq. Here's a good discussion and analysis on the maximum error for a geo point 
using the sphere over an earth model.

The article discusses error in *distance* computation.  I'm talking not about 
measuring distance halfway around the earth, but whether a given point is 
properly detected to be within a given shape.



 Extend Simple GeoPointField Type to 3d 
 ---

 Key: LUCENE-6480
 URL: https://issues.apache.org/jira/browse/LUCENE-6480
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/index
Reporter: Nicholas Knize

 [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] proposes a 
 simple GeoPointField type to lucene core. This field uses 64bit encoding of 2 
 dimensional points to construct sorted term representations of GeoPoints 
 (aka: GeoHashing).
 This feature investigates adding support for encoding 3 dimensional 
 GeoPoints, either by extending GeoPointField to a Geo3DPointField or adding 
 an additional 3d constructor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7377) SOLR Streaming Expressions


[ 
https://issues.apache.org/jira/browse/SOLR-7377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544038#comment-14544038
 ] 

ASF subversion and git services commented on SOLR-7377:
---

Commit 1679407 from [~joel.bernstein] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679407 ]

SOLR-7377,SOLR-7524:Make Streaming Expressions Java 7 Compatible

 SOLR Streaming Expressions
 --

 Key: SOLR-7377
 URL: https://issues.apache.org/jira/browse/SOLR-7377
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Reporter: Dennis Gove
Priority: Minor
 Fix For: Trunk, 5.2

 Attachments: SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, 
 SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, 
 SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch, SOLR-7377.patch


 It would be beneficial to add an expression-based interface to Streaming API 
 described in SOLR-7082. Right now that API requires streaming requests to 
 come in from clients as serialized bytecode of the streaming classes. The 
 suggestion here is to support string expressions which describe the streaming 
 operations the client wishes to perform. 
 {code:java}
 search(collection1, q=*:*, fl=id,fieldA,fieldB, sort=fieldA asc)
 {code}
 With this syntax in mind, one can now express arbitrarily complex stream 
 queries with a single string.
 {code:java}
 // merge two distinct searches together on common fields
 merge(
   search(collection1, q=id:(0 3 4), fl=id,a_s,a_i,a_f, sort=a_f asc, a_s 
 asc),
   search(collection2, q=id:(1 2), fl=id,a_s,a_i,a_f, sort=a_f asc, a_s 
 asc),
   on=a_f asc, a_s asc)
 // find top 20 unique records of a search
 top(
   n=20,
   unique(
 search(collection1, q=*:*, fl=id,a_s,a_i,a_f, sort=a_f desc),
 over=a_f desc),
   sort=a_f desc)
 {code}
 The syntax would support
 1. Configurable expression names (eg. via solrconfig.xml one can map unique 
 to a class implementing a Unique stream class) This allows users to build 
 their own streams and use as they wish.
 2. Named parameters (of both simple and expression types)
 3. Unnamed, type-matched parameters (to support requiring N streams as 
 arguments to another stream)
 4. Positional parameters
 The main goal here is to make streaming as accessible as possible and define 
 a syntax for running complex queries across large distributed systems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6459) [suggest] Query Interface for suggest API

2015-05-14 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated LUCENE-6459:
-
Description: 
This patch factors out common indexing/search API used by the recently 
introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
The motivation is to provide a query interface for FST-based fields 
(*SuggestField* and *ContextSuggestField*) 
for enabling suggestion scoring and more powerful automaton queries. 

Previously, only prefix ‘queries’ with index-time weights were supported but we 
can also support:

* Prefix queries expressed as regular expressions:  get suggestions that match 
multiple prefixes
  ** *Example:* _star\[wa\|tr\]_ matches _starwars_ and _startrek_
* Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions scored 
by how close they are to the query prefix
** *Example:* querying for _seper_ will score _separate_ higher then 
_superstitious_
* Context Queries: get suggestions boosted and/or filtered based on their 
indexed contexts (meta data)
** *Boost example:* get typo tolerant suggestions on song names with prefix 
_like a roling_ boosting songs with 
genre _rock_ and _indie_
** *Filter example:* get suggestion on all file names starting with _finan_ 
only for _user1_ and _user2_

h3. Suggest API

{code}
SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
CompletionQuery query = ...
TopSuggestDocs suggest = searcher.suggest(query, num);
{code}

h3. CompletionQuery

*CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*. A 
*CompletionQuery* produces a *CompletionWeight*, 
which allows *CompletionQuery* implementations to pass in an automaton that 
will be intersected with a FST and allows boosting and 
meta data extraction from the intersected partial paths. A *CompletionWeight* 
produces a *CompletionScorer*. A *CompletionScorer* 
executes a Top N search against the FST with the provided automaton, scoring 
and filtering all matched paths. 

h4. PrefixCompletionQuery
Return documents with values that match the prefix of an analyzed term text 
Documents are sorted according to their suggest field weight. 
{code}
PrefixCompletionQuery(Analyzer analyzer, Term term)
{code}

h4. RegexCompletionQuery
Return documents with values that match the prefix of a regular expression
Documents are sorted according to their suggest field weight.
{code}
RegexCompletionQuery(Term term)
{code}

h4. FuzzyCompletionQuery
Return documents with values that has prefixes within a specified edit distance 
of an analyzed term text.
Documents are ‘boosted’ by the number of matching prefix letters of the 
suggestion with respect to the original term text.

{code}
FuzzyCompletionQuery(Analyzer analyzer, Term term)
{code}

h5. Scoring
{{suggestion_weight + (global_maximum_weight * boost)}}
where {{suggestion_weight}}, {{global_maximum_weight}} and {{boost}} are all 
integers. 
{{boost = # of prefix characters matched}}

h4. ContextQuery
Return documents that match a {{CompletionQuery}} filtered and/or boosted by 
provided context(s). 
{code}
ContextQuery(CompletionQuery query)
contextQuery.addContext(CharSequence context, int boost, boolean exact)
{code}

*NOTE:* {{ContextQuery}} should be used with {{ContextSuggestField}} to query 
suggestions boosted and/or filtered by contexts.
Running {{ContextQuery}} against a {{SuggestField}} will error out.


h5. Scoring
{{suggestion_weight + (global_maximum_weight * context_boost)}}
where {{suggestion_weight}}, {{global_maximum_weight}} and {{context_boost}} 
are all integers

When used with {{FuzzyCompletionQuery}},
{{suggestion_weight + (global_maximum_weight * (context_boost + fuzzy_boost))}}


h3. Context Suggest Field
To use {{ContextQuery}}, use {{ContextSuggestField}} instead of 
{{SuggestField}}. Any {{CompletionQuery}} can be used with 
{{ContextSuggestField}}, the default behaviour is to return suggestions from 
*all* contexts. {{Context}} for every completion hit 
can be accessed through {{SuggestScoreDoc#context}}.
{code}
ContextSuggestField(String name, CollectionCharSequence contexts, String 
value, int weight) 
{code}

  was:
This patch factors out common indexing/search API used by the recently 
introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
The motivation is to provide a query interface for FST-based fields 
(*SuggestField* and *ContextSuggestField*) for enabling suggestion scoring and 
more powerful automaton queries. 

Previously, only prefix ‘queries’ with index-time weights were supported but we 
can also support:

* Prefix queries expressed as regular expressions:  get suggestions that match 
multiple prefixes
  ** Example: _star\[wa\|tr\]_ matches _starwars_ and _startrek_
* Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions scored 
by how close they are to the query prefix
** Example: querying

[jira] [Commented] (SOLR-7524) Make Streaming Expressions Java 7 Compatible


[ 
https://issues.apache.org/jira/browse/SOLR-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544039#comment-14544039
 ] 

ASF subversion and git services commented on SOLR-7524:
---

Commit 1679407 from [~joel.bernstein] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679407 ]

SOLR-7377,SOLR-7524:Make Streaming Expressions Java 7 Compatible

 Make Streaming Expressions Java 7 Compatible
 

 Key: SOLR-7524
 URL: https://issues.apache.org/jira/browse/SOLR-7524
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Affects Versions: Trunk
Reporter: Dennis Gove
Priority: Trivial
 Fix For: 5.2

 Attachments: SOLR-7524.patch


 SOLR-7377 added Streaming Expressions to trunk. It uses, by choice and not 
 necessity, some features of Java 8. This patch is to make minor changes to 
 three files to make Streaming Expressions compatible with Java 7 and therefor 
 able to be included in version 5.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader

Shikhar Bhushan created LUCENE-6482:
---

 Summary: Class loading deadlock relating to NamedSPILoader
 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Shikhar Bhushan


This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
4.9.1), seeing many threads seeming deadlocked but RUNNABLE:
{noformat}
elasticsearch[blabla-es0][clusterService#updateTask][T#1] #79 daemon prio=5 
os_prio=0 tid=0x7fd16988d000 nid=0x6e01 waiting on condition 
[0x7fd0bc279000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x000614a22508 (a 
org.elasticsearch.common.util.concurrent.BaseFuture$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at 
org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:274)
at 
org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:113)
at 
org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:45)
at 
org.elasticsearch.gateway.local.LocalGatewayAllocator.buildShardStores(LocalGatewayAllocator.java:443)
at 
org.elasticsearch.gateway.local.LocalGatewayAllocator.allocateUnassigned(LocalGatewayAllocator.java:281)
at 
org.elasticsearch.cluster.routing.allocation.allocator.ShardsAllocators.allocateUnassigned(ShardsAllocators.java:74)
at 
org.elasticsearch.cluster.routing.allocation.AllocationService.reroute(AllocationService.java:217)
at 
org.elasticsearch.cluster.routing.allocation.AllocationService.applyStartedShards(AllocationService.java:86)
at 
org.elasticsearch.cluster.action.shard.ShardStateAction$4.execute(ShardStateAction.java:278)
at 
org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:328)
at 
org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:153)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}

It didn't really make sense to see RUNNABLE threads in Object.wait(), but this 
seems to be symptomatic of deadlocks in static initialization 
(http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).

I found LUCENE-5573 as an instance of this having come up with Lucene code 
before.

I'm not sure what exactly is going on, but the deadlock in this case seems to 
involve these threads:

{noformat}
elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon prio=5 
os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() [0x7f79daed8000]
   java.lang.Thread.State: RUNNABLE
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at java.lang.Class.newInstance(Class.java:433)
at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:67)
- locked 0x00061fef4968 (a org.apache.lucene.util.NamedSPILoader)
at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:47)
at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:37)
at 
org.apache.lucene.codecs.PostingsFormat.clinit(PostingsFormat.java:44)
at 
org.elasticsearch.index.codec.postingsformat.PostingFormats.clinit(PostingFormats.java:67)
at 
org.elasticsearch.index.codec.CodecModule.configurePostingsFormats(CodecModule.java:126)
at 
org.elasticsearch.index.codec.CodecModule.configure(CodecModule.java:178)
at 
org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
- locked 0x00061fef49e8 (a 
org.elasticsearch.index.codec.CodecModule)
at 
org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:204)
at 
org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:85)
at 
org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:130)
at

[jira] [Commented] (SOLR-7543) Create GraphQuery that allows graph traversal as a query operator.

2015-05-14 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543949#comment-14543949
]

Yonik Seeley commented on SOLR-7543:

Cool stuff!
Lets just focus on the interface/semantics of the qparser - the rest is
implementation detail and can be further optimized later.

Since this sort of looks like a multi-step join, perhaps use the same
convention of using the primary value (or v) as the start query?

{!graph fromField=f1 toField=f2 ...}my_start_query
Which is equivalent to
{!graph fromField=f1 toField=f2 ... v=my_start_query}

Actually, looking at join again, it uses from and to - should we use those
or stick with fromField and toField?

Create GraphQuery that allows graph traversal as a query operator.
--

Key: SOLR-7543
URL: https://issues.apache.org/jira/browse/SOLR-7543
Project: Solr
Issue Type: New Feature
Components: search
Reporter: Kevin Watters
Priority: Minor

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-MacOSX (64bit/jdk1.8.0) - Build # 2257 - Failure!

2015-05-14 Thread ASF subversion and git services (JIRA)

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/2257/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls

Error Message:
Shard split did not complete. Last recorded state: running 
expected:[completed] but was:[running]

Stack Trace:
org.junit.ComparisonFailure: Shard split did not complete. Last recorded state: 
running expected:[completed] but was:[running]
at 
__randomizedtesting.SeedInfo.seed([6ED3D9EDD933393F:36B7558CDF5991EB]:0)
at org.junit.Assert.assertEquals(Assert.java:125)
at 
org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls(CollectionsAPIAsyncDistributedZkTest.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Commented] (SOLR-7544) Refactor and cleanup CollectionsHandler


[ 
https://issues.apache.org/jira/browse/SOLR-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543955#comment-14543955
 ] 

ASF subversion and git services commented on SOLR-7544:
---

Commit 1679397 from [~noble.paul] in branch 'dev/trunk'
[ https://svn.apache.org/r1679397 ]

SOLR-7544: CollectionsHandler refactored to be more modular

 Refactor and cleanup CollectionsHandler
 ---

 Key: SOLR-7544
 URL: https://issues.apache.org/jira/browse/SOLR-7544
 Project: Solr
  Issue Type: Task
Reporter: Noble Paul
Assignee: Noble Paul
 Attachments: SOLR-7544.patch, SOLR-7544.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms


[ 
https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543767#comment-14543767
 ] 

Michael McCandless commented on LUCENE-6481:


bq. The test had the lat and lon ordering incorrect for both GeoPointFieldType 
and the GeoPointInBBoxQuery.

Oh, woops :)  Thanks for fixing1

 Improve GeoPointField type to only visit high precision boundary terms 
 ---

 Key: LUCENE-6481
 URL: https://issues.apache.org/jira/browse/LUCENE-6481
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/index
Reporter: Nicholas Knize
 Attachments: LUCENE-6481.patch, LUCENE-6481.patch, 
 LUCENE-6481_WIP.patch


 Current GeoPointField [LUCENE-6450 | 
 https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges 
 along the space-filling curve that represent a provided bounding box.  This 
 determines which terms to visit in the terms dictionary and which to skip. 
 This is suboptimal for large bounding boxes as we may end up visiting all 
 terms (which could be quite large). 
 This incremental improvement is to improve GeoPointField to only visit high 
 precision terms in boundary ranges and use the postings list for ranges that 
 are completely within the target bounding box.
 A separate improvement is to switch over to auto-prefix and build an 
 Automaton representing the bounding box.  That can be tracked in a separate 
 issue.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7544) Refactor and cleanup CollectionsHandler


 [ 
https://issues.apache.org/jira/browse/SOLR-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-7544:
-
Attachment: SOLR-7544.patch

 Refactor and cleanup CollectionsHandler
 ---

 Key: SOLR-7544
 URL: https://issues.apache.org/jira/browse/SOLR-7544
 Project: Solr
  Issue Type: Task
Reporter: Noble Paul
Assignee: Noble Paul
 Attachments: SOLR-7544.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-7545) SOLR_HOST parameter not honoured in bin/solr and SolrCLI

2015-05-14 Thread Ishan Chattopadhyaya (JIRA)

Ishan Chattopadhyaya created SOLR-7545:
--

 Summary: SOLR_HOST parameter not honoured in bin/solr and SolrCLI
 Key: SOLR-7545
 URL: https://issues.apache.org/jira/browse/SOLR-7545
 Project: Solr
  Issue Type: Bug
Reporter: Ishan Chattopadhyaya


It seems there are a few places in bin/solr (and maybe SolrCLI class?) where 
localhost is hardcoded, and hence the SOLR_HOST parameter isn't taking effect. 
Will add a patch to fix this shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6464) Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()

2015-05-14 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543762#comment-14543762
 ] 

Michael McCandless commented on LUCENE-6464:


bq. Please, have you also noticed the TODO

Oh I missed that, thanks for pointing it out.

Maybe we should fix the lowest level lookup method to do this all MUST_NOT 
check?  I.e., it would handle taking the clauses of the sub-query and moving 
them up to the main query?  This is also cleaner for users, i.e. they should 
not be expected to know about this all MUST_NOT case.  And then toQuery 
doesn't need to handle it ...

Anyway, I think the {{*:*}} should be better (faster) than doing {{context:*}}, 
because {{*:*}} doesn't visit any postings.  Also, the user may not have 
indexed any contexts ...

 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch, LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7544) Refactor and cleanup CollectionsHandler


[ 
https://issues.apache.org/jira/browse/SOLR-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543974#comment-14543974
 ] 

ASF subversion and git services commented on SOLR-7544:
---

Commit 1679399 from [~noble.paul] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679399 ]

SOLR-7544: CollectionsHandler refactored to be more modular

 Refactor and cleanup CollectionsHandler
 ---

 Key: SOLR-7544
 URL: https://issues.apache.org/jira/browse/SOLR-7544
 Project: Solr
  Issue Type: Task
Reporter: Noble Paul
Assignee: Noble Paul
 Attachments: SOLR-7544.patch, SOLR-7544.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_45) - Build # 12677 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/12677/
Java: 64bit/jdk1.8.0_45 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
There were too many update fails (42  40) - we expect it can happen, but 
shouldn't easily

Stack Trace:
java.lang.AssertionError: There were too many update fails (42  40) - we 
expect it can happen, but shouldn't easily
at 
__randomizedtesting.SeedInfo.seed([7F2DDD01CDD23F66:F779E2DB632E529E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:230)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at

[jira] [Commented] (SOLR-7543) Create GraphQuery that allows graph traversal as a query operator.

2015-05-14 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544003#comment-14544003
]

Yonik Seeley commented on SOLR-7543:

bq. I've always been a bit thrown by the !parser_name syntax

Yeah, but I think doing it like all the other qparsers is best for now... and
then we can add support for JSON along with the rest of them later (there's a
ticket open... just haven't gotten around to it yet).

bq. I suppose I just want to make sure that the v=my_start_query can be any
arbitrary lucene query.

Yep, just use the qparser framework and then the query (or queries) can be of
any type parseable by another qparser (including another graph qparser for
instance).

{code}
baseParser = subQuery(localParams.get(QueryParsing.V), null);
Query q = baseParser.getQuery();
{code}

Create GraphQuery that allows graph traversal as a query operator.
--

Key: SOLR-7543
URL: https://issues.apache.org/jira/browse/SOLR-7543
Project: Solr
Issue Type: New Feature
Components: search
Reporter: Kevin Watters
Priority: Minor

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-MacOSX (64bit/jdk1.8.0) - Build # 2258 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/2258/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.TestLBHttpSolrClient.testReliability

Error Message:
No live SolrServers available to handle this request

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request
at 
__randomizedtesting.SeedInfo.seed([E68E26F0F0B38F58:2746FBB651D55EF1]:0)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:576)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
at 
org.apache.solr.client.solrj.TestLBHttpSolrClient.testReliability(TestLBHttpSolrClient.java:219)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at

[jira] [Commented] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


[ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544336#comment-14544336
 ] 

Uwe Schindler commented on LUCENE-6482:
---

Hi,
the problem here has nothing to do with NamedSPI loader. The problem can be one 
of the following:
- there is one codec in the classpath that uses a wrong initialization like the 
issue you mentioned. The problematic thing is in most cases a 
Codec/PostingsFormat/DocvaluesFormat.forName() in a static initializer. We also 
have this in Lucene, but the order is important here. I don't like this, but 
code is not manageable otherwise. So order of static class initialization is 
important. If one of the codecs hangs in one of such clinit locks, the stack 
trace is easy. All those threads that seem to hang while RUNNING are blocked, 
because they access a class that is currently in initialization phase (as you 
said).
- Elasticsearch has several own codecs, maybe those had a bug as described 
before. 1.3.4 is a older one, maybe update to latest 1.3.9 version. We have 
never seen this with plain Lucene.

In addition, make sure that you use latest JVM versions. several Java 7 
realeases had class loading bugs with deadlocks (e.g. 1.7.0_25). To me it looks 
more like one of those issues, because otherwise other people would have 
reported bugs like this already.

What is you Java version? Any special JVM settings?

Uwe

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Shikhar Bhushan

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:268)
   at 
 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
 {noformat}
 It didn't really make sense to see RUNNABLE threads in Object.wait(), but 
 this seems to be symptomatic of deadlocks in static initialization 
 (http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).
 I found LUCENE-5573 as an instance of this having come up with Lucene code 
 before.
 I'm not sure what exactly is going on, but the deadlock in this case seems to 
 involve these threads:
 {noformat}
 elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon 
 prio=5 os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() 
 [0x7f79daed8000]
java.lang.Thread.State: RUNNABLE
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at 
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at

[jira] [Updated] (LUCENE-6459) [suggest] Query Interface for suggest API


 [ 
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated LUCENE-6459:
-
Attachment: LUCENE-6459.patch

Updated Patch:
 - pre-sort context length in ContextQuery
 - added test to ensure longer context takes precedence for boosting

 [suggest] Query Interface for suggest API
 -

 Key: LUCENE-6459
 URL: https://issues.apache.org/jira/browse/LUCENE-6459
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/search
Affects Versions: 5.1
Reporter: Areek Zillur
Assignee: Areek Zillur
 Fix For: Trunk, 5.x, 5.1

 Attachments: LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, 
 LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, 
 LUCENE-6459.patch


 This patch factors out common indexing/search API used by the recently 
 introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
 The motivation is to provide a query interface for FST-based fields 
 (*SuggestField* and *ContextSuggestField*) 
 for enabling suggestion scoring and more powerful automaton queries. 
 Previously, only prefix ‘queries’ with index-time weights were supported but 
 we can also support:
 * Prefix queries expressed as regular expressions:  get suggestions that 
 match multiple prefixes
   ** *Example:* _star\[wa\|tr\]_ matches _starwars_ and _startrek_
 * Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions 
 scored by how close they are to the query prefix
 ** *Example:* querying for _seper_ will score _separate_ higher then 
 _superstitious_
 * Context Queries: get suggestions boosted and/or filtered based on their 
 indexed contexts (meta data)
 ** *Boost example:* get typo tolerant suggestions on song names with 
 prefix _like a roling_ boosting songs with 
 genre _rock_ and _indie_
 ** *Filter example:* get suggestion on all file names starting with 
 _finan_ only for _user1_ and _user2_
 h3. Suggest API
 {code}
 SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
 CompletionQuery query = ...
 TopSuggestDocs suggest = searcher.suggest(query, num);
 {code}
 h3. CompletionQuery
 *CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*. 
 A *CompletionQuery* produces a *CompletionWeight*, 
 which allows *CompletionQuery* implementations to pass in an automaton that 
 will be intersected with a FST and allows boosting and 
 meta data extraction from the intersected partial paths. A *CompletionWeight* 
 produces a *CompletionScorer*. A *CompletionScorer* 
 executes a Top N search against the FST with the provided automaton, scoring 
 and filtering all matched paths. 
 h4. PrefixCompletionQuery
 Return documents with values that match the prefix of an analyzed term text 
 Documents are sorted according to their suggest field weight. 
 {code}
 PrefixCompletionQuery(Analyzer analyzer, Term term)
 {code}
 h4. RegexCompletionQuery
 Return documents with values that match the prefix of a regular expression
 Documents are sorted according to their suggest field weight.
 {code}
 RegexCompletionQuery(Term term)
 {code}
 h4. FuzzyCompletionQuery
 Return documents with values that has prefixes within a specified edit 
 distance of an analyzed term text.
 Documents are ‘boosted’ by the number of matching prefix letters of the 
 suggestion with respect to the original term text.
 {code}
 FuzzyCompletionQuery(Analyzer analyzer, Term term)
 {code}
 h5. Scoring
 {{suggestion_weight + (global_maximum_weight * boost)}}
 where {{suggestion_weight}}, {{global_maximum_weight}} and {{boost}} are all 
 integers. 
 {{boost = # of prefix characters matched}}
 h4. ContextQuery
 Return documents that match a {{CompletionQuery}} filtered and/or boosted by 
 provided context(s). 
 {code}
 ContextQuery(CompletionQuery query)
 contextQuery.addContext(CharSequence context, int boost, boolean exact)
 {code}
 *NOTE:* {{ContextQuery}} should be used with {{ContextSuggestField}} to query 
 suggestions boosted and/or filtered by contexts.
 Running {{ContextQuery}} against a {{SuggestField}} will error out.
 h5. Scoring
 {{suggestion_weight + (global_maximum_weight * context_boost)}}
 where {{suggestion_weight}}, {{global_maximum_weight}} and {{context_boost}} 
 are all integers
 When used with {{FuzzyCompletionQuery}},
 {{suggestion_weight + (global_maximum_weight * (context_boost + 
 fuzzy_boost))}}
 h3. Context Suggest Field
 To use {{ContextQuery}}, use {{ContextSuggestField}} instead of 
 {{SuggestField}}. Any {{CompletionQuery}} can be used with 
 {{ContextSuggestField}}, the default behaviour is to return suggestions from 
 *all* contexts. {{Context}} for every completion hit 
 can be accessed through {{SuggestScoreDoc#context}}.
 {code}
 ContextSuggestField(String name,

[jira] [Comment Edited] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


[ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544336#comment-14544336
 ] 

Uwe Schindler edited comment on LUCENE-6482 at 5/14/15 8:49 PM:


Hi,
the problem here has nothing to do with NamedSPILoader, just how Codecs are 
initialized. The problem can be one of the following:
- there is one codec in the classpath that uses a wrong initialization like the 
issue you mentioned (LUCENE-5573). The problematic thing is in most cases a 
Codec/PostingsFormat/DocValuesFormat.forName() in a static initializer. We also 
have this in Lucene, but the order is important here. I don't like this, but 
code is not manageable otherwise. So order of static class initialization is 
important. If one of the codecs hangs in one of such clinit locks, the stack 
trace is easy. All those threads that seem to hang while RUNNING are blocked, 
because they access a class that is currently in initialization phase (as you 
said).
- Elasticsearch has several own codecs, maybe those had a bug as described 
before. 1.3.4 is a older one, maybe update to latest 1.3.9 version. We have 
never seen this with plain Lucene.

In addition, make sure that you use latest JVM versions. several Java 7 
realeases had class loading bugs with deadlocks (e.g. 1.7.0_25). To me it looks 
more like one of those issues, because otherwise other people would have 
reported bugs like this already.

What is your Java version? Any special JVM settings?

Uwe


was (Author: thetaphi):
Hi,
the problem here has nothing to do with NamedSPI loader. The problem can be one 
of the following:
- there is one codec in the classpath that uses a wrong initialization like the 
issue you mentioned. The problematic thing is in most cases a 
Codec/PostingsFormat/DocvaluesFormat.forName() in a static initializer. We also 
have this in Lucene, but the order is important here. I don't like this, but 
code is not manageable otherwise. So order of static class initialization is 
important. If one of the codecs hangs in one of such clinit locks, the stack 
trace is easy. All those threads that seem to hang while RUNNING are blocked, 
because they access a class that is currently in initialization phase (as you 
said).
- Elasticsearch has several own codecs, maybe those had a bug as described 
before. 1.3.4 is a older one, maybe update to latest 1.3.9 version. We have 
never seen this with plain Lucene.

In addition, make sure that you use latest JVM versions. several Java 7 
realeases had class loading bugs with deadlocks (e.g. 1.7.0_25). To me it looks 
more like one of those issues, because otherwise other people would have 
reported bugs like this already.

What is your Java version? Any special JVM settings?

Uwe

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.9.1
Reporter: Shikhar Bhushan
Assignee: Uwe Schindler

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at

[jira] [Commented] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


[ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544349#comment-14544349
 ] 

Shikhar Bhushan commented on LUCENE-6482:
-

[~thetaphi] This was seen on JDK8u5, but I think this has also happened on 
JDK8u25 (not certain though...). 

The issue is not deterministic and comes up during cluster bounces sometimes, 
so it's hard to say whether an upgrade fixes it.

You're probably right that this has nothing to do with NamedSPILoader but the 
classes being loaded. Is that possible to conclude from the thread dump whether 
it is an ES or Lucene Codec / PostingsFormat is involved?

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.9.1
Reporter: Shikhar Bhushan
Assignee: Uwe Schindler

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:268)
   at 
 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
 {noformat}
 It didn't really make sense to see RUNNABLE threads in Object.wait(), but 
 this seems to be symptomatic of deadlocks in static initialization 
 (http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).
 I found LUCENE-5573 as an instance of this having come up with Lucene code 
 before.
 I'm not sure what exactly is going on, but the deadlock in this case seems to 
 involve these threads:
 {noformat}
 elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon 
 prio=5 os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() 
 [0x7f79daed8000]
java.lang.Thread.State: RUNNABLE
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at 
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at 
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
   at java.lang.Class.newInstance(Class.java:433)
   at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:67)
   - locked 0x00061fef4968 (a org.apache.lucene.util.NamedSPILoader)
   at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:47)
   at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:37)
   at 
 org.apache.lucene.codecs.PostingsFormat.clinit(PostingsFormat.java:44)
   at 
 org.elasticsearch.index.codec.postingsformat.PostingFormats.clinit(PostingFormats.java:67)

[jira] [Comment Edited] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


[ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544349#comment-14544349
 ] 

Shikhar Bhushan edited comment on LUCENE-6482 at 5/14/15 8:57 PM:
--

[~thetaphi] This was seen on JDK8u5, but I think this has also happened on 
JDK8u25 (not certain though...). 

The issue is not deterministic and comes up during cluster bounces sometimes, 
so it's hard to say whether an ES upgrade fixes it.

You're probably right that this has nothing to do with NamedSPILoader but the 
classes being loaded. Is it possible to conclude from the thread dump whether 
an ES or Lucene Codec/PostingsFormat/etc is involved?


was (Author: shikhar):
[~thetaphi] This was seen on JDK8u5, but I think this has also happened on 
JDK8u25 (not certain though...). 

The issue is not deterministic and comes up during cluster bounces sometimes, 
so it's hard to say whether an upgrade fixes it.

You're probably right that this has nothing to do with NamedSPILoader but the 
classes being loaded. Is that possible to conclude from the thread dump whether 
it is an ES or Lucene Codec / PostingsFormat is involved?

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.9.1
Reporter: Shikhar Bhushan
Assignee: Uwe Schindler

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:268)
   at 
 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
 {noformat}
 It didn't really make sense to see RUNNABLE threads in Object.wait(), but 
 this seems to be symptomatic of deadlocks in static initialization 
 (http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).
 I found LUCENE-5573 as an instance of this having come up with Lucene code 
 before.
 I'm not sure what exactly is going on, but the deadlock in this case seems to 
 involve these threads:
 {noformat}
 elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon 
 prio=5 os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() 
 [0x7f79daed8000]
java.lang.Thread.State: RUNNABLE
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at 
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at 
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
   at

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2304 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2304/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Error from server at http://127.0.0.1:63790//collection1: 
java.lang.NullPointerException  at 
org.apache.solr.search.grouping.distributed.responseprocessor.TopGroupsShardResponseProcessor.process(TopGroupsShardResponseProcessor.java:102)
  at 
org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:744)
  at 
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:727)
  at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:388)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:2047)  at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:642)  at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:442)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:175)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:168)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:105)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
  at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) 
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  
at org.eclipse.jetty.server.Server.handle(Server.java:497)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) 
 at java.lang.Thread.run(Thread.java:745) 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:63790//collection1: 
java.lang.NullPointerException
at 
org.apache.solr.search.grouping.distributed.responseprocessor.TopGroupsShardResponseProcessor.process(TopGroupsShardResponseProcessor.java:102)
at 
org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:744)
at 
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:727)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:388)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2047)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:642)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:442)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:175)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:168)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:105)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at

[jira] [Comment Edited] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


[ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544336#comment-14544336
 ] 

Uwe Schindler edited comment on LUCENE-6482 at 5/14/15 8:47 PM:


Hi,
the problem here has nothing to do with NamedSPI loader. The problem can be one 
of the following:
- there is one codec in the classpath that uses a wrong initialization like the 
issue you mentioned. The problematic thing is in most cases a 
Codec/PostingsFormat/DocvaluesFormat.forName() in a static initializer. We also 
have this in Lucene, but the order is important here. I don't like this, but 
code is not manageable otherwise. So order of static class initialization is 
important. If one of the codecs hangs in one of such clinit locks, the stack 
trace is easy. All those threads that seem to hang while RUNNING are blocked, 
because they access a class that is currently in initialization phase (as you 
said).
- Elasticsearch has several own codecs, maybe those had a bug as described 
before. 1.3.4 is a older one, maybe update to latest 1.3.9 version. We have 
never seen this with plain Lucene.

In addition, make sure that you use latest JVM versions. several Java 7 
realeases had class loading bugs with deadlocks (e.g. 1.7.0_25). To me it looks 
more like one of those issues, because otherwise other people would have 
reported bugs like this already.

What is your Java version? Any special JVM settings?

Uwe


was (Author: thetaphi):
Hi,
the problem here has nothing to do with NamedSPI loader. The problem can be one 
of the following:
- there is one codec in the classpath that uses a wrong initialization like the 
issue you mentioned. The problematic thing is in most cases a 
Codec/PostingsFormat/DocvaluesFormat.forName() in a static initializer. We also 
have this in Lucene, but the order is important here. I don't like this, but 
code is not manageable otherwise. So order of static class initialization is 
important. If one of the codecs hangs in one of such clinit locks, the stack 
trace is easy. All those threads that seem to hang while RUNNING are blocked, 
because they access a class that is currently in initialization phase (as you 
said).
- Elasticsearch has several own codecs, maybe those had a bug as described 
before. 1.3.4 is a older one, maybe update to latest 1.3.9 version. We have 
never seen this with plain Lucene.

In addition, make sure that you use latest JVM versions. several Java 7 
realeases had class loading bugs with deadlocks (e.g. 1.7.0_25). To me it looks 
more like one of those issues, because otherwise other people would have 
reported bugs like this already.

What is you Java version? Any special JVM settings?

Uwe

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.9.1
Reporter: Shikhar Bhushan

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
   at

[jira] [Updated] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


 [ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6482:
--
Affects Version/s: 4.9.1
 Assignee: Uwe Schindler

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.9.1
Reporter: Shikhar Bhushan
Assignee: Uwe Schindler

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:268)
   at 
 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
 {noformat}
 It didn't really make sense to see RUNNABLE threads in Object.wait(), but 
 this seems to be symptomatic of deadlocks in static initialization 
 (http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).
 I found LUCENE-5573 as an instance of this having come up with Lucene code 
 before.
 I'm not sure what exactly is going on, but the deadlock in this case seems to 
 involve these threads:
 {noformat}
 elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon 
 prio=5 os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() 
 [0x7f79daed8000]
java.lang.Thread.State: RUNNABLE
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at 
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at 
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
   at java.lang.Class.newInstance(Class.java:433)
   at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:67)
   - locked 0x00061fef4968 (a org.apache.lucene.util.NamedSPILoader)
   at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:47)
   at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:37)
   at 
 org.apache.lucene.codecs.PostingsFormat.clinit(PostingsFormat.java:44)
   at 
 org.elasticsearch.index.codec.postingsformat.PostingFormats.clinit(PostingFormats.java:67)
   at 
 org.elasticsearch.index.codec.CodecModule.configurePostingsFormats(CodecModule.java:126)
   at 
 org.elasticsearch.index.codec.CodecModule.configure(CodecModule.java:178)
   at 
 org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
   - locked 0x00061fef49e8 (a 
 org.elasticsearch.index.codec.CodecModule)
   at

[jira] [Updated] (SOLR-7547) SDF should short circuit for static content request


 [ 
https://issues.apache.org/jira/browse/SOLR-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-7547:
---
Attachment: SOLR-7547.patch

 SDF should short circuit for static content request
 ---

 Key: SOLR-7547
 URL: https://issues.apache.org/jira/browse/SOLR-7547
 Project: Solr
  Issue Type: Bug
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Attachments: SOLR-7547.patch


 As of now, when we request the Admin UI page, I see those requests coming 
 into SDF and creating the HttpSolrCall object. This shouldn't happen and 
 requests for those paths should just short circuit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-7547) SDF should short circuit for static content request

Anshum Gupta created SOLR-7547:
--

 Summary: SDF should short circuit for static content request
 Key: SOLR-7547
 URL: https://issues.apache.org/jira/browse/SOLR-7547
 Project: Solr
  Issue Type: Bug
Reporter: Anshum Gupta
Assignee: Anshum Gupta


As of now, when we request the Admin UI page, I see those requests coming into 
SDF and creating the HttpSolrCall object. This shouldn't happen and requests 
for those paths should just short circuit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6464) Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()


 [ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arcadius Ahouansou updated LUCENE-6464:
---
Attachment: LUCENE-6464.patch

Updated patch.

- Renamed addToQuery()-addContextToQuery()
- Added reference to addContextToQuery() in the javadoc of the lookup method 
that takes a contextQuery as parameter
- Removed the usage of MatchAllDocsQuery() and moved the all MUST_NOT  logic 
from toQuery() into the main lookup() method

 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch, LUCENE-6464.patch, LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


[ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544392#comment-14544392
 ] 

Uwe Schindler commented on LUCENE-6482:
---

bq. The issue is not deterministic and comes up during cluster bounces 
sometimes, so it's hard to say whether an ES upgrade fixes it.

I have no idea what you mean with cluster bounce, but this should only happen 
during startup of Elasticsearch nodes: From the last 2 stack traces, you see 
that there are 2 things happening in parallel: Loading of Codec.class (because 
an Index was opened), but in parallel, Elasticsearch seems to initialize the 
PostingsFormats.class in the class CodecModule (Elasticsearch). In my opinion, 
this should not happen in parallel, but a fix would maybe that CodecModule 
should also call Codecs.forName() so those classes are initialized sequentially 
at one single place. The problem with Codec.class and PostingsFormat.class 
clinit running in parallel in different threads may have similar effects like 
you have seen in the blog post (Codecs depend on PostingsFormat and some 
PostingsFormats depend on the Codec class, which then hangs if PostingsFormats 
and Codecs are initialized from 2 different threads at same time, waiting for 
each other). But we have no chance to prevent this (unfortunately).

I cannot say for sure, but something seems to be fishy while initializing 
Elasticsearch, because there is too much happening at the same time. In my 
opinion, Codecs and Postingsformats and Docvalues classes should be initialized 
sequentially, but I have no idea how to enforce this.

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.9.1
Reporter: Shikhar Bhushan
Assignee: Uwe Schindler

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:268)
   at 
 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
 {noformat}
 It didn't really make sense to see RUNNABLE threads in Object.wait(), but 
 this seems to be symptomatic of deadlocks in static initialization 
 (http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).
 I found LUCENE-5573 as an instance of this having come up with Lucene code 
 before.
 I'm not sure what exactly is going on, but the deadlock in this case seems to 
 involve these threads:
 {noformat}
 elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon 
 prio=5 os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait()

[jira] [Updated] (LUCENE-6464) Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()


 [ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arcadius Ahouansou updated LUCENE-6464:
---
Attachment: LUCENE-6464.patch

 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch, LUCENE-6464.patch, LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6464) Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()


 [ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arcadius Ahouansou updated LUCENE-6464:
---
Attachment: (was: LUCENE-6464.patch)

 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6464) Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()


 [ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arcadius Ahouansou updated LUCENE-6464:
---
Attachment: (was: LUCENE-6464.patch)

 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch, LUCENE-6464.patch, LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6464) Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()


 [ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arcadius Ahouansou updated LUCENE-6464:
---
Attachment: (was: LUCENE-6464.patch)

 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (LUCENE-6482) Class loading deadlock relating to NamedSPILoader


 [ 
https://issues.apache.org/jira/browse/LUCENE-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan closed LUCENE-6482.
---
Resolution: Not A Problem

Thanks [~thetaphi], makes sense and it does not seem like a Lucene issue, so 
I'll close this.

It might have been due to using a custom Elasticsearch discovery plugin which 
is purely asynchronous that those 2 bits ended up happening in parallel, and 
caused the deadlock.

 Class loading deadlock relating to NamedSPILoader
 -

 Key: LUCENE-6482
 URL: https://issues.apache.org/jira/browse/LUCENE-6482
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.9.1
Reporter: Shikhar Bhushan
Assignee: Uwe Schindler

 This issue came up for us several times with Elasticsearch 1.3.4 (Lucene 
 4.9.1), with many threads seeming deadlocked but RUNNABLE:
 {noformat}
 elasticsearch[search77-es2][generic][T#43] #160 daemon prio=5 os_prio=0 
 tid=0x7f79180c5800 nid=0x3d1f in Object.wait() [0x7f79d9289000]
java.lang.Thread.State: RUNNABLE
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:457)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:912)
   at 
 org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:758)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:453)
   at 
 org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:98)
   at org.elasticsearch.index.store.Store.readSegmentsInfo(Store.java:126)
   at org.elasticsearch.index.store.Store.access$300(Store.java:76)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.buildMetadata(Store.java:465)
   at 
 org.elasticsearch.index.store.Store$MetadataSnapshot.init(Store.java:456)
   at 
 org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:281)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:186)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:140)
   at 
 org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:61)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:277)
   at 
 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:268)
   at 
 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
 {noformat}
 It didn't really make sense to see RUNNABLE threads in Object.wait(), but 
 this seems to be symptomatic of deadlocks in static initialization 
 (http://ternarysearch.blogspot.ru/2013/07/static-initialization-deadlock.html).
 I found LUCENE-5573 as an instance of this having come up with Lucene code 
 before.
 I'm not sure what exactly is going on, but the deadlock in this case seems to 
 involve these threads:
 {noformat}
 elasticsearch[search77-es2][clusterService#updateTask][T#1] #79 daemon 
 prio=5 os_prio=0 tid=0x7f7b155ff800 nid=0xd49 in Object.wait() 
 [0x7f79daed8000]
java.lang.Thread.State: RUNNABLE
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at 
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at 
 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
   at java.lang.Class.newInstance(Class.java:433)
   at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:67)
   - locked 0x00061fef4968 (a org.apache.lucene.util.NamedSPILoader)
   at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:47)
   at org.apache.lucene.util.NamedSPILoader.init(NamedSPILoader.java:37)
   at 
 org.apache.lucene.codecs.PostingsFormat.clinit(PostingsFormat.java:44)
   at 
 org.elasticsearch.index.codec.postingsformat.PostingFormats.clinit(PostingFormats.java:67)
   at 
 org.elasticsearch.index.codec.CodecModule.configurePostingsFormats(CodecModule.java:126)
   at 
 org.elasticsearch.index.codec.CodecModule.configure(CodecModule.java:178)
   at

[jira] [Commented] (LUCENE-6480) Extend Simple GeoPointField Type to 3d

2015-05-14 Thread Karl Wright (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544686#comment-14544686
]

Karl Wright commented on LUCENE-6480:
-

I looked at the LUCENE-6481 patch and the mapping methods. Thanks for
including them.

It is not clear how these could be used to modify geo3D, since (as I mentioned
before) the math intersects planes with the unit sphere algebraically. The
logic which determines if a point is within a shape is largely robust against
differences in altitude, with some exceptions, so as I stated before shape
membership is not very sensitive to oblate spheroid vs. true sphere. There are
some exceptions to that, however -- for example, the side edges of paths do not
go through the origin, and therefore altitude will matter somewhat. Also,
membership in a GeoCircle is computed as being on the appropriate side of a
plane into which the circle is transcribed, which means that it is more
sensitive to altitude than any other shape geo3d supports. Polygons are
completely insensitive because their edges all go through the origin. Bounding
boxes are also sensitive, especially when top or bottom latitude is near a
pole, for very similar reasons to GeoCircles.

Public goe3D distance computations are done along GeoPaths and from the centers
of GeoCircles. While GeoCircles can have radii of up to 180 degrees, typical
usage is much smaller than that. The same is true for GeoPath segments, where
a kilometer or two would be typical. There is no distance computation for
large area-covering objects like polygons.

I doubt very much that it would be possible to *algebraically* change geo3D
completely to handle an oblate spheroid rather than a sphere. The places where
it breaks mathematically are as follows:

(1) Finding lat/lon bounds of a shape, which requires computing the maximum and
minimum latitude and longitude of the intersection of a plane and the unit
sphere
(2) Determining relationships between shapes, which requires intersecting
planes with the unit sphere
(3) Geocircles, because a plane is only an approximation for membership for an
oblate spheroid; you really require a warped plane, which is way beyond simple
mathematics to compute

Part-way solutions can be found, however, for the distance issue, since
distance computation is pretty much completely independent of everything else.
This will, of course, lead to nonsense answers where a point in a circle has
a greater distance to the center in some cases than the circle's radius, but
that can't be helped.

Extend Simple GeoPointField Type to 3d
---

Key: LUCENE-6480
URL: https://issues.apache.org/jira/browse/LUCENE-6480
Project: Lucene - Core
Issue Type: New Feature
Components: core/index
Reporter: Nicholas Knize

[LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] proposes a
simple GeoPointField type to lucene core. This field uses 64bit encoding of 2
dimensional points to construct sorted term representations of GeoPoints
(aka: GeoHashing).
This feature investigates adding support for encoding 3 dimensional
GeoPoints, either by extending GeoPointField to a Geo3DPointField or adding
an additional 3d constructor.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7548) CloudSolrStream Limits Max Results to rows Param


[ 
https://issues.apache.org/jira/browse/SOLR-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544785#comment-14544785
 ] 

Joel Bernstein commented on SOLR-7548:
--

The Streaming API really was designed to be paired with the /export handler, 
which always returns all rows. 

We should consider making the /export handler the default handler.

 CloudSolrStream Limits Max Results to rows Param
 

 Key: SOLR-7548
 URL: https://issues.apache.org/jira/browse/SOLR-7548
 Project: Solr
  Issue Type: Bug
Affects Versions: Trunk
Reporter: Dennis Gove
Priority: Minor
  Labels: Streaming
 Fix For: Trunk


 The CloudSolrStream stream class accepts a set of params to be passed to the 
 standard query handler. If the provided params doesn't include rows=N then 
 maximum # of records returned by this stream is the configured default rows 
 value (generally 10, but perhaps more). 
 As CloudSolrStream would generally be the first part of a larger set of 
 stream expressions it seems counterintuitive to limit the first set by this 
 value.
 This ticket is to address this so that either we set pass a param of rows=MAX 
 where MAX is the max value we can pass (max int or max long I suppose) or 
 make it so that default value is ignored when in a streaming context.
 Example:
 Imagine we have a collection people with 90 documents in it
 The following query would return at most 10 documents (assuming 10 is the 
 default)
 {code}
 search(people,q=*:*,fl=id,name_s,gender_s,nick_s,sort=name_s desc)
 {code}
 The following query would return all documents
 {code}
 search(people,q=*:*,fl=id,name_s,gender_s,nick_s,sort=name_s 
 desc,rows=100)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-7548) CloudSolrStream Limits Max Results to rows Param

Dennis Gove created SOLR-7548:
-

 Summary: CloudSolrStream Limits Max Results to rows Param
 Key: SOLR-7548
 URL: https://issues.apache.org/jira/browse/SOLR-7548
 Project: Solr
  Issue Type: Bug
Affects Versions: Trunk
Reporter: Dennis Gove
Priority: Minor
 Fix For: Trunk


The CloudSolrStream stream class accepts a set of params to be passed to the 
standard query handler. If the provided params doesn't include rows=N then 
maximum # of records returned by this stream is the configured default rows 
value (generally 10, but perhaps more). 

As CloudSolrStream would generally be the first part of a larger set of stream 
expressions it seems counterintuitive to limit the first set by this value.

This ticket is to address this so that either we set pass a param of rows=MAX 
where MAX is the max value we can pass (max int or max long I suppose) or make 
it so that default value is ignored when in a streaming context.

Example:
Imagine we have a collection people with 90 documents in it

The following query would return at most 10 documents (assuming 10 is the 
default)
{code}
search(people,q=*:*,fl=id,name_s,gender_s,nick_s,sort=name_s desc)
{code}

The following query would return all documents
{code}
search(people,q=*:*,fl=id,name_s,gender_s,nick_s,sort=name_s desc,rows=100)
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2305 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2305/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.test

Error Message:
There should be 3 documents because there should be two id=1 docs due to 
overwrite=false expected:3 but was:1

Stack Trace:
java.lang.AssertionError: There should be 3 documents because there should be 
two id=1 docs due to overwrite=false expected:3 but was:1
at 
__randomizedtesting.SeedInfo.seed([DCBBFDFCCA2F667F:54EFC22664D30B87]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.testOverwriteOption(CloudSolrClientTest.java:171)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.test(CloudSolrClientTest.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at

[jira] [Updated] (LUCENE-6459) [suggest] Query Interface for suggest API


 [ 
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated LUCENE-6459:
-
Description: 
This patch factors out common indexing/search API used by the recently 
introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
The motivation is to provide a query interface for FST-based fields 
(*SuggestField* and *ContextSuggestField*) 
to enable suggestion scoring and more powerful automaton queries. 

Previously, only prefix ‘queries’ with index-time weights were supported but we 
can also support:

* Prefix queries expressed as regular expressions:  get suggestions that match 
multiple prefixes
  ** *Example:* _star\[wa\|tr\]_ matches _starwars_ and _startrek_
* Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions scored 
by how close they are to the query prefix
** *Example:* querying for _seper_ will score _separate_ higher then 
_superstitious_
* Context Queries: get suggestions boosted and/or filtered based on their 
indexed contexts (meta data)
** *Boost example:* get typo tolerant suggestions on song names with prefix 
_like a roling_ boosting songs with 
genre _rock_ and _indie_
** *Filter example:* get suggestion on all file names starting with _finan_ 
only for _user1_ and _user2_

h3. Suggest API

{code}
SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
CompletionQuery query = ...
TopSuggestDocs suggest = searcher.suggest(query, num);
{code}

h3. CompletionQuery

*CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*. A 
*CompletionQuery* produces a *CompletionWeight*, 
which allows *CompletionQuery* implementations to pass in an automaton that 
will be intersected with a FST and allows boosting and 
meta data extraction from the intersected partial paths. A *CompletionWeight* 
produces a *CompletionScorer*. A *CompletionScorer* 
executes a Top N search against the FST with the provided automaton, scoring 
and filtering all matched paths. 

h4. PrefixCompletionQuery
Return documents with values that match the prefix of an analyzed term text 
Documents are sorted according to their suggest field weight. 
{code}
PrefixCompletionQuery(Analyzer analyzer, Term term)
{code}

h4. RegexCompletionQuery
Return documents with values that match the prefix of a regular expression
Documents are sorted according to their suggest field weight.
{code}
RegexCompletionQuery(Term term)
{code}

h4. FuzzyCompletionQuery
Return documents with values that has prefixes within a specified edit distance 
of an analyzed term text.
Documents are ‘boosted’ by the number of matching prefix letters of the 
suggestion with respect to the original term text.

{code}
FuzzyCompletionQuery(Analyzer analyzer, Term term)
{code}

h5. Scoring
{{suggestion_weight + (global_maximum_weight * boost)}}
where {{suggestion_weight}}, {{global_maximum_weight}} and {{boost}} are all 
integers. 
{{boost = # of prefix characters matched}}

h4. ContextQuery
Return documents that match a {{CompletionQuery}} filtered and/or boosted by 
provided context(s). 
{code}
ContextQuery(CompletionQuery query)
contextQuery.addContext(CharSequence context, int boost, boolean exact)
{code}

*NOTE:* {{ContextQuery}} should be used with {{ContextSuggestField}} to query 
suggestions boosted and/or filtered by contexts.
Running {{ContextQuery}} against a {{SuggestField}} will error out.


h5. Scoring
{{suggestion_weight + (global_maximum_weight * context_boost)}}
where {{suggestion_weight}}, {{global_maximum_weight}} and {{context_boost}} 
are all integers

When used with {{FuzzyCompletionQuery}},
{{suggestion_weight + (global_maximum_weight * (context_boost + fuzzy_boost))}}


h3. Context Suggest Field
To use {{ContextQuery}}, use {{ContextSuggestField}} instead of 
{{SuggestField}}. Any {{CompletionQuery}} can be used with 
{{ContextSuggestField}}, the default behaviour is to return suggestions from 
*all* contexts. {{Context}} for every completion hit 
can be accessed through {{SuggestScoreDoc#context}}.
{code}
ContextSuggestField(String name, CollectionCharSequence contexts, String 
value, int weight) 
{code}

  was:
This patch factors out common indexing/search API used by the recently 
introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339]. 
The motivation is to provide a query interface for FST-based fields 
(*SuggestField* and *ContextSuggestField*) 
for enabling suggestion scoring and more powerful automaton queries. 

Previously, only prefix ‘queries’ with index-time weights were supported but we 
can also support:

* Prefix queries expressed as regular expressions:  get suggestions that match 
multiple prefixes
  ** *Example:* _star\[wa\|tr\]_ matches _starwars_ and _startrek_
* Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions scored 
by how close they are to the query prefix
** *Example:*

[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3118 - Failure

2015-05-14 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/3118/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestBlobHandler

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestBlobHandler:
 1) Thread[id=98, name=qtp1425994173-98, state=RUNNABLE, 
group=TGRP-TestBlobHandler] at 
java.net.PlainSocketImpl.socketConnect(Native Method) at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
 at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
 at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)  
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at 
java.net.Socket.connect(Socket.java:579) at 
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
 at 
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
 at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
 at 
org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
 at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
 at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
 at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
 at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
 at 
org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:528) 
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:426) at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:175)
 at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:168)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
 at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:105)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
 at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
 at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300) 
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
 at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  
   at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
 at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)   
  at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)   
  at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  
   at org.eclipse.jetty.server.Server.handle(Server.java:497) at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) 
at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  
   at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.handler.TestBlobHandler: 
   1) Thread[id=98, name=qtp1425994173-98, state=RUNNABLE, 
group=TGRP-TestBlobHandler]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at 
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
at 
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at

[jira] [Commented] (SOLR-6637) Solr should have a way to restore a core

2015-05-14 Thread Greg Solovyev (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544636#comment-14544636
 ] 

Greg Solovyev commented on SOLR-6637:
-

Would it make sense to add an ability to upload index files to target Solr 
instance via HTTP using ContentStreams or is it a bad idea for any reason?

 Solr should have a way to restore a core
 

 Key: SOLR-6637
 URL: https://issues.apache.org/jira/browse/SOLR-6637
 Project: Solr
  Issue Type: Improvement
Reporter: Varun Thacker
Assignee: Varun Thacker
 Fix For: Trunk, 5.2

 Attachments: SOLR-6637.patch, SOLR-6637.patch, SOLR-6637.patch, 
 SOLR-6637.patch, SOLR-6637.patch, SOLR-6637.patch, SOLR-6637.patch, 
 SOLR-6637.patch, SOLR-6637.patch, SOLR-6637.patch


 We have a core backup command which backs up the index. We should have a 
 restore command too. 
 This would restore any named snapshots created by the replication handlers 
 backup command.
 While working on this patch right now I realized that during backup we only 
 backup the index. Should we backup the conf files also? Any thoughts? I could 
 separate Jira for this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-7543) Create GraphQuery that allows graph traversal as a query operator.


[ 
https://issues.apache.org/jira/browse/SOLR-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544713#comment-14544713
 ] 

Dennis Gove edited comment on SOLR-7543 at 5/15/15 1:20 AM:


For interface/semantics, I think this might be able to benefit from the 
Expression stuff recently added for streams (SOLR-7377). With that, you could 
do something like

{code}
graph(root=search(collection1, q=some query, fl=used fields), 
traverse=search(collection1, q=some dynamic query,fl=used fields), 
on=parent.field=child.field, maxDepth=5, returnRoot=true, 
returnOnlyLeaf=false)
{code}

This would also allow you to do other things like make use of stream merging, 
uniquing, etc

Would even allow for tree traversal across multiple collections.


was (Author: dpgove):
For interface/semantics, I think this might be able to benefit from the 
Expression stuff recently added for streams (SOLR-7377). With that, you could 
do something like

{code}
graph(root=search(collection1, q=some query, fl=used fields), 
traverse=search(collection1, q=some dynamic query,fl=used fields), 
on=parent.field=child.field, maxDepth=5, returnRoot=true, 
returnOnlyLeaf=false)
{code}

This would also allow you to do other things like make use of stream merging, 
uniquing, etc

Would even allow for tree traversal across collections.

 Create GraphQuery that allows graph traversal as a query operator.
 --

 Key: SOLR-7543
 URL: https://issues.apache.org/jira/browse/SOLR-7543
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Kevin Watters
Priority: Minor

 I have a GraphQuery that I implemented a long time back that allows a user to 
 specify a startQuery to identify which documents to start graph traversal 
 from.  It then gathers up the edge ids for those documents , optionally 
 applies an additional filter.  The query is then re-executed continually 
 until no new edge ids are identified.  I am currently hosting this code up at 
 https://github.com/kwatters/solrgraph and I would like to work with the 
 community to get some feedback and ultimately get it committed back in as a 
 lucene query.
 Here's a bit more of a description of the parameters for the query / graph 
 traversal:
 q - the initial start query that identifies the universe of documents to 
 start traversal from.
 fromField - the field name that contains the node id
 toField - the name of the field that contains the edge id(s).
 traversalFilter - this is an additional query that can be supplied to limit 
 the scope of graph traversal to just the edges that satisfy the 
 traversalFilter query.
 maxDepth - integer specifying how deep the breadth first search should go.
 returnStartNodes - boolean to determine if the documents that matched the 
 original q should be returned as part of the graph.
 onlyLeafNodes - boolean that filters the graph query to only return 
 documents/nodes that have no edges.
 We identify a set of documents with q as any arbitrary lucene query.  It 
 will collect the values in the fromField, create an OR query with those 
 values , optionally apply an additional constraint from the traversalFilter 
 and walk the result set until no new edges are detected.  Traversal can also 
 be stopped at N hops away as defined with the maxDepth.  This is a BFS 
 (Breadth First Search) algorithm.  Cycle detection is done by not revisiting 
 the same document for edge extraction.  
 This query operator does not keep track of how you arrived at the document, 
 but only that the traversal did arrive at the document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 847 - Failure

2015-05-14 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/847/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
IOException occured when talking to server at: 
http://127.0.0.1:64714/x_n/qx/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://127.0.0.1:64714/x_n/qx/collection1
at 
__randomizedtesting.SeedInfo.seed([5072D7AB398227EF:D826E871977E4A17]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
at 
org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:224)
at 
org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:166)
at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:676)
at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Commented] (SOLR-7543) Create GraphQuery that allows graph traversal as a query operator.

[
https://issues.apache.org/jira/browse/SOLR-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544713#comment-14544713
]

Dennis Gove commented on SOLR-7543:
---

For interface/semantics, I think this might be able to benefit from the
Expression stuff recently added for streams (SOLR-7377). With that, you could
do something like

{code}
graph(root=search(collection1, q=some query, fl=used fields),
traverse=search(collection1, q=some dynamic query,fl=used fields),
on=parent.field=child.field, maxDepth=5, returnRoot=true,
returnOnlyLeaf=false)
{code}

This would also allow you to do other things like make use of stream merging,
uniquing, etc

Create GraphQuery that allows graph traversal as a query operator.
--

Key: SOLR-7543
URL: https://issues.apache.org/jira/browse/SOLR-7543
Project: Solr
Issue Type: New Feature
Components: search
Reporter: Kevin Watters
Priority: Minor

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-7543) Create GraphQuery that allows graph traversal as a query operator.


[ 
https://issues.apache.org/jira/browse/SOLR-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544713#comment-14544713
 ] 

Dennis Gove edited comment on SOLR-7543 at 5/15/15 1:19 AM:


For interface/semantics, I think this might be able to benefit from the 
Expression stuff recently added for streams (SOLR-7377). With that, you could 
do something like

{code}
graph(root=search(collection1, q=some query, fl=used fields), 
traverse=search(collection1, q=some dynamic query,fl=used fields), 
on=parent.field=child.field, maxDepth=5, returnRoot=true, 
returnOnlyLeaf=false)
{code}

This would also allow you to do other things like make use of stream merging, 
uniquing, etc

Would even allow for tree traversal across collections.


was (Author: dpgove):
For interface/semantics, I think this might be able to benefit from the 
Expression stuff recently added for streams (SOLR-7377). With that, you could 
do something like

{code}
graph(root=search(collection1, q=some query, fl=used fields), 
traverse=search(collection1, q=some dynamic query,fl=used fields), 
on=parent.field=child.field, maxDepth=5, returnRoot=true, 
returnOnlyLeaf=false)
{code}

This would also allow you to do other things like make use of stream merging, 
uniquing, etc

 Create GraphQuery that allows graph traversal as a query operator.
 --

 Key: SOLR-7543
 URL: https://issues.apache.org/jira/browse/SOLR-7543
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Kevin Watters
Priority: Minor

 I have a GraphQuery that I implemented a long time back that allows a user to 
 specify a startQuery to identify which documents to start graph traversal 
 from.  It then gathers up the edge ids for those documents , optionally 
 applies an additional filter.  The query is then re-executed continually 
 until no new edge ids are identified.  I am currently hosting this code up at 
 https://github.com/kwatters/solrgraph and I would like to work with the 
 community to get some feedback and ultimately get it committed back in as a 
 lucene query.
 Here's a bit more of a description of the parameters for the query / graph 
 traversal:
 q - the initial start query that identifies the universe of documents to 
 start traversal from.
 fromField - the field name that contains the node id
 toField - the name of the field that contains the edge id(s).
 traversalFilter - this is an additional query that can be supplied to limit 
 the scope of graph traversal to just the edges that satisfy the 
 traversalFilter query.
 maxDepth - integer specifying how deep the breadth first search should go.
 returnStartNodes - boolean to determine if the documents that matched the 
 original q should be returned as part of the graph.
 onlyLeafNodes - boolean that filters the graph query to only return 
 documents/nodes that have no edges.
 We identify a set of documents with q as any arbitrary lucene query.  It 
 will collect the values in the fromField, create an OR query with those 
 values , optionally apply an additional constraint from the traversalFilter 
 and walk the result set until no new edges are detected.  Traversal can also 
 be stopped at N hops away as defined with the maxDepth.  This is a BFS 
 (Breadth First Search) algorithm.  Cycle detection is done by not revisiting 
 the same document for edge extraction.  
 This query operator does not keep track of how you arrived at the document, 
 but only that the traversal did arrive at the document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7537) Could not find or load main class org.apache.solr.util.SimplePostTool

2015-05-14 Thread Peng Li (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544810#comment-14544810
 ] 

Peng Li commented on SOLR-7537:
---

You are right Erick. In Linux, the below command works well:
$ bin/solr create -c name
$ bin/post -c gettingstarted example/exampledocs/*.xml

In windows, I can use `solr create -c gettingstarted` to create a collection. 
But I can't use `post` command. I could only use `example/exampledoc/java 
-Dc=gettingstarted -jar post.jar *.xml` to post. But I think this is absurd.

So, now the question is how do I use post command in Windows. Thanks!

 Could not find or load main class org.apache.solr.util.SimplePostTool
 -

 Key: SOLR-7537
 URL: https://issues.apache.org/jira/browse/SOLR-7537
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 5.1
 Environment: Windows 8.1, cygwin4.3.33
Reporter: Peng Li

 In solr-5.1.0/bin folder, I typed below command ../doc folder has 
 readme.docx
 sh post -c gettingstarted ../doc
 And I got below exception:
 c:\Java\jdk1.8.0_20/bin/java -classpath 
 /cygdrive/c/Users/lipeng/_Main/Servers/solr-5.1.0/dist/solr-core-5.1.0.jar 
 -Dauto=yes -Dc=gettingstarted -Ddata=files -Drecursive=yes 
 org.apache.solr.util.SimplePostTool ../doc
 Error: Could not find or load main class org.apache.solr.util.SimplePostTool
 I followed instruction from here: 
 http://lucene.apache.org/solr/quickstart.html
 Can you help me to take a look at? Thank you!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-7508) SolrParams.toMultiMap() does not handle arrays

2015-05-14 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-7508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-7508.
--
Resolution: Fixed
  Assignee: Noble Paul

 SolrParams.toMultiMap() does not handle arrays
 --

 Key: SOLR-7508
 URL: https://issues.apache.org/jira/browse/SOLR-7508
 Project: Solr
  Issue Type: Bug
  Components: SolrJ
Affects Versions: 5.0, 5.1
Reporter: Thomas Scheffler
Assignee: Noble Paul
Priority: Trivial
  Labels: easyfix, easytest
 Attachments: SOLRJ-7508.patch


 Following JUnit test to show what I mean:
 {code}
 ModifiableSolrParams params = new ModifiableSolrParams();
 String[] paramValues = new String[] { title:junit, author:john };
 String paramName = fq;
 params.add(paramName, paramValues);
 NamedListObject namedList = params.toNamedList();
 assertEquals(parameter values are not equal, paramValues, 
 namedList.get(paramName));
 MapString, String[] multiMap = SolrParams.toMultiMap(namedList);
 assertEquals(Expected  + paramValues.length +  values, 
 paramValues.length, multiMap.get(paramName).length);
 {code}
 The first {{assertEquals()}} will run fine, while the last one triggers the 
 error. Suddenly the length of the array is 1 and it's value of {{fq}} is 
 like {{[Ljava.lang.String;@6f09c9c0}}. Looking into the code I see that the 
 toMultiMap() method does not even look for arrays.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7508) SolrParams.toMultiMap() does not handle arrays


[ 
https://issues.apache.org/jira/browse/SOLR-7508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544970#comment-14544970
 ] 

ASF subversion and git services commented on SOLR-7508:
---

Commit 1679495 from [~noble.paul] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679495 ]

SOLR-7508: SolrParams.toMultiMap() does not handle arrays

 SolrParams.toMultiMap() does not handle arrays
 --

 Key: SOLR-7508
 URL: https://issues.apache.org/jira/browse/SOLR-7508
 Project: Solr
  Issue Type: Bug
  Components: SolrJ
Affects Versions: 5.0, 5.1
Reporter: Thomas Scheffler
Priority: Trivial
  Labels: easyfix, easytest
 Attachments: SOLRJ-7508.patch


 Following JUnit test to show what I mean:
 {code}
 ModifiableSolrParams params = new ModifiableSolrParams();
 String[] paramValues = new String[] { title:junit, author:john };
 String paramName = fq;
 params.add(paramName, paramValues);
 NamedListObject namedList = params.toNamedList();
 assertEquals(parameter values are not equal, paramValues, 
 namedList.get(paramName));
 MapString, String[] multiMap = SolrParams.toMultiMap(namedList);
 assertEquals(Expected  + paramValues.length +  values, 
 paramValues.length, multiMap.get(paramName).length);
 {code}
 The first {{assertEquals()}} will run fine, while the last one triggers the 
 error. Suddenly the length of the array is 1 and it's value of {{fq}} is 
 like {{[Ljava.lang.String;@6f09c9c0}}. Looking into the code I see that the 
 toMultiMap() method does not even look for arrays.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.9.0-ea-b60) - Build # 12499 - Still Failing!

2015-05-14 Thread ASF subversion and git services (JIRA)

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/12499/
Java: 32bit/jdk1.9.0-ea-b60 -server -XX:+UseSerialGC

No tests ran.

Build Log:
[...truncated 169 lines...]
ERROR: Publisher 'Publish JUnit test result report' failed: No test report 
files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6478) Test execution can hang with java.security.debug

2015-05-14 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543500#comment-14543500
 ] 

Dawid Weiss commented on LUCENE-6478:
-

Note how recursive interaction between the security framework and gson is the 
problem here:
{code}
WARN: Unhandled exception in event serialization. - 
java.lang.IllegalStateException
at 
com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson$FutureTypeAdapter.write(Gson.java:897)
at 
com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:600)
at 
com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:100)
at 
com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81)
at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$4.write(SlaveMain.java:434)
at 
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
at java.io.PrintStream.write(PrintStream.java:480)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:806)
at sun.security.util.Debug.println(Debug.java:162)
at 
java.security.AccessControlContext.checkPermission(AccessControlContext.java:463)
at 
java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at 
java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:128)
at 
com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.ConstructorConstructor.newDefaultCons
tructor(ConstructorConstructor.java:97)
at 
com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.ConstructorConstructor.get(Constructo
rConstructor.java:79)
at 
com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.cre
ate(ReflectiveTypeAdapterFactory.java:82)
at 
com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.getAdapter(Gson.java:359)
at 
com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:592)
at 
com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:100)
at 
com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81)
at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$4.write(SlaveMain.java:434)
at 
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
at java.io.PrintStream.write(PrintStream.java:480)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:806)
at sun.security.util.Debug.println(Debug.java:162)
at 
java.security.AccessController.checkPermission(AccessController.java:878)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.io.File.exists(File.java:814)
at java.io.WinNTFileSystem.canonicalize(WinNTFileSystem.java:434)
at java.io.File.getCanonicalPath(File.java:618)
at java.io.FilePermission$1.run(FilePermission.java:215)
at java.io.FilePermission$1.run(FilePermission.java:203)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.FilePermission.init(FilePermission.java:203)
at java.io.FilePermission.init(FilePermission.java:277)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.io.File.exists(File.java:814)
at sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1081)
at sun.misc.URLClassPath.getResource(URLClassPath.java:199)
at java.net.URLClassLoader$1.run(URLClassLoader.java:364)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at

[jira] [Commented] (SOLR-7531) Config API is merging certain key names together

2015-05-14 Thread Shalin Shekhar Mangar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543256#comment-14543256
 ] 

Shalin Shekhar Mangar commented on SOLR-7531:
-

I don't think the test you have added is actually testing what was reported in 
this issue. I don't see any mention of autoCommitMaxDocs in the test.

 Config API is merging certain key names together
 

 Key: SOLR-7531
 URL: https://issues.apache.org/jira/browse/SOLR-7531
 Project: Solr
  Issue Type: Bug
Affects Versions: 5.0, 5.1
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: Trunk, 5.2


 Starting from a new Solr 5.0 install
 {code}
 ./bin/solr start -e schemaless
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that there is a key called autoCommmitMaxDocs 
 under the updateHandler section.
 {code}
 curl 'http://localhost:8983/solr/gettingstarted/config' -H 
 'Content-type:application/json' -d '{set-property : 
 {updateHandler.autoCommit.maxDocs : 5000}}'
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that both the value of updateHandler  autoCommit  
 maxDocs and updateHandler  autoCommitMaxDocs is now set to 5000



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7531) Config API is merging certain key names together

2015-05-14 Thread Shalin Shekhar Mangar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543255#comment-14543255
 ] 

Shalin Shekhar Mangar commented on SOLR-7531:
-

What is this Pending Closed resolution? Why not Fixed?

 Config API is merging certain key names together
 

 Key: SOLR-7531
 URL: https://issues.apache.org/jira/browse/SOLR-7531
 Project: Solr
  Issue Type: Bug
Affects Versions: 5.0, 5.1
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: Trunk, 5.2


 Starting from a new Solr 5.0 install
 {code}
 ./bin/solr start -e schemaless
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that there is a key called autoCommmitMaxDocs 
 under the updateHandler section.
 {code}
 curl 'http://localhost:8983/solr/gettingstarted/config' -H 
 'Content-type:application/json' -d '{set-property : 
 {updateHandler.autoCommit.maxDocs : 5000}}'
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that both the value of updateHandler  autoCommit  
 maxDocs and updateHandler  autoCommitMaxDocs is now set to 5000



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7531) Config API is merging certain key names together


[ 
https://issues.apache.org/jira/browse/SOLR-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543282#comment-14543282
 ] 

ASF subversion and git services commented on SOLR-7531:
---

Commit 1679320 from [~noble.paul] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679320 ]

SOLR-7531: some more tests

 Config API is merging certain key names together
 

 Key: SOLR-7531
 URL: https://issues.apache.org/jira/browse/SOLR-7531
 Project: Solr
  Issue Type: Bug
Affects Versions: 5.0, 5.1
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: Trunk, 5.2


 Starting from a new Solr 5.0 install
 {code}
 ./bin/solr start -e schemaless
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that there is a key called autoCommmitMaxDocs 
 under the updateHandler section.
 {code}
 curl 'http://localhost:8983/solr/gettingstarted/config' -H 
 'Content-type:application/json' -d '{set-property : 
 {updateHandler.autoCommit.maxDocs : 5000}}'
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that both the value of updateHandler  autoCommit  
 maxDocs and updateHandler  autoCommitMaxDocs is now set to 5000



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7275) Pluggable authorization module in Solr


 [ 
https://issues.apache.org/jira/browse/SOLR-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-7275:
---
Fix Version/s: 5.2

 Pluggable authorization module in Solr
 --

 Key: SOLR-7275
 URL: https://issues.apache.org/jira/browse/SOLR-7275
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Fix For: 5.2

 Attachments: SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch


 Solr needs an interface that makes it easy for different authorization 
 systems to be plugged into it. Here's what I plan on doing:
 Define an interface {{SolrAuthorizationPlugin}} with one single method 
 {{isAuthorized}}. This would take in a {{SolrRequestContext}} object and 
 return an {{SolrAuthorizationResponse}} object. The object as of now would 
 only contain a single boolean value but in the future could contain more 
 information e.g. ACL for document filtering etc.
 The reason why we need a context object is so that the plugin doesn't need to 
 understand Solr's capabilities e.g. how to extract the name of the collection 
 or other information from the incoming request as there are multiple ways to 
 specify the target collection for a request. Similarly request type can be 
 specified by {{qt}} or {{/handler_name}}.
 Flow:
 Request - SolrDispatchFilter - isAuthorized(context) - Process/Return.
 {code}
 public interface SolrAuthorizationPlugin {
   public SolrAuthorizationResponse isAuthorized(SolrRequestContext context);
 }
 {code}
 {code}
 public  class SolrRequestContext {
   UserInfo; // Will contain user context from the authentication layer.
   HTTPRequest request;
   Enum OperationType; // Correlated with user roles.
   String[] CollectionsAccessed;
   String[] FieldsAccessed;
   String Resource;
 }
 {code}
 {code}
 public class SolrAuthorizationResponse {
   boolean authorized;
   public boolean isAuthorized();
 }
 {code}
 User Roles: 
 * Admin
 * Collection Level:
   * Query
   * Update
   * Admin
 Using this framework, an implementation could be written for specific 
 security systems e.g. Apache Ranger or Sentry. It would keep all the security 
 system specific code out of Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-7500) Remove pathPrefix from SolrDispatchFilter as Solr no longer runs as a part of a bigger webapp

2015-05-14 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta resolved SOLR-7500.

Resolution: Pending Closed

 Remove pathPrefix from SolrDispatchFilter as Solr no longer runs as a part of 
 a bigger webapp
 -

 Key: SOLR-7500
 URL: https://issues.apache.org/jira/browse/SOLR-7500
 Project: Solr
  Issue Type: Improvement
Reporter: Anshum Gupta
Assignee: Anshum Gupta
Priority: Minor
 Fix For: 5.2

 Attachments: SOLR-7500.patch


 SolrDispatchFilter has support for Solr running as part of a bigger webapp 
 but as we've moved away from that concept, it makes sense to clean up the 
 code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7503) Recovery after ZK session expiration happens in a single thread for all cores in a node

2015-05-14 Thread Shalin Shekhar Mangar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543261#comment-14543261
 ] 

Shalin Shekhar Mangar commented on SOLR-7503:
-

+1 LGTM

 Recovery after ZK session expiration happens in a single thread for all cores 
 in a node
 ---

 Key: SOLR-7503
 URL: https://issues.apache.org/jira/browse/SOLR-7503
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 5.1
Reporter: Shalin Shekhar Mangar
Assignee: Timothy Potter
  Labels: impact-high
 Fix For: Trunk, 5.2

 Attachments: SOLR-7503.patch


 Currently cores are registered in parallel in an executor. However, when 
 there's a ZK expiration, the recovery, which also happens in the register 
 call, happens in a single thread:
 https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/cloud/ZkController.java#L300
 We should make these happen in parallel as well so that recovery after ZK 
 expiration doesn't take forever.
 Thanks to [~mewmewball] for catching this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7275) Pluggable authorization module in Solr


[ 
https://issues.apache.org/jira/browse/SOLR-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543269#comment-14543269
 ] 

ASF subversion and git services commented on SOLR-7275:
---

Commit 1679317 from [~anshumg] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679317 ]

SOLR-7275: Authorization framework for Solr. It defines an interface and a 
mechanism to create, load and use an Authorization plugin.(merge from trunk)

 Pluggable authorization module in Solr
 --

 Key: SOLR-7275
 URL: https://issues.apache.org/jira/browse/SOLR-7275
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Attachments: SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch


 Solr needs an interface that makes it easy for different authorization 
 systems to be plugged into it. Here's what I plan on doing:
 Define an interface {{SolrAuthorizationPlugin}} with one single method 
 {{isAuthorized}}. This would take in a {{SolrRequestContext}} object and 
 return an {{SolrAuthorizationResponse}} object. The object as of now would 
 only contain a single boolean value but in the future could contain more 
 information e.g. ACL for document filtering etc.
 The reason why we need a context object is so that the plugin doesn't need to 
 understand Solr's capabilities e.g. how to extract the name of the collection 
 or other information from the incoming request as there are multiple ways to 
 specify the target collection for a request. Similarly request type can be 
 specified by {{qt}} or {{/handler_name}}.
 Flow:
 Request - SolrDispatchFilter - isAuthorized(context) - Process/Return.
 {code}
 public interface SolrAuthorizationPlugin {
   public SolrAuthorizationResponse isAuthorized(SolrRequestContext context);
 }
 {code}
 {code}
 public  class SolrRequestContext {
   UserInfo; // Will contain user context from the authentication layer.
   HTTPRequest request;
   Enum OperationType; // Correlated with user roles.
   String[] CollectionsAccessed;
   String[] FieldsAccessed;
   String Resource;
 }
 {code}
 {code}
 public class SolrAuthorizationResponse {
   boolean authorized;
   public boolean isAuthorized();
 }
 {code}
 User Roles: 
 * Admin
 * Collection Level:
   * Query
   * Update
   * Admin
 Using this framework, an implementation could be written for specific 
 security systems e.g. Apache Ranger or Sentry. It would keep all the security 
 system specific code out of Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7275) Pluggable authorization module in Solr

2015-05-14 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543266#comment-14543266
 ] 

ASF subversion and git services commented on SOLR-7275:
---

Commit 1679316 from [~anshumg] in branch 'dev/trunk'
[ https://svn.apache.org/r1679316 ]

SOLR-7275: Authorization framework for Solr. It defines an interface and a 
mechanism to create, load and use an Authorization plugin.

 Pluggable authorization module in Solr
 --

 Key: SOLR-7275
 URL: https://issues.apache.org/jira/browse/SOLR-7275
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Attachments: SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch


 Solr needs an interface that makes it easy for different authorization 
 systems to be plugged into it. Here's what I plan on doing:
 Define an interface {{SolrAuthorizationPlugin}} with one single method 
 {{isAuthorized}}. This would take in a {{SolrRequestContext}} object and 
 return an {{SolrAuthorizationResponse}} object. The object as of now would 
 only contain a single boolean value but in the future could contain more 
 information e.g. ACL for document filtering etc.
 The reason why we need a context object is so that the plugin doesn't need to 
 understand Solr's capabilities e.g. how to extract the name of the collection 
 or other information from the incoming request as there are multiple ways to 
 specify the target collection for a request. Similarly request type can be 
 specified by {{qt}} or {{/handler_name}}.
 Flow:
 Request - SolrDispatchFilter - isAuthorized(context) - Process/Return.
 {code}
 public interface SolrAuthorizationPlugin {
   public SolrAuthorizationResponse isAuthorized(SolrRequestContext context);
 }
 {code}
 {code}
 public  class SolrRequestContext {
   UserInfo; // Will contain user context from the authentication layer.
   HTTPRequest request;
   Enum OperationType; // Correlated with user roles.
   String[] CollectionsAccessed;
   String[] FieldsAccessed;
   String Resource;
 }
 {code}
 {code}
 public class SolrAuthorizationResponse {
   boolean authorized;
   public boolean isAuthorized();
 }
 {code}
 User Roles: 
 * Admin
 * Collection Level:
   * Query
   * Update
   * Admin
 Using this framework, an implementation could be written for specific 
 security systems e.g. Apache Ranger or Sentry. It would keep all the security 
 system specific code out of Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7531) Config API is merging certain key names together

2015-05-14 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543273#comment-14543273
 ] 

ASF subversion and git services commented on SOLR-7531:
---

Commit 1679318 from [~noble.paul] in branch 'dev/trunk'
[ https://svn.apache.org/r1679318 ]

SOLR-7531: some more tests

 Config API is merging certain key names together
 

 Key: SOLR-7531
 URL: https://issues.apache.org/jira/browse/SOLR-7531
 Project: Solr
  Issue Type: Bug
Affects Versions: 5.0, 5.1
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: Trunk, 5.2


 Starting from a new Solr 5.0 install
 {code}
 ./bin/solr start -e schemaless
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that there is a key called autoCommmitMaxDocs 
 under the updateHandler section.
 {code}
 curl 'http://localhost:8983/solr/gettingstarted/config' -H 
 'Content-type:application/json' -d '{set-property : 
 {updateHandler.autoCommit.maxDocs : 5000}}'
 curl 'http://localhost:8983/solr/gettingstarted/config'  config.json
 {code}
 Open config.json and note that both the value of updateHandler  autoCommit  
 maxDocs and updateHandler  autoCommitMaxDocs is now set to 5000



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7275) Pluggable authorization module in Solr

2015-05-14 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543279#comment-14543279
 ] 

ASF subversion and git services commented on SOLR-7275:
---

Commit 1679319 from [~noble.paul] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1679319 ]

SOLR-7275: compile error

 Pluggable authorization module in Solr
 --

 Key: SOLR-7275
 URL: https://issues.apache.org/jira/browse/SOLR-7275
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Attachments: SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch


 Solr needs an interface that makes it easy for different authorization 
 systems to be plugged into it. Here's what I plan on doing:
 Define an interface {{SolrAuthorizationPlugin}} with one single method 
 {{isAuthorized}}. This would take in a {{SolrRequestContext}} object and 
 return an {{SolrAuthorizationResponse}} object. The object as of now would 
 only contain a single boolean value but in the future could contain more 
 information e.g. ACL for document filtering etc.
 The reason why we need a context object is so that the plugin doesn't need to 
 understand Solr's capabilities e.g. how to extract the name of the collection 
 or other information from the incoming request as there are multiple ways to 
 specify the target collection for a request. Similarly request type can be 
 specified by {{qt}} or {{/handler_name}}.
 Flow:
 Request - SolrDispatchFilter - isAuthorized(context) - Process/Return.
 {code}
 public interface SolrAuthorizationPlugin {
   public SolrAuthorizationResponse isAuthorized(SolrRequestContext context);
 }
 {code}
 {code}
 public  class SolrRequestContext {
   UserInfo; // Will contain user context from the authentication layer.
   HTTPRequest request;
   Enum OperationType; // Correlated with user roles.
   String[] CollectionsAccessed;
   String[] FieldsAccessed;
   String Resource;
 }
 {code}
 {code}
 public class SolrAuthorizationResponse {
   boolean authorized;
   public boolean isAuthorized();
 }
 {code}
 User Roles: 
 * Admin
 * Collection Level:
   * Query
   * Update
   * Admin
 Using this framework, an implementation could be written for specific 
 security systems e.g. Apache Ranger or Sentry. It would keep all the security 
 system specific code out of Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7275) Pluggable authorization module in Solr


[ 
https://issues.apache.org/jira/browse/SOLR-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543290#comment-14543290
 ] 

Anshum Gupta commented on SOLR-7275:


Damn! Thanks for fixing this. Just discovered I ran the test using Java8.

 Pluggable authorization module in Solr
 --

 Key: SOLR-7275
 URL: https://issues.apache.org/jira/browse/SOLR-7275
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
Assignee: Anshum Gupta
 Attachments: SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, SOLR-7275.patch, 
 SOLR-7275.patch, SOLR-7275.patch


 Solr needs an interface that makes it easy for different authorization 
 systems to be plugged into it. Here's what I plan on doing:
 Define an interface {{SolrAuthorizationPlugin}} with one single method 
 {{isAuthorized}}. This would take in a {{SolrRequestContext}} object and 
 return an {{SolrAuthorizationResponse}} object. The object as of now would 
 only contain a single boolean value but in the future could contain more 
 information e.g. ACL for document filtering etc.
 The reason why we need a context object is so that the plugin doesn't need to 
 understand Solr's capabilities e.g. how to extract the name of the collection 
 or other information from the incoming request as there are multiple ways to 
 specify the target collection for a request. Similarly request type can be 
 specified by {{qt}} or {{/handler_name}}.
 Flow:
 Request - SolrDispatchFilter - isAuthorized(context) - Process/Return.
 {code}
 public interface SolrAuthorizationPlugin {
   public SolrAuthorizationResponse isAuthorized(SolrRequestContext context);
 }
 {code}
 {code}
 public  class SolrRequestContext {
   UserInfo; // Will contain user context from the authentication layer.
   HTTPRequest request;
   Enum OperationType; // Correlated with user roles.
   String[] CollectionsAccessed;
   String[] FieldsAccessed;
   String Resource;
 }
 {code}
 {code}
 public class SolrAuthorizationResponse {
   boolean authorized;
   public boolean isAuthorized();
 }
 {code}
 User Roles: 
 * Admin
 * Collection Level:
   * Query
   * Update
   * Admin
 Using this framework, an implementation could be written for specific 
 security systems e.g. Apache Ranger or Sentry. It would keep all the security 
 system specific code out of Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7541) CollectionsHandler#createNodeIfNotExists is a duplicate of ZkCmdExecutor#ensureExists

2015-05-14 Thread Varun Thacker (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-7541:

Attachment: SOLR-7541.patch

Simple patch with attached CHANGES entry. Tests and precommit passes. 

 CollectionsHandler#createNodeIfNotExists is a duplicate of 
 ZkCmdExecutor#ensureExists
 -

 Key: SOLR-7541
 URL: https://issues.apache.org/jira/browse/SOLR-7541
 Project: Solr
  Issue Type: Improvement
Reporter: Varun Thacker
Priority: Minor
 Attachments: SOLR-7541.patch


 Looks like CollectionsHandler#createNodeIfNotExists is a duplicate of 
 ZkCmdExecutor#ensureExists . Both do the same thing so we could remove 
 CollectionsHandler#createNodeIfNotExists.
 Also looking at {{ZkCmdExecutor#ensureExists(final String path, final byte[] 
 data,CreateMode createMode, final SolrZkClient zkClient)}} the createMode 
 parameter is getting discarded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7274) Pluggable authentication module in Solr


 [ 
https://issues.apache.org/jira/browse/SOLR-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-7274:
---
Attachment: SOLR-7274.patch

 Pluggable authentication module in Solr
 ---

 Key: SOLR-7274
 URL: https://issues.apache.org/jira/browse/SOLR-7274
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
 Attachments: SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch, 
 SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch


 It would be good to have Solr support different authentication protocols.
 To begin with, it'd be good to have support for kerberos and basic auth.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7274) Pluggable authentication module in Solr


 [ 
https://issues.apache.org/jira/browse/SOLR-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-7274:
---
Attachment: (was: SOLR-7274.patch)

 Pluggable authentication module in Solr
 ---

 Key: SOLR-7274
 URL: https://issues.apache.org/jira/browse/SOLR-7274
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
 Attachments: SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch, 
 SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch


 It would be good to have Solr support different authentication protocols.
 To begin with, it'd be good to have support for kerberos and basic auth.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7274) Pluggable authentication module in Solr


[ 
https://issues.apache.org/jira/browse/SOLR-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543359#comment-14543359
 ] 

Anshum Gupta commented on SOLR-7274:


Thanks for the patch Ishan. Here's some feedback:
# You've converted protected methods to be static protected, that doesn't sound 
right to me. It makes more sense to merge the AuthenticationLayerFilter with 
SDF and not change those methods. SDF is just a wrapper now and so this should 
be easy to plugin. I'm not sure but it's perhaps just a matter of moving the 
code into SDF.
# There are unused imports in your code, you should clean them up.

 Pluggable authentication module in Solr
 ---

 Key: SOLR-7274
 URL: https://issues.apache.org/jira/browse/SOLR-7274
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
 Attachments: SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch, 
 SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch


 It would be good to have Solr support different authentication protocols.
 To begin with, it'd be good to have support for kerberos and basic auth.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7274) Pluggable authentication module in Solr


 [ 
https://issues.apache.org/jira/browse/SOLR-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-7274:
---
Attachment: SOLR-7274.patch

Got rid of the AuthenticationLayerFilter and moved all of that into SDF. Also 
cleaned up unused imports.
Another pair of eyes would be good to confirm that it's all good. 
[~ichattopadhyaya] and [~gchanan]?

 Pluggable authentication module in Solr
 ---

 Key: SOLR-7274
 URL: https://issues.apache.org/jira/browse/SOLR-7274
 Project: Solr
  Issue Type: Sub-task
Reporter: Anshum Gupta
 Attachments: SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch, 
 SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch, SOLR-7274.patch


 It would be good to have Solr support different authentication protocols.
 To begin with, it'd be good to have support for kerberos and basic auth.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6480) Extend Simple GeoPointField Type to 3d

2015-05-14 Thread Karl Wright (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543396#comment-14543396
 ] 

Karl Wright commented on LUCENE-6480:
-

Here are (untested) code snippets for converting a single double value to a 
long and back, such that locality is preserved etc.  In order to use these with 
x,y,z you would need to use a modified three-part Morton encoding (similar to 
what was done for GeoPointField) to interleave the bits resulting from the 
encoding method, and the reverse operation to de-convolve the long into three 
sets of bits.

{code}
  public static long toGeoBits(double value) {
// Get the bits for the value
long lvalue = Double.doubleToLongBits(value);
// The exponent will range from 0 to -1023.  Use it to compute the
// amount to shift the mantissa.  But shift left one first, so we 
sign-extend properly.
int exponent = (int)((lvalue  1)  53);
// Shift the mantissa to the leftmost position and add the hidden bit
long mantissa = (lvalue  0x000FL)  11;
// Special value of zero occurs when exponent == 0 and mantissa == 0.  But 
a rigorous interpretation
// of this value is 2^(-1023), which is close enough to zero for our work, 
so we do nothing special.
long fullMantissa = mantissa | 0x8000L;
// Now, compute the amount to shift the mantissa.  We want the mantissa,
// plus the hidden bit, to wind up in the right place.  This will allow 
room for the sign bit too
return (mantissa  (1024-exponent)) | ((lvalue  0x8000L) ^ 
0x8000L);
  }
  
  protected final static double SHIFT_RIGHT_AMT =
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 *
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 *
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 *
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 *
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 *
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 *
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 *
0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5 * 0.5;
  
  public static double fromGeoBits(long bits) {
// To compute the exponent, we need to count the number of 0 bits from 
the next-to-highest position down to
// the first 1.  This is not trivial to do without a loop, which would 
slow us down.  So a better
// way to do the job is to just cast from the long to a double, and extract 
the exponent from
// the result.
double castedDouble = (double)(bits  0x7FFFL);
// We need to divide by 2^63 to get the number back in range, and set the 
appropriate sign.
castedDouble *= SHIFT_RIGHT_AMT;
if ((bits  0x8000L) == 0)
  return -castedDouble;
return castedDouble;
  }
{code}

 Extend Simple GeoPointField Type to 3d 
 ---

 Key: LUCENE-6480
 URL: https://issues.apache.org/jira/browse/LUCENE-6480
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/index
Reporter: Nicholas Knize

 [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] proposes a 
 simple GeoPointField type to lucene core. This field uses 64bit encoding of 2 
 dimensional points to construct sorted term representations of GeoPoints 
 (aka: GeoHashing).
 This feature investigates adding support for encoding 3 dimensional 
 GeoPoints, either by extending GeoPointField to a Geo3DPointField or adding 
 an additional 3d constructor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6478) Test execution can hang with java.security.debug

2015-05-14 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543480#comment-14543480
 ] 

Dawid Weiss commented on LUCENE-6478:
-

I know what's causing this problem. I don't know if I can fix it, but I'll try.

 Test execution can hang with java.security.debug
 

 Key: LUCENE-6478
 URL: https://issues.apache.org/jira/browse/LUCENE-6478
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Dawid Weiss
Assignee: Dawid Weiss

 As reported by Robert:
 {code}
 # clone trunk
 cd lucene/core/
 ant test -Dargs=-Djava.security.debug=access:failure -Dtestcase=TestDemo
 {code}
 Hangs the test runner. The same problem appears to be present in ES builds 
 too. It seems like some kind of weird stream buffer problem, the security 
 framework seems to be writing to the native descriptors directly. Will have 
 to dig (deep...).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Query related to Solr Search

2015-05-14 Thread Dhawan, Kapil (Perkins Coie)

Hi,

I have a query related to Solr Search. I have developed a project through which 
I want ASP.NET webforms to communicate with Solr and search will be faster.
For that You can check below link :
http://stackoverflow.com/questions/29981532/interaction-with-solr-and-asp-net-mvc

Please help me in the above question, Any help will be valuable to me .

Regards
Kapil Dhawan




NOTICE: This communication may contain privileged or other confidential 
information. If you have received it in error, please advise the sender by 
reply email and immediately delete the message and any attachments without 
copying or disclosing the contents. Thank you.

[jira] [Commented] (LUCENE-6464) Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()


[ 
https://issues.apache.org/jira/browse/LUCENE-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543436#comment-14543436
 ] 

Michael McCandless commented on LUCENE-6464:


Thanks [~arcadius], this looks great!

Maybe rename .addToQuery - .addContextToQuery?

In the javadoc for the new uber-expert lookup method, can you link to 
#addContextToQuery as the way to build up the {{contextQuery}} arg?

 Allow possibility to group contexts in AnalyzingInfixSuggester.loockup()
 

 Key: LUCENE-6464
 URL: https://issues.apache.org/jira/browse/LUCENE-6464
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 5.1
Reporter: Arcadius Ahouansou
 Attachments: LUCENE-6464.patch, LUCENE-6464.patch


 This is an enhancement to LUCENE-6050 
 LUCENE-6050 added
 {code}
 lookup(CharSequence key, MapBytesRef, BooleanClause.Occur contextInfo, int 
 num, boolean allTermsRequired, boolean doHighlight)
 {code}
 which allowed to do something like
 (A OR B AND C OR D ...)
 In our use-case, we realise that we need grouping i.e
 (A OR B) AND (C OR D) AND (...)
 In other words, we need the intersection of multiple contexts.
 The attached patch allows to pass in a varargs of map, each one representing 
 the each group. Looks a bit heavy IMHO.
 This is an initial patch.
 The question to [~mikemccand] and [~janechang] is:
 is it better to expose a FilteredQuery/Query and let the user build their own 
 query instead of passing a map?
 Exposing a filteredQuery will probably give the best flexibility to the 
 end-users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Linux (32bit/ibm-j9-jdk7) - Build # 12498 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/12498/
Java: 32bit/ibm-j9-jdk7 
-Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

No tests ran.

Build Log:
[...truncated 31 lines...]
ERROR: Publisher 'Publish JUnit test result report' failed: No test report 
files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_60-ea-b12) - Build # 12671 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/12671/
Java: 32bit/jdk1.8.0_60-ea-b12 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.test

Error Message:
Error from server at http://127.0.0.1:58488/t_s/lw/checkStateVerCol: no servers 
hosting shard: 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:58488/t_s/lw/checkStateVerCol: no servers 
hosting shard: 
at 
__randomizedtesting.SeedInfo.seed([99FED086E7B1E884:11AAEF5C494D857C]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.stateVersionParamTest(CloudSolrClientTest.java:554)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.test(CloudSolrClientTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Commented] (LUCENE-6459) [suggest] Query Interface for suggest API

[
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543434#comment-14543434
]

Michael McCandless commented on LUCENE-6459:

bq. We should use this API for other suggesters, but maybe in a separate issue

Yes, definitely separate!

bq. In terms of new functionality, fuzzy, regex and context queries are added.

OK, cool, these are nice additions.

bq. query-time context boosts

Cool: so you boost some contexts more than others, using ContextQuery.

bq. As we only allow integer index-time weights

I thought we accept long (not int) as index-time weight? (But, I
think that really is overkill... maybe they should just floats, like
per-field boosting at index time). But we can worry about this
later...

bq. one possibility would be to use index-weight + (Int.MAX * boost) instead of
using MaxWeight of suggestions

Sorry I don't understand the idea here?

{quote}
bq. If you try to use ContextQuery against a field that you had not indexed
contexts with (using ContextSuggestField) do you see any error? Maybe this is
too hard.

There should not be any error. A ContextQuery will never be run on a
SuggestField
{quote}

It seems like we could detect this mis-use, since CompletionTerms seems to know
whether the field was indexed with contexts or not? I.e, if I accidentally try
to run a ContextQuery against a field indexed with only SuggestField, it seems
like I should get an exception saying I screwed up ... (similar to trying to
run a PhraseQuery on a field that did not index positions)? Maybe add a simple
test case?

{quote}
A ContextQuery will never be run on a SuggestField,
CompletionQuery rewrites appropriately given the type of the field
(context-enabled or not).
{quote}

OK maybe at that rewrite is the time to throw the exc?

{quote}
This also makes non-context queries work as expected when run against
ContextSuggestField
(as in the query is wrapped as a ContextQuery with no context
filtering/boosting).
{quote}

OK, for that direction it makes sense allow ... good.

{quote}
bq. Are you allowed to mix ContextSuggestField and SuggestField even for the
same field name, within one suggester?

No you are not. If mixed, CompletionFieldsConsumer will throw
IllegalArgumentException upon indexing.
{quote}

OK, excellent.

Can we rename {{TopSuggestDocsCollector.num()}} to maybe .getCountToCollect or
something a bit more verbose?

Net/net this is a nice change, thanks [~areek]!

[suggest] Query Interface for suggest API
-

Key: LUCENE-6459
URL: https://issues.apache.org/jira/browse/LUCENE-6459
Project: Lucene - Core
Issue Type: New Feature
Components: core/search
Affects Versions: 5.1
Reporter: Areek Zillur
Assignee: Areek Zillur
Fix For: Trunk, 5.x, 5.1

Attachments: LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch,
LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch

This patch factors out common indexing/search API used by the recently
introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339].
The motivation is to provide a query interface for FST-based fields
(*SuggestField* and *ContextSuggestField*) for enabling suggestion scoring
and more powerful automaton queries.
Previously, only prefix ‘queries’ with index-time weights were supported but
we can also support:
* Prefix queries expressed as regular expressions: get suggestions that
match multiple prefixes
** Example: _star\[wa\|tr\]_ matches _starwars_ and _startrek_
* Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions
scored by how close they are to the query prefix
** Example: querying for _seper_ will score _separate_ higher then
_superstitious_
* Context Queries: get suggestions boosted and/or filtered based on their
indexed contexts (meta data)
** Example: get typo tolerant suggestions on song names with prefix _like
a roling_ boosting songs with genre _rock_ and _indie_
** Example: get suggestion on all file names starting with _finan_ only
for _user1_ and _user2_
h3. Suggest API
{code}
SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
CompletionQuery query = ...
TopSuggestDocs suggest = searcher.suggest(query, num);
{code}
h3. CompletionQuery
*CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*.
A *CompletionQuery* produces a *CompletionWeight*, which allows
*CompletionQuery* implementations to pass in an automaton that will be
intersected with a FST and allows boosting and meta data extraction from the
intersected partial paths. A *CompletionWeight* produces a
*CompletionScorer*. A *CompletionScorer* executes a Top N search against the
FST with the provided automaton, scoring

[jira] [Commented] (LUCENE-6365) Optimized iteration of finite strings