date:20140315


 [ 
https://issues.apache.org/jira/browse/LUCENE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson reassigned LUCENE-1486:
--

Assignee: Erick Erickson

 Wildcards, ORs etc inside Phrase queries
 

 Key: LUCENE-1486
 URL: https://issues.apache.org/jira/browse/LUCENE-1486
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/queryparser
Affects Versions: 2.4
Reporter: Mark Harwood
Assignee: Erick Erickson
Priority: Minor
 Fix For: 4.7

 Attachments: ComplexPhraseQueryParser.java, LUCENE-1486.patch, 
 LUCENE-1486.patch, LUCENE-1486.patch, LUCENE-1486.patch, LUCENE-1486.patch, 
 LUCENE-1486.patch, LUCENE-1486.patch, Lucene-1486 non default field.patch, 
 TestComplexPhraseQuery.java, junit_complex_phrase_qp_07_21_2009.patch, 
 junit_complex_phrase_qp_07_22_2009.patch


 An extension to the default QueryParser that overrides the parsing of 
 PhraseQueries to allow more complex syntax e.g. wildcards in phrase queries.
 The implementation feels a little hacky - this is arguably better handled in 
 QueryParser itself. This works as a proof of concept  for much of the query 
 parser syntax. Examples from the Junit test include:
   checkMatches(\j*   smyth~\, 1,2); //wildcards and fuzzies 
 are OK in phrases
   checkMatches(\(jo* -john)  smith\, 2); // boolean logic 
 works
   checkMatches(\jo*  smith\~2, 1,2,3); // position logic 
 works.
   
   checkBadQuery(\jo*  id:1 smith\); //mixing fields in a 
 phrase is bad
   checkBadQuery(\jo* \smith\ \); //phrases inside phrases 
 is bad
   checkBadQuery(\jo* [sma TO smZ]\ \); //range queries 
 inside phrases not supported
 Code plus Junit test to follow...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-3758) Allow the ComplexPhraseQueryParser to search order or un-order proximity queries.


 [ 
https://issues.apache.org/jira/browse/LUCENE-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson reassigned LUCENE-3758:
--

Assignee: Erick Erickson

 Allow the ComplexPhraseQueryParser to search order or un-order proximity 
 queries.
 -

 Key: LUCENE-3758
 URL: https://issues.apache.org/jira/browse/LUCENE-3758
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/queryparser
Affects Versions: 4.0-ALPHA
Reporter: Tomás Fernández Löbbe
Assignee: Erick Erickson
Priority: Minor
 Fix For: 4.7

 Attachments: LUCENE-3758.patch


 The ComplexPhraseQueryParser use SpanNearQuery, but always set the inOrder 
 value hardcoded to true. This could be configurable.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1604) Wildcards, ORs etc inside Phrase Queries

[
https://issues.apache.org/jira/browse/SOLR-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936189#comment-13936189
]

Erick Erickson commented on SOLR-1604:
--

OK, I was looking around the patch and think I understand at least some of
what's going on. To drive this forward, I need a couple of things:

1 Vitaliy and Ahmet to resolve the two patches and let me know what the right
one to use is. BTW, Vitaliy, please use svn diff or the equivalent Git
command to create patches. Zipped up sources are much harder to work with.

2 Some idea of a roadmap from here. Straw-man proposal:
2a Close 1486 and open a new JIRA if there's a fix for that if necessary. It
looks to me like this patch can be committed without 1486 and we'll generate a
separate fix.
2b commit 3758, and remove inOrder from this patch, then commit this patch.
2c I've assigned these to myself so I don't lose track of them. I'll look
desperately for cycles to work on them :). But I have a couple of long plane
flights in my future...

3 Of course we need to document the syntax and behavior here, [~ctargett] can
probably point us in the right direction for doing this right by putting it in
the new documentation!

4 I'm also curious what we know now in terms of performance, resource
requirements, that kind of stuff.

5 I notice there's a patch labeled as having to do with license stuff. What's
up there? Is this just putting the headers in the source files?

5 Anything else? Does anyone out there object to moving forward with this?

Wildcards, ORs etc inside Phrase Queries

Key: SOLR-1604
URL: https://issues.apache.org/jira/browse/SOLR-1604
Project: Solr
Issue Type: Improvement
Components: query parsers, search
Affects Versions: 1.4
Reporter: Ahmet Arslan
Assignee: Erick Erickson
Priority: Minor
Attachments: ASF.LICENSE.NOT.GRANTED--ComplexPhrase.zip,
ComplexPhrase-4.2.1.zip, ComplexPhrase-4.7.zip, ComplexPhrase.zip,
ComplexPhrase.zip, ComplexPhrase.zip, ComplexPhrase.zip, ComplexPhrase.zip,
ComplexPhrase.zip, ComplexPhraseQueryParser.java, ComplexPhrase_solr_3.4.zip,
SOLR-1604-alternative.patch, SOLR-1604.patch, SOLR-1604.patch,
SOLR-1604.patch, SOLR-1604.patch

Solr Plugin for ComplexPhraseQueryParser (LUCENE-1486) which supports
wildcards, ORs, ranges, fuzzies inside phrase queries.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Analytics test errors

2014-03-15 Thread Erick Erickson

I was all excited by the lack of errors coming from these tests until
I noticed they were BadApples.

So I took the ExpressionTest BadApple designation out and ran the test
20K times without error (it used to fail on my Mac).

I'm going to pull the other BadApple designations out now that I'm
stealing some cycles to work with this run all the tests a bunch of
times on my laptop and, if I can't repro the problem, un-bad-apple
them and commit to trunk unless there are lots of objections.
Otherwise I don't see how to make forward progress on these.

Apologies for the long period when they generated test noise, I've
been unable to devote any time to it for far too long.

Erick

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1604) Wildcards, ORs etc inside Phrase Queries


[ 
https://issues.apache.org/jira/browse/SOLR-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936212#comment-13936212
 ] 

Ahmet Arslan commented on SOLR-1604:


Here are some clarification regarding zipped attachments :

Zipped attachments are not meant for source code inclusion but for to be 
consumed as solr plugin. They will never be committed. Mainly because zipped 
version(s) include a duplicate code from lucene code base. Duplicated class is 
org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser. 
Duplication is done for two reasons :
* To enable fielded queries. this duplicate code changes package name to 
org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.  Originally 
Somehow this feature forgotten accidentally in LUCENE1468, while committing 
lucene.ComplexPhraseQueryParser. After that commit, package name changed from 
classic to complexPhrase. For this fix it needs to access a field from super 
class. After realizing this, chancing this fields visibility to protected is 
accepted by lazy consensus. This is the 
[patch|https://issues.apache.org/jira/secure/attachment/12513804/LUCENE-1486.patch]
 for this.
* To enable ability change inOrder parameter. In original lucene code inOrder 
parameter is barcoded to true inSpanNearQuery classes. Separate jira for this 
is LUCENE-3758.

By the way, why LUCENE-1486 is re-opened is a mystery. It is not re-opened 
because of unforgotten non-default patch. 

 Wildcards, ORs etc inside Phrase Queries
 

 Key: SOLR-1604
 URL: https://issues.apache.org/jira/browse/SOLR-1604
 Project: Solr
  Issue Type: Improvement
  Components: query parsers, search
Affects Versions: 1.4
Reporter: Ahmet Arslan
Assignee: Erick Erickson
Priority: Minor
 Attachments: ASF.LICENSE.NOT.GRANTED--ComplexPhrase.zip, 
 ComplexPhrase-4.2.1.zip, ComplexPhrase-4.7.zip, ComplexPhrase.zip, 
 ComplexPhrase.zip, ComplexPhrase.zip, ComplexPhrase.zip, ComplexPhrase.zip, 
 ComplexPhrase.zip, ComplexPhraseQueryParser.java, ComplexPhrase_solr_3.4.zip, 
 SOLR-1604-alternative.patch, SOLR-1604.patch, SOLR-1604.patch, 
 SOLR-1604.patch, SOLR-1604.patch


 Solr Plugin for ComplexPhraseQueryParser (LUCENE-1486) which supports 
 wildcards, ORs, ranges, fuzzies inside phrase queries.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1604) Wildcards, ORs etc inside Phrase Queries

[
https://issues.apache.org/jira/browse/SOLR-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936221#comment-13936221
]

Ahmet Arslan commented on SOLR-1604:

bq. Vitaliy and Ahmet to resolve the two patches and let me know what the right
one to use is.
none of them actually. They include source code (ComplexPhraseQueryParser.java)
duplication from lucene. I will attach a patch that consumes lucene's
ComplexPhraseQueryParser created against trunk.

bq. Close 1486 and open a new JIRA if there's a fix for that if necessary. It
looks to me like this patch can be committed without 1486 and we'll generate a
separate fix.
+1. Yes this patch can be committed without LUCENE-1486. +1 for closing
LUCENE-1486 given that it is re-opened mysteriously. +1 for creating a separate
jira for
[this|https://issues.apache.org/jira/secure/attachment/12513804/LUCENE-1486.patch]
functionality just because it is less confusing.

bq. commit 3758, and remove inOrder from this patch, then commit this patch.
Request ability change inOrder parameter came from a user. Robert had
[this|https://issues.apache.org/jira/browse/LUCENE-3758?focusedCommentId=13206996page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13206996]
comment about this.

bq. I notice there's a patch labeled as having to do with license stuff.
This attachment is old. I accidentally forget selection 'ASF inclusion radio
box then. Jira weren't displaying feather icon for that. After that incident
jira had removed that radio button selection option. Attachments are ASF
granted by default now. That file is renamed automatically by infra.

Wildcards, ORs etc inside Phrase Queries

Solr Plugin for ComplexPhraseQueryParser (LUCENE-1486) which supports
wildcards, ORs, ranges, fuzzies inside phrase queries.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-2878) Allow Scorer to expose positions and payloads aka. nuke spans

2014-03-15 Thread Robert Muir (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir reassigned LUCENE-2878:
---

Assignee: Robert Muir  (was: Simon Willnauer)

 Allow Scorer to expose positions and payloads aka. nuke spans 
 --

 Key: LUCENE-2878
 URL: https://issues.apache.org/jira/browse/LUCENE-2878
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/search
Affects Versions: Positions Branch
Reporter: Simon Willnauer
Assignee: Robert Muir
  Labels: gsoc2014
 Fix For: Positions Branch

 Attachments: LUCENE-2878-OR.patch, LUCENE-2878-vs-trunk.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878_trunk.patch, LUCENE-2878_trunk.patch, PosHighlighter.patch, 
 PosHighlighter.patch


 Currently we have two somewhat separate types of queries, the one which can 
 make use of positions (mainly spans) and payloads (spans). Yet Span*Query 
 doesn't really do scoring comparable to what other queries do and at the end 
 of the day they are duplicating lot of code all over lucene. Span*Queries are 
 also limited to other Span*Query instances such that you can not use a 
 TermQuery or a BooleanQuery with SpanNear or anthing like that. 
 Beside of the Span*Query limitation other queries lacking a quiet interesting 
 feature since they can not score based on term proximity since scores doesn't 
 expose any positional information. All those problems bugged me for a while 
 now so I stared working on that using the bulkpostings API. I would have done 
 that first cut on trunk but TermScorer is working on BlockReader that do not 
 expose positions while the one in this branch does. I started adding a new 
 Positions class which users can pull from a scorer, to prevent unnecessary 
 positions enums I added ScorerContext#needsPositions and eventually 
 Scorere#needsPayloads to create the corresponding enum on demand. Yet, 
 currently only TermQuery / TermScorer implements this API and other simply 
 return null instead. 
 To show that the API really works and our BulkPostings work fine too with 
 positions I cut over TermSpanQuery to use a TermScorer under the hood and 
 nuked TermSpans entirely. A nice sideeffect of this was that the Position 
 BulkReading implementation got some exercise which now :) work all with 
 positions while Payloads for bulkreading are kind of experimental in the 
 patch and those only work with Standard codec. 
 So all spans now work on top of TermScorer ( I truly hate spans since today ) 
 including the ones that need Payloads (StandardCodec ONLY)!!  I didn't bother 
 to implement the other codecs yet since I want to get feedback on the API and 
 on this first cut before I go one with it. I will upload the corresponding 
 patch in a minute. 
 I also had to cut over SpanQuery.getSpans(IR) to 
 SpanQuery.getSpans(AtomicReaderContext) which I should probably do on trunk 
 first but after that pain today I need a break first :).
 The patch passes all core tests 
 (org.apache.lucene.search.highlight.HighlighterTest still fails but I didn't 
 look into the MemoryIndex BulkPostings API yet)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5865) Provide a MiniSolrCloudCluster to enable easier testing

2014-03-15 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936239#comment-13936239
 ] 

Mark Miller commented on SOLR-5865:
---

Looks great!

+  // We could upload the minimum set of files rather than the directory, 
but that requires keeping the list up to date
+  ZkController.uploadToZK(zkClient, new File(configDir), 
ZkController.CONFIGS_ZKNODE + / + configName);

The main reason most of the cloud tests have gone with specifying which config 
files to put in zk was that uploading the entire directory of test configs was 
damn slow and then repeated for all cloud tests.

A better solution at some point would be a new test config folder just for 
solrcloud. We already have a lot of configs, but we could probably merge some 
things into this - like the common solrconfig and schema that almost all cloud 
tests use anyway. If we kept it to one set, I think it would be an improvement 
for cloud tests.

 Provide a MiniSolrCloudCluster to enable easier testing
 ---

 Key: SOLR-5865
 URL: https://issues.apache.org/jira/browse/SOLR-5865
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.7, 5.0
Reporter: Gregory Chanan
 Attachments: SOLR-5865.patch


 Today, the SolrCloud tests are based on the LuceneTestCase class hierarchy, 
 which has a couple of issues around support for downstream projects:
 - It's difficult to test SolrCloud support in a downstream project that may 
 have its own test framework.  For example, some projects have support for 
 different storage backends (e.g. Solr/ElasticSearch/HBase) and want tests 
 against each of the different backends.  This is difficult to do cleanly, 
 because the Solr tests require derivation from LuceneTestCase, while the 
 other don't
 - The LuceneTestCase class hierarchy is really designed for internal solr 
 tests (e.g. it randomizes a lot of parameters to get test coverage, but a 
 downstream project probably doesn't care about that).  It's also quite 
 complicated and dense, much more so than a downstream project would want.
 Given these reasons, it would be nice to provide a simple 
 MiniSolrCloudCluster, similar to how HDFS provides a MiniHdfsCluster or 
 HBase provides a MiniHBaseCluster.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-2878) Allow Scorer to expose positions and payloads aka. nuke spans

2014-03-15 Thread Simon Willnauer (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936240#comment-13936240
 ] 

Simon Willnauer commented on LUCENE-2878:
-

Now we are talking

Sent from my iPhone



 Allow Scorer to expose positions and payloads aka. nuke spans 
 --

 Key: LUCENE-2878
 URL: https://issues.apache.org/jira/browse/LUCENE-2878
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/search
Affects Versions: Positions Branch
Reporter: Simon Willnauer
Assignee: Robert Muir
  Labels: gsoc2014
 Fix For: Positions Branch

 Attachments: LUCENE-2878-OR.patch, LUCENE-2878-vs-trunk.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878_trunk.patch, LUCENE-2878_trunk.patch, PosHighlighter.patch, 
 PosHighlighter.patch


 Currently we have two somewhat separate types of queries, the one which can 
 make use of positions (mainly spans) and payloads (spans). Yet Span*Query 
 doesn't really do scoring comparable to what other queries do and at the end 
 of the day they are duplicating lot of code all over lucene. Span*Queries are 
 also limited to other Span*Query instances such that you can not use a 
 TermQuery or a BooleanQuery with SpanNear or anthing like that. 
 Beside of the Span*Query limitation other queries lacking a quiet interesting 
 feature since they can not score based on term proximity since scores doesn't 
 expose any positional information. All those problems bugged me for a while 
 now so I stared working on that using the bulkpostings API. I would have done 
 that first cut on trunk but TermScorer is working on BlockReader that do not 
 expose positions while the one in this branch does. I started adding a new 
 Positions class which users can pull from a scorer, to prevent unnecessary 
 positions enums I added ScorerContext#needsPositions and eventually 
 Scorere#needsPayloads to create the corresponding enum on demand. Yet, 
 currently only TermQuery / TermScorer implements this API and other simply 
 return null instead. 
 To show that the API really works and our BulkPostings work fine too with 
 positions I cut over TermSpanQuery to use a TermScorer under the hood and 
 nuked TermSpans entirely. A nice sideeffect of this was that the Position 
 BulkReading implementation got some exercise which now :) work all with 
 positions while Payloads for bulkreading are kind of experimental in the 
 patch and those only work with Standard codec. 
 So all spans now work on top of TermScorer ( I truly hate spans since today ) 
 including the ones that need Payloads (StandardCodec ONLY)!!  I didn't bother 
 to implement the other codecs yet since I want to get feedback on the API and 
 on this first cut before I go one with it. I will upload the corresponding 
 patch in a minute. 
 I also had to cut over SpanQuery.getSpans(IR) to 
 SpanQuery.getSpans(AtomicReaderContext) which I should probably do on trunk 
 first but after that pain today I need a break first :).
 The patch passes all core tests 
 (org.apache.lucene.search.highlight.HighlighterTest still fails but I didn't 
 look into the MemoryIndex BulkPostings API yet)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-1604) Wildcards, ORs etc inside Phrase Queries


 [ 
https://issues.apache.org/jira/browse/SOLR-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Arslan updated SOLR-1604:
---

Attachment: SOLR-1604.patch

This is solr-only patch (solr/core/src/) and does not touch lucene code case. 
It adds two new java classes (ComplexPhraseQParserPlugin and 
TestComplexPhraseQParserPlugin) and consumes 
o.a.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser

 Wildcards, ORs etc inside Phrase Queries
 

 Key: SOLR-1604
 URL: https://issues.apache.org/jira/browse/SOLR-1604
 Project: Solr
  Issue Type: Improvement
  Components: query parsers, search
Affects Versions: 1.4
Reporter: Ahmet Arslan
Assignee: Erick Erickson
Priority: Minor
 Attachments: ASF.LICENSE.NOT.GRANTED--ComplexPhrase.zip, 
 ComplexPhrase-4.2.1.zip, ComplexPhrase-4.7.zip, ComplexPhrase.zip, 
 ComplexPhrase.zip, ComplexPhrase.zip, ComplexPhrase.zip, ComplexPhrase.zip, 
 ComplexPhrase.zip, ComplexPhraseQueryParser.java, ComplexPhrase_solr_3.4.zip, 
 SOLR-1604-alternative.patch, SOLR-1604.patch, SOLR-1604.patch, 
 SOLR-1604.patch, SOLR-1604.patch, SOLR-1604.patch


 Solr Plugin for ComplexPhraseQueryParser (LUCENE-1486) which supports 
 wildcards, ORs, ranges, fuzzies inside phrase queries.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5189) Numeric DocValues Updates

2014-03-15 Thread Mikhail Khludnev (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936265#comment-13936265
]

Mikhail Khludnev commented on LUCENE-5189:
--

Just want to leave one caveat for memories. When you call
{code}IW.updateNumericDocValue(Term, String, Long){code} make sure that the
term is deeply cloned before. Otherwise, if you modify term or bytes, then the
modified version will be applied. That's might be a problem.

Numeric DocValues Updates
-

Key: LUCENE-5189
URL: https://issues.apache.org/jira/browse/LUCENE-5189
Project: Lucene - Core
Issue Type: New Feature
Components: core/index
Reporter: Shai Erera
Assignee: Shai Erera
Fix For: 4.6, 5.0

Attachments: LUCENE-5189-4x.patch, LUCENE-5189-4x.patch,
LUCENE-5189-no-lost-updates.patch, LUCENE-5189-renames.patch,
LUCENE-5189-segdv.patch, LUCENE-5189-updates-order.patch,
LUCENE-5189-updates-order.patch, LUCENE-5189.patch, LUCENE-5189.patch,
LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch,
LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch,
LUCENE-5189.patch, LUCENE-5189_process_events.patch,
LUCENE-5189_process_events.patch

In LUCENE-4258 we started to work on incremental field updates, however the
amount of changes are immense and hard to follow/consume. The reason is that
we targeted postings, stored fields, DV etc., all from the get go.
I'd like to start afresh here, with numeric-dv-field updates only. There are
a couple of reasons to that:
* NumericDV fields should be easier to update, if e.g. we write all the
values of all the documents in a segment for the updated field (similar to
how livedocs work, and previously norms).
* It's a fairly contained issue, attempting to handle just one data type to
update, yet requires many changes to core code which will also be useful for
updating other data types.
* It has value in and on itself, and we don't need to allow updating all the
data types in Lucene at once ... we can do that gradually.
I have some working patch already which I'll upload next, explaining the
changes.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5189) Numeric DocValues Updates

2014-03-15 Thread Shai Erera (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936279#comment-13936279
]

Shai Erera commented on LUCENE-5189:

I checked the code and it looks the same with e.g. deleteDocuments(Term) - the
Term isn't cloned internally. So your comment pertains to other IW methods.

Numeric DocValues Updates
-

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5770) All attempts to match a SolrCore with it's state in clusterstate.json should be done with the NodeName rather than the baseUrl.

2014-03-15 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936290#comment-13936290
 ] 

Mark Miller commented on SOLR-5770:
---

Awesome, thanks Steve - had not had a chance to look further at this yet. I'll 
try your patch this weekend.

 All attempts to match a SolrCore with it's state in clusterstate.json should 
 be done with the NodeName rather than the baseUrl.
 ---

 Key: SOLR-5770
 URL: https://issues.apache.org/jira/browse/SOLR-5770
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.8, 5.0

 Attachments: SOLR-5770.patch, SOLR-5770.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5488) Fix up test failures for Analytics Component


 [ 
https://issues.apache.org/jira/browse/SOLR-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated SOLR-5488:
-

Attachment: SOLR-5488.patch

Takes of @Ignore and @BadApple.

See comments here:
https://issues.apache.org/jira/browse/SOLR-5685

This fix suddently caused FieldFacetTest to start failing. It fails first time, 
every time. Interestingly, when it does fail it's because 
MinMaxStatsCollection.getStat is looking for the stat min, but this.min is 
null. Seems like it _may_ be related to the mysterious failures we were seeing, 
but I'm grasping at straws. 

I'll be trying ExpressionTest repeatedly to see if we're back now...

 Fix up test failures for Analytics Component
 

 Key: SOLR-5488
 URL: https://issues.apache.org/jira/browse/SOLR-5488
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.7, 5.0
Reporter: Erick Erickson
Assignee: Erick Erickson
 Attachments: SOLR-5488.patch, SOLR-5488.patch, SOLR-5488.patch, 
 SOLR-5488.patch, SOLR-5488.patch, SOLR-5488.patch, SOLR-5488.patch, eoe.errors


 The analytics component has a few test failures, perhaps 
 environment-dependent. This is just to collect the test fixes in one place 
 for convenience when we merge back into 4.x



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_51) - Build # 9800 - Failure!

2014-03-15 Thread Mark Miller

Hmm…only interesting logging I see is this:

57473 T32 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in 
SyncThread:0 took 50531ms which will adversely effect operation latency. See 
the ZooKeeper troubleshooting guide
I wonder if that means that if i boost the connect timeout from 45 to 60 
seconds, it will pass.
Perhaps this machine has some IO issues?

-- 
Mark Miller
about.me/markrmiller

On March 15, 2014 at 9:23:25 AM, Policeman Jenkins Server (jenk...@thetaphi.de) 
wrote:

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9800/  
Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC  

1 tests failed.  
REGRESSION: 
org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch  

Error Message:  
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 
127.0.0.1:44565 within 45000 ms  

Stack Trace:  
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: 
Could not connect to ZooKeeper 127.0.0.1:44565 within 45000 ms  
at __randomizedtesting.SeedInfo.seed([D09CC97019C4AF45:517A47686E9BCF79]:0)  
at org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:150)  
at org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:101)  
at org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:91)  
at 
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
  
at 
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
  
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
  
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
  
at 
org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:78)
  
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)  
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  
at java.lang.reflect.Method.invoke(Method.java:606)  
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
  
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
  
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
  
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
  
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
  
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
  
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
  
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
  
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
  
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
  
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
  
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
  
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
  
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
  
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
  
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
  
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
  
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
  
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
  
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
  
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
  
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
  
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
  
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
  
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  
at

[jira] [Commented] (LUCENE-2878) Allow Scorer to expose positions and payloads aka. nuke spans

2014-03-15 Thread Alan Woodward (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936304#comment-13936304
 ] 

Alan Woodward commented on LUCENE-2878:
---

Ooh, hello.

So the LUCENE-2878 branch is a bit of a mess, in that it has two semi-working 
versions of this code: Simon's initial IntervalIterator API, in the 
o.a.l.search.intervals package, and my DocsEnum.nextPosition() API in 
o.a.l.search.positions.  Simon's code is much more complete, and I've been 
using a separately maintained version of that in production code for various 
clients, which you can see at 
https://github.com/flaxsearch/lucene-solr-intervals.  I think the 
nextPosition() API is nicer, but the IntervalIterator API has the advantage of 
actually working.

The github repository has some other stuff on it too, around making the 
intervals code work across different fields.  The API that I've come up with 
there is not very nice, though.

It would be ace to get this moving again!

 Allow Scorer to expose positions and payloads aka. nuke spans 
 --

 Key: LUCENE-2878
 URL: https://issues.apache.org/jira/browse/LUCENE-2878
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/search
Affects Versions: Positions Branch
Reporter: Simon Willnauer
Assignee: Robert Muir
  Labels: gsoc2014
 Fix For: Positions Branch

 Attachments: LUCENE-2878-OR.patch, LUCENE-2878-vs-trunk.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, LUCENE-2878.patch, 
 LUCENE-2878_trunk.patch, LUCENE-2878_trunk.patch, PosHighlighter.patch, 
 PosHighlighter.patch


 Currently we have two somewhat separate types of queries, the one which can 
 make use of positions (mainly spans) and payloads (spans). Yet Span*Query 
 doesn't really do scoring comparable to what other queries do and at the end 
 of the day they are duplicating lot of code all over lucene. Span*Queries are 
 also limited to other Span*Query instances such that you can not use a 
 TermQuery or a BooleanQuery with SpanNear or anthing like that. 
 Beside of the Span*Query limitation other queries lacking a quiet interesting 
 feature since they can not score based on term proximity since scores doesn't 
 expose any positional information. All those problems bugged me for a while 
 now so I stared working on that using the bulkpostings API. I would have done 
 that first cut on trunk but TermScorer is working on BlockReader that do not 
 expose positions while the one in this branch does. I started adding a new 
 Positions class which users can pull from a scorer, to prevent unnecessary 
 positions enums I added ScorerContext#needsPositions and eventually 
 Scorere#needsPayloads to create the corresponding enum on demand. Yet, 
 currently only TermQuery / TermScorer implements this API and other simply 
 return null instead. 
 To show that the API really works and our BulkPostings work fine too with 
 positions I cut over TermSpanQuery to use a TermScorer under the hood and 
 nuked TermSpans entirely. A nice sideeffect of this was that the Position 
 BulkReading implementation got some exercise which now :) work all with 
 positions while Payloads for bulkreading are kind of experimental in the 
 patch and those only work with Standard codec. 
 So all spans now work on top of TermScorer ( I truly hate spans since today ) 
 including the ones that need Payloads (StandardCodec ONLY)!!  I didn't bother 
 to implement the other codecs yet since I want to get feedback on the API and 
 on this first cut before I go one with it. I will upload the corresponding 
 patch in a minute. 
 I also had to cut over SpanQuery.getSpans(IR) to 
 SpanQuery.getSpans(AtomicReaderContext) which I should probably do on trunk 
 first but after that pain today I need a break first :).
 The patch passes all core tests 
 (org.apache.lucene.search.highlight.HighlighterTest still fails but I didn't 
 look into the MemoryIndex BulkPostings API yet)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-fcs-b132) - Build # 9804 - Still Failing!

2014-03-15 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9804/
Java: 32bit/jdk1.8.0-fcs-b132 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch

Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 
127.0.0.1:58601 within 45000 ms

Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: 
Could not connect to ZooKeeper 127.0.0.1:58601 within 45000 ms
at 
__randomizedtesting.SeedInfo.seed([2C01501183016211:ADE7DE09F45E022D]:0)
at 
org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:150)
at 
org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:101)
at 
org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:91)
at 
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
at 
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
at 
org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at

[jira] [Commented] (SOLR-5488) Fix up test failures for Analytics Component


[ 
https://issues.apache.org/jira/browse/SOLR-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936313#comment-13936313
 ] 

Erick Erickson commented on SOLR-5488:
--

OK, maybe we're on to something, ExpressionTest (run with a bunch of 
iterations) failed with a very similar message to FieldFacetTest.

FWIW

 Fix up test failures for Analytics Component
 

 Key: SOLR-5488
 URL: https://issues.apache.org/jira/browse/SOLR-5488
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.7, 5.0
Reporter: Erick Erickson
Assignee: Erick Erickson
 Attachments: SOLR-5488.patch, SOLR-5488.patch, SOLR-5488.patch, 
 SOLR-5488.patch, SOLR-5488.patch, SOLR-5488.patch, SOLR-5488.patch, eoe.errors


 The analytics component has a few test failures, perhaps 
 environment-dependent. This is just to collect the test fixes in one place 
 for convenience when we merge back into 4.x



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_51) - Build # 9800 - Failure!

2014-03-15 Thread Uwe Schindler

No IO issues and it runs on SSD. Machine is also stable and has no SATA 
timeouts or similar stuff.

It is just a 3 year old server CPU and its running a Vbox VM in parallel.

Uwe

On 15. März 2014 21:31:10 MEZ, Mark Miller markrmil...@gmail.com wrote:
Hmm…only interesting logging I see is this:

57473 T32 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in
SyncThread:0 took 50531ms which will adversely effect operation
latency. See the ZooKeeper troubleshooting guide
I wonder if that means that if i boost the connect timeout from 45 to
60 seconds, it will pass.
Perhaps this machine has some IO issues?

-- 
Mark Miller
about.me/markrmiller

On March 15, 2014 at 9:23:25 AM, Policeman Jenkins Server
(jenk...@thetaphi.de) wrote:

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9800/  
Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC  

1 tests failed.  
REGRESSION:
org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
 

Error Message:  
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper
127.0.0.1:44565 within 45000 ms  

Stack Trace:  
org.apache.solr.common.SolrException:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper
127.0.0.1:44565 within 45000 ms  
at
__randomizedtesting.SeedInfo.seed([D09CC97019C4AF45:517A47686E9BCF79]:0)
 
at
org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:150)
 
at
org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:101)
 
at
org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:91) 

at
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
 
at
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
 
at
org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
 
at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:201)
 
at
org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:78)
 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 
at java.lang.reflect.Method.invoke(Method.java:606)  
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
 
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:860)
 
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
 
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
 
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
 
at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
 
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
 
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
 
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
 
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
 
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
 
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
 
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
 
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
 
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
 
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
 
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
 
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
 
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
 
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
 
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
 
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
 
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
 
at

[jira] [Updated] (LUCENE-3758) Allow the ComplexPhraseQueryParser to search order or un-order proximity queries.


 [ 
https://issues.apache.org/jira/browse/LUCENE-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Arslan updated LUCENE-3758:
-

Attachment: LUCENE-3758.patch

patch for trunk (revision 1577942)

 Allow the ComplexPhraseQueryParser to search order or un-order proximity 
 queries.
 -

 Key: LUCENE-3758
 URL: https://issues.apache.org/jira/browse/LUCENE-3758
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/queryparser
Affects Versions: 4.0-ALPHA
Reporter: Tomás Fernández Löbbe
Assignee: Erick Erickson
Priority: Minor
 Fix For: 4.7

 Attachments: LUCENE-3758.patch, LUCENE-3758.patch


 The ComplexPhraseQueryParser use SpanNearQuery, but always set the inOrder 
 value hardcoded to true. This could be configurable.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5866) UpdateShardHandler needs to use the system default scheme registry to properly handle https via javax.net.ssl.* properties

Steve Davids created SOLR-5866:
--

 Summary: UpdateShardHandler needs to use the system default scheme 
registry to properly handle https via javax.net.ssl.* properties
 Key: SOLR-5866
 URL: https://issues.apache.org/jira/browse/SOLR-5866
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.7
Reporter: Steve Davids
 Fix For: 4.8


The UpdateShardHandler configures it's own PoolingClientConnectionManager which 
*doesn't* use the system default scheme registry factory which interrogates the 
javax.net.ssl.* system properties to wire up the https scheme into HttpClient. 
To ease configuration the system default registry should be used.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5866) UpdateShardHandler needs to use the system default scheme registry to properly handle https via javax.net.ssl.* properties


 [ 
https://issues.apache.org/jira/browse/SOLR-5866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Davids updated SOLR-5866:
---

Attachment: SOLR-5866.patch

Attached the trivial patch.

 UpdateShardHandler needs to use the system default scheme registry to 
 properly handle https via javax.net.ssl.* properties
 --

 Key: SOLR-5866
 URL: https://issues.apache.org/jira/browse/SOLR-5866
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.7
Reporter: Steve Davids
 Fix For: 4.8

 Attachments: SOLR-5866.patch


 The UpdateShardHandler configures it's own PoolingClientConnectionManager 
 which *doesn't* use the system default scheme registry factory which 
 interrogates the javax.net.ssl.* system properties to wire up the https 
 scheme into HttpClient. To ease configuration the system default registry 
 should be used.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5867) OverseerCollectionProcessor isn't properly generating https urls in some cases

Steve Davids created SOLR-5867:
--

 Summary: OverseerCollectionProcessor isn't properly generating 
https urls in some cases
 Key: SOLR-5867
 URL: https://issues.apache.org/jira/browse/SOLR-5867
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.7
Reporter: Steve Davids
 Fix For: 4.8


All URLs should generated using a call out to the zk state reader:
{code}
zkStateReader.getBaseUrlForNodeName(nodeName);
{code}

This is because the url scheme is stored in the clusterprops.json file and is 
necessary to properly build the correct URL to propagate the request. Please 
note that if the base_url is available, that should be used since it does have 
the properly built schemed url without the need to check zk.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5867) OverseerCollectionProcessor isn't properly generating https urls in some cases


 [ 
https://issues.apache.org/jira/browse/SOLR-5867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Davids updated SOLR-5867:
---

Attachment: SOLR-5867.patch

Attached patch.

 OverseerCollectionProcessor isn't properly generating https urls in some cases
 --

 Key: SOLR-5867
 URL: https://issues.apache.org/jira/browse/SOLR-5867
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.7
Reporter: Steve Davids
 Fix For: 4.8

 Attachments: SOLR-5867.patch


 All URLs should generated using a call out to the zk state reader:
 {code}
 zkStateReader.getBaseUrlForNodeName(nodeName);
 {code}
 This is because the url scheme is stored in the clusterprops.json file and is 
 necessary to properly build the correct URL to propagate the request. Please 
 note that if the base_url is available, that should be used since it does 
 have the properly built schemed url without the need to check zk.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks


[ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936377#comment-13936377
 ] 

Steve Davids commented on SOLR-5477:


You should drop the unnecessary assignment:
{code}
String replica = zkStateReader.getBaseUrlForNodeName(nodeName);
{code}

on line 1829, making an unnecessary call out to zk for a value that isn't being 
used.

 Async execution of OverseerCollectionProcessor tasks
 

 Key: SOLR-5477
 URL: https://issues.apache.org/jira/browse/SOLR-5477
 Project: Solr
  Issue Type: Sub-task
  Components: SolrCloud
Reporter: Noble Paul
Assignee: Anshum Gupta
 Attachments: SOLR-5477-CoreAdminStatus.patch, 
 SOLR-5477-updated.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.urlschemefix.patch


 Typical collection admin commands are long running and it is very common to 
 have the requests get timed out.  It is more of a problem if the cluster is 
 very large.Add an option to run these commands asynchronously
 add an extra param async=true for all collection commands
 the task is written to ZK and the caller is returned a task id. 
 as separate collection admin command will be added to poll the status of the 
 task
 command=statusid=7657668909
 if id is not passed all running async tasks should be listed
 A separate queue is created to store in-process tasks . After the tasks are 
 completed the queue entry is removed. OverSeerColectionProcessor will perform 
 these tasks in multiple threads



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5868) HttpClient should be configured to use ALLOW_ALL_HOSTNAME hostname verifier to simplify SSL setup

2014-03-15 Thread ASF subversion and git services (JIRA)

Steve Davids created SOLR-5868:
--

 Summary: HttpClient should be configured to use ALLOW_ALL_HOSTNAME 
hostname verifier to simplify SSL setup
 Key: SOLR-5868
 URL: https://issues.apache.org/jira/browse/SOLR-5868
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.7
Reporter: Steve Davids
 Fix For: 4.8


The default HttpClient hostname verifier is the 
BROWSER_COMPATIBLE_HOSTNAME_VERIFIER which verifies the hostname that is being 
connected to matches the hostname presented within the certificate. This is 
meant to protect clients that are making external requests out across the 
internet, but requests within the the SOLR cluster should be trusted and can be 
relaxed to simplify the SSL/certificate setup process.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks


[ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936380#comment-13936380
 ] 

ASF subversion and git services commented on SOLR-5477:
---

Commit 1577965 from [~anshumg] in branch 'dev/trunk'
[ https://svn.apache.org/r1577965 ]

SOLR-5477: Removing an unwanted call to zk

 Async execution of OverseerCollectionProcessor tasks
 

 Key: SOLR-5477
 URL: https://issues.apache.org/jira/browse/SOLR-5477
 Project: Solr
  Issue Type: Sub-task
  Components: SolrCloud
Reporter: Noble Paul
Assignee: Anshum Gupta
 Attachments: SOLR-5477-CoreAdminStatus.patch, 
 SOLR-5477-updated.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.patch, SOLR-5477.patch, SOLR-5477.patch, 
 SOLR-5477.urlschemefix.patch


 Typical collection admin commands are long running and it is very common to 
 have the requests get timed out.  It is more of a problem if the cluster is 
 very large.Add an option to run these commands asynchronously
 add an extra param async=true for all collection commands
 the task is written to ZK and the caller is returned a task id. 
 as separate collection admin command will be added to poll the status of the 
 task
 command=statusid=7657668909
 if id is not passed all running async tasks should be listed
 A separate queue is created to store in-process tasks . After the tasks are 
 completed the queue entry is removed. OverSeerColectionProcessor will perform 
 these tasks in multiple threads



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5868) HttpClient should be configured to use ALLOW_ALL_HOSTNAME hostname verifier to simplify SSL setup