[jira] [Commented] (SOLR-8088) Distributed grouping seems to require docValues in 5.x, didn't in 4.x

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14905923#comment-14905923
 ] 

Shawn Heisey commented on SOLR-8088:


I'm not sure whether it's the same problem.  It's not a numeric field, it's a 
TextField.  I'm also not doing facets, just grouping.

> Distributed grouping seems to require docValues in 5.x, didn't in 4.x
> -
>
> Key: SOLR-8088
> URL: https://issues.apache.org/jira/browse/SOLR-8088
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.2.1
>Reporter: Shawn Heisey
>
> I have a field in my index that is lowercased after the KeywordTokenizer.  I 
> wish to do grouping on this field.  It is a distributed index.
> This works fine in Solr 4.9.1 running on Java 8.
> When I try the distributed grouping request (with the same schema) on Solr 
> 5.2.1, it fails, with this exception:
> {code}
> java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for 
> field 'ip' (expected=SORTED). Use UninvertingReader or index with docvalues.
> {code}
> If I make the same request directly to one of the shards on 5.2.1, it works.  
> If I create a copyField to a field using StrField with docValues, the 
> distributed request works ... but then I lose the lowercasing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6813) OfflineSorter.sort isn't thread-safe

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-6813:
---
Attachment: LUCENE-6813.patch

Patch, I think it's ready.

I added a thread safety test, but it passes even before my fix.  I'll
update the issue title ... sorry for the false blame!  Now I don't
know why BKD was angry before ... but it seems happy now.

I also fixed test infra to set the default temp dir used by
OfflineSorter, so we get MockFS checking, and (YAY!) it caught the
offending place in RangeTreeWriter where it was leaking a file handle!
This should fix the windows failures (finally!).


> OfflineSorter.sort isn't thread-safe
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-6813:
---
Summary: OfflineSorter.sort shouldn't remove the output Path up front  
(was: OfflineSorter.sort isn't thread-safe)

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_60) - Build # 14285 - Still Failing!

[Michael McCandless]

Thank you Anshum.

Mike McCandless

http://blog.mikemccandless.com


On Wed, Sep 23, 2015 at 8:00 PM, Anshum Gupta  wrote:
> I've fixed this on both trunk and 5x.
>
> On Wed, Sep 23, 2015 at 4:37 PM, Policeman Jenkins Server
>  wrote:
>>
>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/14285/
>> Java: 32bit/jdk1.8.0_60 -client -XX:+UseParallelGC
>>
>> All tests passed
>>
>> Build Log:
>> [...truncated 54728 lines...]
>> BUILD FAILED
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:775: The
>> following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:655: The
>> following error occurred while executing this line:
>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:642: Source
>> checkout is dirty after running tests!!! Offending files:
>> * ./solr/licenses/spatial4j-0.4.1.jar.sha1
>>
>> Total time: 60 minutes 37 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> [WARNINGS] Skipping publisher since build result is FAILURE
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_60) - Build # 14285 - Still Failing!

[Michael McCandless]

I think (unfortunately) "ant precommit" maybe doesn't catch this?

It means after committing you should try to be vigilant on the build
failures to see if they're yours :)

"commit and run" is dangerous.

Mike McCandless

http://blog.mikemccandless.com


On Wed, Sep 23, 2015 at 8:49 PM, david.w.smi...@gmail.com
 wrote:
> Thanks! I'm not sure why I didn't find this in my testing.
> On Wed, Sep 23, 2015 at 8:00 PM Anshum Gupta  wrote:
>>
>> I've fixed this on both trunk and 5x.
>>
>> On Wed, Sep 23, 2015 at 4:37 PM, Policeman Jenkins Server
>>  wrote:
>>>
>>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/14285/
>>> Java: 32bit/jdk1.8.0_60 -client -XX:+UseParallelGC
>>>
>>> All tests passed
>>>
>>> Build Log:
>>> [...truncated 54728 lines...]
>>> BUILD FAILED
>>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:775: The
>>> following error occurred while executing this line:
>>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:655: The
>>> following error occurred while executing this line:
>>> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:642: Source
>>> checkout is dirty after running tests!!! Offending files:
>>> * ./solr/licenses/spatial4j-0.4.1.jar.sha1
>>>
>>> Total time: 60 minutes 37 seconds
>>> Build step 'Invoke Ant' marked build as failure
>>> Archiving artifacts
>>> [WARNINGS] Skipping publisher since build result is FAILURE
>>> Recording test results
>>> Email was triggered for: Failure - Any
>>> Sending email for trigger: Failure - Any
>>>
>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906151#comment-14906151
 ] 

Dawid Weiss commented on LUCENE-6813:
-

I'd have to look into it. Filesystem providers have paths conversion utilities 
so a "mockfs" path in the property should be doable... I've been burnt by 
static initializers wish side effects (exceptions) so many time that it always 
displays a red flag for me.

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Alex Chow (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906171#comment-14906171
 ] 

Alex Chow edited comment on LUCENE-6814 at 9/24/15 11:05 AM:
-

Thanks for taking a look at this, [~mikemccand]!

It'd be great if this can get added to 4.10 so elasticsearch 1.x can pull it in 
too.

(Never mind the extra stuff that was here. It all made sense when I tried 
looking through it)


was (Author: chow):
Thanks for taking a look at this, [~mikemccand]!

It'd be great if this can get added to 4.10 so elasticsearch 1.x can pull it in 
too.

I'm curious why there shouldn't there be some trimming in `end()` as well? Or 
is a `TokenStream` meant to be used only once (no multiple `reset()`, 
`incrementToken()`, `end()` on the same `TokenStream`)?

Elasticsearch seems to never reinstantiate Tokenizers and just reuses them for 
each field in an index, though I may be wrong. Or elasticsearch is using 
TokenStream the wrong way?

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906263#comment-14906263
 ] 

Robert Muir commented on LUCENE-6813:
-

Can we please not do this? it was intentional that we dont change the default 
provider. At the very least we have test classes that do not want the wrapping: 
and we have annotations to allow them to disable it.

All we need is a setter here.

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Noble Paul]

I'm about to send the announcement mail.

lease take a look at http://lucene.us.apache.org to see if I have
missed anything

--Noble

On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
> Vote passed . I shall start the release process
>
> On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  wrote:
>> +1 for both Java7 and Java8 !
>>
>> SUCCESS! [1:12:33.649607]
>>
>> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  wrote:
>>>
>>> The artifacts can be downloaded from:
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>>
>>> You can run the smoke tester directly with this command:
>>> python3 -u dev-tools/scripts/smokeTestRelease.py
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>>
>>>
>>> +1
>>> SUCCESS! [0:50:32.819792]
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Anshum Gupta
>
>
>
> --
> -
> Noble Paul



-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Shalin Shekhar Mangar]

Hi Noble,

Have you created release notes for the 5.3.1 release? Can you share
them with us?

On Thu, Sep 24, 2015 at 4:58 PM, Noble Paul  wrote:
> I'm about to send the announcement mail.
>
> lease take a look at http://lucene.us.apache.org to see if I have
> missed anything
>
> --Noble
>
> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
>> Vote passed . I shall start the release process
>>
>> On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
>> wrote:
>>> +1 for both Java7 and Java8 !
>>>
>>> SUCCESS! [1:12:33.649607]
>>>
>>> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  wrote:

 The artifacts can be downloaded from:

 https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/

 You can run the smoke tester directly with this command:
 python3 -u dev-tools/scripts/smokeTestRelease.py

 https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/


 +1
 SUCCESS! [0:50:32.819792]


 --
 -
 Noble Paul

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>>>
>>>
>>>
>>> --
>>> Anshum Gupta
>>
>>
>>
>> --
>> -
>> Noble Paul
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



-- 
Regards,
Shalin Shekhar Mangar.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906257#comment-14906257
 ] 

Dawid Weiss commented on LUCENE-6813:
-

It'd have to be a global override of 
{{java.nio.file.spi.DefaultFileSystemProvider}} which would wrap all of 
{{file:///}} URIs. We could then install arbitrary filesystem operation 
interceptors or tracking. The problem is, as far as I recall, that this SPI is 
not frequently used and [~rcmuir] mentioned to me once that it's not actually 
implementable... 

It'd be an interesting experiment to try to do leak checking via the default 
system provider. 

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Alex Chow (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906171#comment-14906171
 ] 

Alex Chow edited comment on LUCENE-6814 at 9/24/15 10:33 AM:
-

Thanks for taking a look at this, [~mikemccand]!

It'd be great if this can get added to 4.10 so elasticsearch 1.x can pull it in 
too.

I'm curious why there shouldn't there be some trimming in `end()` as well? Or 
is a `TokenStream` meant to be used only once (no multiple `reset()`, 
`incrementToken()`, `end()` on the same `TokenStream`)?

Elasticsearch seems to never reinstantiate Tokenizers and just reuses them for 
each field in an index, though I may be wrong. Or elasticsearch is using 
TokenStream the wrong way?


was (Author: chow):
Thanks for taking a look at this, [~mikemccand]!

I'm curious why there shouldn't there be some trimming in `end()` as well? Or 
is a `TokenStream` meant to be used only once (no multiple `reset()`, 
`incrementToken()`, `end()` on the same `TokenStream`)?

Elasticsearch seems to never reinstantiate Tokenizers and just reuses them for 
each field in an index, though I may be wrong. Or elasticsearch is using 
TokenStream the wrong way?

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3929) org.apache.lucene.analysis.fr.FrenchAnalyzer could introduce french accent insensitive search.

[Jon Harper (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906192#comment-14906192
 ] 

Jon Harper commented on LUCENE-3929:


Actually I can not reopen an issue. Trying to ping [~rcmuir] [~rcmuir ]

> org.apache.lucene.analysis.fr.FrenchAnalyzer could introduce french accent 
> insensitive search.
> --
>
> Key: LUCENE-3929
> URL: https://issues.apache.org/jira/browse/LUCENE-3929
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Geoffroy Schneck
>Assignee: Robert Muir
>Priority: Minor
> Fix For: 3.6, 4.0-ALPHA
>
>
> The GermanAnalyzer does the same with the Umlaut for example. Searching for 
> 'gehort' will return 'gehört' and 'gehort' .
> I expected that the FrenchAnalyzer would also return 'sécuritaires' and 
> 'securitaires' and searching for any of them, but it's not the case



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3929) org.apache.lucene.analysis.fr.FrenchAnalyzer could introduce french accent insensitive search.

[Jon Harper (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906190#comment-14906190
 ] 

Jon Harper commented on LUCENE-3929:


But what about the many words with less than 5 letters ?
Here's an example list of almost 400 words that can not be searched without the 
correct accents:
For me, this makes it unuseable. But feel free to reclose this issue if the 
current behavior is the expected one.
{noformat}
{
"à"
"abbé"
"aéra"
"aère"
"aéré"
"âgé"
"âgée"
"âgés"
"aidé"
"ailé"
"aimé"
"aîné"
"aisé"
"aléa"
"allé"
"allô"
"armé"
"axé"
"axée"
"axés"
"basé"
"bâté"
"bavé"
"béat"
"bébé"
"bée"
"béer"
"bées"
"béni"
"bêta"
"bête"
"blé"
"blés"
"boxé"
"buée"
"buté"
"çà"
"café"
"calé"
"casé"
"céda"
"cède"
"cédé"
"cène"
"cèpe"
"ciré"
"cité"
"clé"
"clés"
"clôt"
"codé"
"cône"
"côte"
"côté"
"coté"
"créa"
"crée"
"créé"
"crié"
"cubé"
"curé"
"cuvé"
"daté"
"dé"
"déc."
"deçà"
"déçu"
"défi"
"déjà"
"delà"
"déni"
"dép."
"dès"
"dés"
"dîné"
"dôme"
"dopé"
"doré"
"dosé"
"doté"
"doué"
"dupé"
"duré"
"ébat"
"écho"
"échu"
"écot"
"écru"
"écu"
"écus"
"édam"
"édit"
"égal"
"élan"
"élis"
"élit"
"élu"
"élue"
"élus"
"élut"
"ème"
"émet"
"émir"
"émis"
"émit"
"émoi"
"ému"
"émue"
"émus"
"émut"
"épée"
"épi"
"épia"
"épie"
"épié"
"épis"
"ère"
"ères"
"erré"
"ès"
"étai"
"étal"
"état"
"étau"
"été"
"êtes"
"étés"
"être"
"étui"
"famé"
"fané"
"fée"
"fées"
"fêla"
"fêle"
"fêlé"
"féru"
"fêta"
"fête"
"fêté"
"fétu"
"fève"
"fié"
"fiée"
"fiés"
"figé"
"filé"
"fixé"
"foré"
"fumé"
"fusé"
"futé"
"gagé"
"garé"
"gâté"
"gavé"
"gazé"
"gèle"
"gelé"
"gémi"
"gêna"
"gène"
"gêne"
"gêné"
"géra"
"gère"
"géré"
"gobé"
"gré"
"grès"
"gué"
"gués"
"hâlé"
"hâté"
"héla"
"hèle"
"hélé"
"hère"
"hôte"
"hué"
"huée"
"hués"
"humé"
"idée"
"inné"
"iodé"
"jasé"
"jeté"
"joué"
"jubé"
"jugé"
"juré"
"képi"
"là"
"lacé"
"lamé"
"lavé"
"lès"
"lésa"
"lèse"
"lésé"
"lève"
"levé"
"lié"
"liée"
"liés"
"limé"
"logé"
"loué"
"luné"
"luxé"
"méat"
"mêla"
"mêle"
"mêlé"
"même"
"mène"
"mené"
"mère"
"mimé"
"miné"
"miré"
"misé"
"mité"
"mixé"
"môle"
"mué"
"muée"
"mués"
"muré"
"muté"
"nagé"
"né"
"née"
"nées"
"néon"
"nés"
"névé"
"nié"
"niée"
"niés"
"nô"
"nôs"
"noté"
"noué"
"noyé"
"nuée"
"ô"
"obéi"
"opté"
"orée"
"orné"
"osé"
"osée"
"osés"
"ôta"
"ôte"
"ôté"
"ôtée"
"ôter"
"ôtes"
"ôtés"
"ôtez"
"pâmé"
"pané"
"paré"
"pâté"
"pavé"
"payé"
"pèle"
"pelé"
"pêne"
"père"
"péri"
"pèse"
"pesé"
"pète"
"pété"
"pilé"
"pisé"
"plié"
"pôle"
"posé"
"pré"
"près"
"prés"
"prêt"
"prié"
"pué"
"racé"
"ragé"
"râlé"
"ramé"
"râpé"
"rasé"
"raté"
"rayé"
"ré"
"réel"
"réer"
"réf."
"régi"
"rêne"
"rêva"
"rêve"
"rêvé"
"ridé"
"rimé"
"rivé"
"rôda"
"rôde"
"rôdé"
"rodé"
"rôle"
"rosé"
"rôt"
"rôti"
"rôts"
"roué"
"rué"
"ruée"
"rués"
"rusé"
"saké"
"salé"
"sapé"
"scié"
"sème"
"semé"
"sève"
"sévi"
"skié"
"sucé"
"sué"
"suée"
"sués"
"talé"
"tapé"
"taré"
"tâté"
"taxé"
"tél."
"télé"
"ténu"
"tête"
"têtu"
"thé"
"thés"
"tiré"
"tôle"
"tôt"
"très"
"trié"
"tué"
"tuée"
"tués"
"typé"
"urée"
"urgé"
"usé"
"usée"
"usés"
"vécu"
"vélo"
"vêt"
"vête"
"vêts"
"vêtu"
"vexé"
"vidé"
"viré"
"visé"
"volé"
"voté"
"voué"
"zébu"
"zèle"
"zélé"
"zéro"
"zoné"
}
{noformat}

> org.apache.lucene.analysis.fr.FrenchAnalyzer could introduce french accent 
> insensitive search.
> --
>
> Key: LUCENE-3929
> URL: https://issues.apache.org/jira/browse/LUCENE-3929
> Project: Lucene - Core
>   

[jira] [Created] (LUCENE-6815) Should DisjunctionScorer advance more lazily?

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-6815:


 Summary: Should DisjunctionScorer advance more lazily?
 Key: LUCENE-6815
 URL: https://issues.apache.org/jira/browse/LUCENE-6815
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor


Today if you call DisjunctionScorer.advance(X), it will try to advance all sub 
scorers to X. However, if DisjunctionScorer is being intersected with another 
scorer (which is almost always the case as we use BooleanScorer for top-level 
disjunctions), we could stop as soon as we find one matching sub scorer, and 
only advance the remaining sub scorers when freq() or score() is called. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-6814:
---
Attachment: LUCENE-6814.patch

New patch, moving the clearing to {{close}} and adding a fun test case.

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Alex Chow (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906171#comment-14906171
 ] 

Alex Chow commented on LUCENE-6814:
---

Thanks for taking a look at this, [~mikemccand]!

I'm curious why there shouldn't there be some trimming in `end()` as well? Or 
is a `TokenStream` meant to be used only once (no multiple `reset()`, 
`incrementToken()`, `end()` on the same `TokenStream`)?

Elasticsearch seems to never reinstantiate Tokenizers and just reuses them for 
each field in an index, though I may be wrong. Or elasticsearch is using 
TokenStream the wrong way?

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3929) org.apache.lucene.analysis.fr.FrenchAnalyzer could introduce french accent insensitive search.

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906215#comment-14906215
 ] 

Robert Muir commented on LUCENE-3929:
-

The issue is one of time. If we want to change how that stemmer works or 
whatever, then we should at least measure and test that it doesnt send 
relevance down the tubes. If it is left to me to run those experiments, it will 
take some time, even though I do want to fix it. It is a lot of work, I can't 
test everyone's idea about how the analyzers should work.

> org.apache.lucene.analysis.fr.FrenchAnalyzer could introduce french accent 
> insensitive search.
> --
>
> Key: LUCENE-3929
> URL: https://issues.apache.org/jira/browse/LUCENE-3929
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Geoffroy Schneck
>Assignee: Robert Muir
>Priority: Minor
> Fix For: 3.6, 4.0-ALPHA
>
>
> The GermanAnalyzer does the same with the Umlaut for example. Searching for 
> 'gehort' will return 'gehört' and 'gehort' .
> I expected that the FrenchAnalyzer would also return 'sécuritaires' and 
> 'securitaires' and searching for any of them, but it's not the case



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906265#comment-14906265
 ] 

Robert Muir commented on LUCENE-6813:
-

Mike, your setter can also be package private. Maybe that will make it more 
digestible.

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_60) - Build # 14288 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/14288/
Java: 64bit/jdk1.8.0_60 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestMiniSolrCloudCluster

Error Message:
94 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestMiniSolrCloudCluster: 1) Thread[id=1092, 
name=coreContainerWorkExecutor-545-thread-1, state=TIMED_WAITING, 
group=TGRP-TestMiniSolrCloudCluster] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)2) Thread[id=1107, 
name=OverseerThreadFactory-572-thread-2, state=WAITING, group=Overseer 
collection creation process.] at sun.misc.Unsafe.park(Native Method)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)   
  at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)3) Thread[id=1072, 
name=zkCallback-177-thread-1, state=TIMED_WAITING, 
group=TGRP-TestMiniSolrCloudCluster] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)4) Thread[id=1026, 
name=org.eclipse.jetty.server.session.HashSessionManager@d65c875Timer, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster] at 
sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
 at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)5) Thread[id=1087, 
name=OverseerStateUpdate-94573843225575437-127.0.0.1:38509_solr-n_00, 
state=TIMED_WAITING, group=Overseer state updater.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)   
  at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
 at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:333)
 at 
org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:310)
 at 
org.apache.solr.cloud.DistributedQueue.firstChild(DistributedQueue.java:289)
 at 
org.apache.solr.cloud.DistributedQueue.firstElement(DistributedQueue.java:358)  
   at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:165) 
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:143)   
  at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:193)  
   at java.lang.Thread.run(Thread.java:745)6) 

[jira] [Comment Edited] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906062#comment-14906062
 ] 

Dawid Weiss edited comment on LUCENE-6813 at 9/24/15 8:59 AM:
--

Hmm... I don't like the move of defaultTempDir() to a static initializer -- it 
means if anything fails (property read, whatever) then you'd get an early 
exception upon class linking which can often be very confusing. I'd rather have 
a lazy return as it was before.

{code}
+// So all code using OfflineSorter (suggesters, BKD tree, 
NumericRangeTree) see MockFS goodness, e.g. catching leaked file handles:
+OfflineSorter.setDefaultTempDir(javaTempDir);
{code}

Wouldn't it be possible to set the {{java.io.tmpdir}} property to a path that 
resolves to mockfs instead? Then any paths resolved from java.io.tmpdir would 
be "wrapped" by MockFS, no matter where they originate from and without the 
(dodgy) static test-only variable substitution methods... Don't know how hard 
it'd be though.




was (Author: dweiss):
Hmm... I don't like the move of defaultTempDir() to a static initializer -- it 
means if anything fails (property read, whatever) then you'd get an early 
exception upon class linking which can be often be very confusing. I'd rather 
have a lazy return as it was before.

{code}
+// So all code using OfflineSorter (suggesters, BKD tree, 
NumericRangeTree) see MockFS goodness, e.g. catching leaked file handles:
+OfflineSorter.setDefaultTempDir(javaTempDir);
{code}

Wouldn't it be possible to set the {{java.io.tmpdir}} property to a path that 
resolves to mockfs instead? Then any paths resolved from java.io.tmpdir would 
be "wrapped" by MockFS, no matter where they originate from and without the 
(dodgy) static test-only variable substitution methods... Don't know how hard 
it'd be though.



> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906093#comment-14906093
 ] 

Michael McCandless commented on LUCENE-6813:


bq. Hmm... I don't like the move of defaultTempDir() to a static initializer 

OK ... if there is a cleaner way I would love to do that instead.

bq. Wouldn't it be possible to set the java.io.tmpdir property to a path that 
resolves to mockfs instead? 

I don't know enough about mockfs to know if this is possible :)  I thought we 
must use the {{Path}} we had set up (backed by the mock {{FileSystem}} we 
created) ... but maybe there is a way to "install" the mock filesystem 
differently?

I do think it's really useful to have {{OfflineSorter}} use our mock 
filesystems: it caught the bug in {{RangeTreeWriter}}...

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-SmokeRelease-trunk - Build # 284 - Failure

[Anshum Gupta]

I think this is because Noble's working on the 5.3.1 release and is yet to
generate the back compat indexes, which is usually the last step.

On Wed, Sep 23, 2015 at 9:56 PM, Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-trunk/284/
>
> No tests ran.
>
> Build Log:
> [...truncated 52498 lines...]
> prepare-release-no-sign:
> [mkdir] Created dir:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/smokeTestRelease/dist
>  [copy] Copying 461 files to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/smokeTestRelease/dist/lucene
>  [copy] Copying 245 files to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/smokeTestRelease/dist/solr
>[smoker] Java 1.8
> JAVA_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
>[smoker] NOTE: output encoding is UTF-8
>[smoker]
>[smoker] Load release URL
> "file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/smokeTestRelease/dist/"...
>[smoker]
>[smoker] Test Lucene...
>[smoker]   test basics...
>[smoker]   get KEYS
>[smoker] 0.1 MB in 0.01 sec (11.4 MB/sec)
>[smoker]   check changes HTML...
>[smoker]   download lucene-6.0.0-src.tgz...
>[smoker] 28.2 MB in 0.04 sec (748.5 MB/sec)
>[smoker] verify md5/sha1 digests
>[smoker]   download lucene-6.0.0.tgz...
>[smoker] 65.2 MB in 0.09 sec (700.8 MB/sec)
>[smoker] verify md5/sha1 digests
>[smoker]   download lucene-6.0.0.zip...
>[smoker] 75.7 MB in 0.11 sec (690.4 MB/sec)
>[smoker] verify md5/sha1 digests
>[smoker]   unpack lucene-6.0.0.tgz...
>[smoker] verify JAR metadata/identity/no javax.* or java.*
> classes...
>[smoker] test demo with 1.8...
>[smoker]   got 5904 hits for query "lucene"
>[smoker] checkindex with 1.8...
>[smoker] check Lucene's javadoc JAR
>[smoker]   unpack lucene-6.0.0.zip...
>[smoker] verify JAR metadata/identity/no javax.* or java.*
> classes...
>[smoker] test demo with 1.8...
>[smoker]   got 5904 hits for query "lucene"
>[smoker] checkindex with 1.8...
>[smoker] check Lucene's javadoc JAR
>[smoker]   unpack lucene-6.0.0-src.tgz...
>[smoker] make sure no JARs/WARs in src dist...
>[smoker] run "ant validate"
>[smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
>[smoker] test demo with 1.8...
>[smoker]   got 209 hits for query "lucene"
>[smoker] checkindex with 1.8...
>[smoker] generate javadocs w/ Java 8...
>[smoker]
>[smoker] Crawl/parse...
>[smoker]
>[smoker] Verify...
>[smoker]   confirm all releases have coverage in
> TestBackwardsCompatibility
>[smoker] find all past Lucene releases...
>[smoker] run TestBackwardsCompatibility..
>[smoker] Releases that don't seem to be tested:
>[smoker]   5.3.1
>[smoker] Traceback (most recent call last):
>[smoker]   File
> "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
> line 1416, in 
>[smoker] main()
>[smoker]   File
> "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
> line 1361, in main
>[smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir,
> c.is_signed, ' '.join(c.test_args))
>[smoker]   File
> "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
> line 1399, in smokeTest
>[smoker] unpackAndVerify(java, 'lucene', tmpDir,
> 'lucene-%s-src.tgz' % version, svnRevision, version, testArgs, baseURL)
>[smoker]   File
> "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
> line 583, in unpackAndVerify
>[smoker] verifyUnpacked(java, project, artifact, unpackPath,
> svnRevision, version, testArgs, tmpDir, baseURL)
>[smoker]   File
> "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
> line 728, in verifyUnpacked
>[smoker] confirmAllReleasesAreTestedForBackCompat(unpackPath)
>[smoker]   File
> "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
> line 1354, in confirmAllReleasesAreTestedForBackCompat
>[smoker] raise RuntimeError('some releases are not tested by
> TestBackwardsCompatibility?')
>[smoker] RuntimeError: some releases are not tested by
> TestBackwardsCompatibility?
>
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-trunk/build.xml:527:
> exec returned: 1
>
> Total time: 40 minutes 35 seconds
> Build step 'Invoke Ant' marked build as failure
> Email was triggered for: Failure
> 

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906061#comment-14906061
 ] 

Uwe Schindler commented on LUCENE-6814:
---

It should call something like setLength(MIN_SIZE).trimToSize() on close().

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906062#comment-14906062
 ] 

Dawid Weiss commented on LUCENE-6813:
-

Hmm... I don't like the move of defaultTempDir() to a static initializer -- it 
means if anything fails (property read, whatever) then you'd get an early 
exception upon class linking which can be often be very confusing. I'd rather 
have a lazy return as it was before.

{code}
+// So all code using OfflineSorter (suggesters, BKD tree, 
NumericRangeTree) see MockFS goodness, e.g. catching leaked file handles:
+OfflineSorter.setDefaultTempDir(javaTempDir);
{code}

Wouldn't it be possible to set the {{java.io.tmpdir}} property to a path that 
resolves to mockfs instead? Then any paths resolved from java.io.tmpdir would 
be "wrapped" by MockFS, no matter where they originate from and without the 
(dodgy) static test-only variable substitution methods... Don't know how hard 
it'd be though.



> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-6814:
---
Attachment: LUCENE-6814.patch

Patch, setting length and trimming in {{end()}} 

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906080#comment-14906080
 ] 

Michael McCandless commented on LUCENE-6814:


bq. makes no sense. close() is correct - see documentation

OK right, sorry :)

{{close()}} is in fact called each time we analyze a String/Reader ... I always 
get confused by this :)

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8089) Support query parsers being able to set enablePositionIncrements

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-8089:

Attachment: SOLR-8089.patch

Patch with a test case of EDismax . The tests are failing for queries with 
enablePositionIncrements=false

For the following two failing queries here is how the positions are getting put

q="the sky is blue"
sky at pos=0
blue at pos=1

q="sky blue"
sky at pos=0
blue at pos=1

The relevant code from QueryBuilder#analyzePhrase

{code}
while (stream.incrementToken()) {
  if (enablePositionIncrements) {
position += posIncrAtt.getPositionIncrement();
  } else {
position += 1;
  }
  builder.add(new Term(field, termAtt.getBytesRef()), position);
}
{code}

So I can see why they are not matching the document. 

What am I missing here ? 

> Support query parsers being able to set enablePositionIncrements
> 
>
> Key: SOLR-8089
> URL: https://issues.apache.org/jira/browse/SOLR-8089
> Project: Solr
>  Issue Type: Bug
>Reporter: Varun Thacker
> Attachments: SOLR-8089.patch
>
>
> In 5.0 LUCENE-4963 disallowed setting enablePositionIncrements=false on any 
> token filter. The idea being no filter should change the token stream in a 
> way that could mess up preceding or following analysis components
> So if a user wants to be able to have PhraseQueries that match across 
> stopwords they cannot unless the parser is configured to not take position 
> increments into account when generating phrase queries .
> This is documented in the "Token Position Increments" section here : 
> https://lucene.apache.org/core/5_3_0/core/org/apache/lucene/analysis/package-summary.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8089) Support query parsers being able to set enablePositionIncrements

[Varun Thacker (JIRA)]

Varun Thacker created SOLR-8089:
---

 Summary: Support query parsers being able to set 
enablePositionIncrements
 Key: SOLR-8089
 URL: https://issues.apache.org/jira/browse/SOLR-8089
 Project: Solr
  Issue Type: Bug
Reporter: Varun Thacker


In 5.0 LUCENE-4963 disallowed setting enablePositionIncrements=false on any 
token filter. The idea being no filter should change the token stream in a way 
that could mess up preceding or following analysis components

So if a user wants to be able to have PhraseQueries that match across stopwords 
they cannot unless the parser is configured to not take position increments 
into account when generating phrase queries .

This is documented in the "Token Position Increments" section here : 
https://lucene.apache.org/core/5_3_0/core/org/apache/lucene/analysis/package-summary.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906076#comment-14906076
 ] 

Uwe Schindler commented on LUCENE-6814:
---

makes no sense. close() is correct - see documentation :-)

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Michael McCandless (JIRA)]

Michael McCandless created LUCENE-6814:
--

 Summary: PatternTokenizer should free heap after it's done
 Key: LUCENE-6814
 URL: https://issues.apache.org/jira/browse/LUCENE-6814
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: Trunk, 5.4


Caught by Alex Chow in this Elasticsearch issue: 
https://github.com/elastic/elasticsearch/issues/13721

Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free its 
heap usage after tokenizing is done.  We can either stop reusing, or ask it to 
{{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906065#comment-14906065
 ] 

Michael McCandless commented on LUCENE-6814:


bq. on close().

Maybe on {{end()}} instead?

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 965 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/965/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15224, name=collection1, 
state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=15224, name=collection1, state=RUNNABLE, 
group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:46029: Could not find collection : 
awholynewstresscollection_collection1_5
at __randomizedtesting.SeedInfo.seed([909429C3CC447CA5]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=105154, name=collection5, 
state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=105154, name=collection5, state=RUNNABLE, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:57765/gn: Could not find collection : 
awholynewstresscollection_collection5_0
at __randomizedtesting.SeedInfo.seed([909429C3CC447CA5]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10683 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_909429C3CC447CA5-001/init-core-data-001
   [junit4]   2> 1744228 INFO  
(SUITE-CollectionsAPIDistributedZkTest-seed#[909429C3CC447CA5]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1744228 INFO  
(SUITE-CollectionsAPIDistributedZkTest-seed#[909429C3CC447CA5]-worker) [] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1744232 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1744233 INFO  (Thread-8564) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1744233 INFO  (Thread-8564) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1744333 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [] 
o.a.s.c.ZkTestServer start zk server on port:40896
   [junit4]   2> 1744333 INFO  
(TEST-CollectionsAPIDistributedZkTest.test-seed#[909429C3CC447CA5]) [] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1744333 INFO  

[jira] [Commented] (SOLR-8088) Distributed grouping seems to require docValues in 5.x, didn't in 4.x

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906293#comment-14906293
 ] 

Yonik Seeley commented on SOLR-8088:


Yet another problem caused by LUCENE-5666


> Distributed grouping seems to require docValues in 5.x, didn't in 4.x
> -
>
> Key: SOLR-8088
> URL: https://issues.apache.org/jira/browse/SOLR-8088
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.2.1
>Reporter: Shawn Heisey
>
> I have a field in my index that is lowercased after the KeywordTokenizer.  I 
> wish to do grouping on this field.  It is a distributed index.
> This works fine in Solr 4.9.1 running on Java 8.
> When I try the distributed grouping request (with the same schema) on Solr 
> 5.2.1, it fails, with this exception:
> {code}
> java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for 
> field 'ip' (expected=SORTED). Use UninvertingReader or index with docvalues.
> {code}
> If I make the same request directly to one of the shards on 5.2.1, it works.  
> If I create a copyField to a field using StrField with docValues, the 
> distributed request works ... but then I lose the lowercasing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Alexandre Rafalovitch]

I know SOLR-8073 has not made it into 5.3.1, but should we document a
workaround somewhere? (tl;dr: on Windows, startup fails when using
relative path to solr home; use absolute path).

Regards,
   Alex.

Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/


On 24 September 2015 at 08:57, Noble Paul  wrote:
> Release notes are here . Please feel free to edit them
>
> https://wiki.apache.org/solr/ReleaseNote531
>
> https://wiki.apache.org/lucene-java/ReleaseNote531
>
> On Thu, Sep 24, 2015 at 5:15 PM, Shalin Shekhar Mangar
>  wrote:
>> Hi Noble,
>>
>> Have you created release notes for the 5.3.1 release? Can you share
>> them with us?
>>
>> On Thu, Sep 24, 2015 at 4:58 PM, Noble Paul  wrote:
>>> I'm about to send the announcement mail.
>>>
>>> lease take a look at http://lucene.us.apache.org to see if I have
>>> missed anything
>>>
>>> --Noble
>>>
>>> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
 Vote passed . I shall start the release process

 On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
 wrote:
> +1 for both Java7 and Java8 !
>
> SUCCESS! [1:12:33.649607]
>
> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  wrote:
>>
>> The artifacts can be downloaded from:
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>
>> You can run the smoke tester directly with this command:
>> python3 -u dev-tools/scripts/smokeTestRelease.py
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>
>>
>> +1
>> SUCCESS! [0:50:32.819792]
>>
>>
>> --
>> -
>> Noble Paul
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> Anshum Gupta



 --
 -
 Noble Paul
>>>
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Noble Paul]

Do we have a known issues section where we can add this

On Thu, Sep 24, 2015 at 7:23 PM, Alexandre Rafalovitch
 wrote:
> I know SOLR-8073 has not made it into 5.3.1, but should we document a
> workaround somewhere? (tl;dr: on Windows, startup fails when using
> relative path to solr home; use absolute path).
>
> Regards,
>Alex.
> 
> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
> http://www.solr-start.com/
>
>
> On 24 September 2015 at 08:57, Noble Paul  wrote:
>> Release notes are here . Please feel free to edit them
>>
>> https://wiki.apache.org/solr/ReleaseNote531
>>
>> https://wiki.apache.org/lucene-java/ReleaseNote531
>>
>> On Thu, Sep 24, 2015 at 5:15 PM, Shalin Shekhar Mangar
>>  wrote:
>>> Hi Noble,
>>>
>>> Have you created release notes for the 5.3.1 release? Can you share
>>> them with us?
>>>
>>> On Thu, Sep 24, 2015 at 4:58 PM, Noble Paul  wrote:
 I'm about to send the announcement mail.

 lease take a look at http://lucene.us.apache.org to see if I have
 missed anything

 --Noble

 On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
> Vote passed . I shall start the release process
>
> On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
> wrote:
>> +1 for both Java7 and Java8 !
>>
>> SUCCESS! [1:12:33.649607]
>>
>> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  wrote:
>>>
>>> The artifacts can be downloaded from:
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>>
>>> You can run the smoke tester directly with this command:
>>> python3 -u dev-tools/scripts/smokeTestRelease.py
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>>
>>>
>>> +1
>>> SUCCESS! [0:50:32.819792]
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Anshum Gupta
>
>
>
> --
> -
> Noble Paul



 --
 -
 Noble Paul

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>>>
>>>
>>>
>>> --
>>> Regards,
>>> Shalin Shekhar Mangar.
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>>
>>
>> --
>> -
>> Noble Paul
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906267#comment-14906267
 ] 

Dawid Weiss commented on LUCENE-6813:
-

Yeah... I noticed that while looking at the code. These are not really 
filesystem providers, they're default filesystem decorators...

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7495) Unexpected docvalues type NUMERIC when grouping by a int facet

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906294#comment-14906294
 ] 

Yonik Seeley commented on SOLR-7495:


These bugs started with LUCENE-5666

> Unexpected docvalues type NUMERIC when grouping by a int facet
> --
>
> Key: SOLR-7495
> URL: https://issues.apache.org/jira/browse/SOLR-7495
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.3
>Reporter: Fabio Batista da Silva
> Attachments: SOLR-7495.patch
>
>
> Hey All,
> After upgrading from solr 4.10 to 5.1 with solr could
> I'm getting a IllegalStateException when i try to facet a int field.
> IllegalStateException: unexpected docvalues type NUMERIC for field 'year' 
> (expected=SORTED). Use UninvertingReader or index with docvalues.
> schema.xml
> {code}
> 
> 
> 
> 
> 
> 
>  multiValued="false" required="true"/>
>  multiValued="false" required="true"/>
> 
> 
>  stored="true"/>
> 
> 
> 
>  />
>  sortMissingLast="true"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  precisionStep="0" positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="100">
> 
> 
>  words="stopwords.txt" />
> 
>  maxGramSize="15"/>
> 
> 
> 
>  words="stopwords.txt" />
>  synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> 
> 
> 
>  positionIncrementGap="100">
> 
> 
>  words="stopwords.txt" />
> 
>  maxGramSize="15"/>
> 
> 
> 
>  words="stopwords.txt" />
>  synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> 
> 
> 
>  class="solr.SpatialRecursivePrefixTreeFieldType" geo="true" 
> distErrPct="0.025" maxDistErr="0.09" units="degrees" />
> 
> id
> name
> 
> 
> {code}
> query :
> {code}
> http://solr.dev:8983/solr/my_collection/select?wt=json=id=index_type:foobar=true=year_make_model=true=true=year
> {code}
> Exception :
> {code}
> ull:org.apache.solr.common.SolrException: Exception during facet.field: year
> at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:627)
> at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:612)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:566)
> at 
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:637)
> at 
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:280)
> at 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:106)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:222)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at 
> 

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Shalin Shekhar Mangar]

I fixed the release date for Solr 5.3.1 and added SOLR-8058 to the
list of major fixes.

+1 to both Lucene and Solr release notes.

On Thu, Sep 24, 2015 at 6:27 PM, Noble Paul  wrote:
> Release notes are here . Please feel free to edit them
>
> https://wiki.apache.org/solr/ReleaseNote531
>
> https://wiki.apache.org/lucene-java/ReleaseNote531
>
> On Thu, Sep 24, 2015 at 5:15 PM, Shalin Shekhar Mangar
>  wrote:
>> Hi Noble,
>>
>> Have you created release notes for the 5.3.1 release? Can you share
>> them with us?
>>
>> On Thu, Sep 24, 2015 at 4:58 PM, Noble Paul  wrote:
>>> I'm about to send the announcement mail.
>>>
>>> lease take a look at http://lucene.us.apache.org to see if I have
>>> missed anything
>>>
>>> --Noble
>>>
>>> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
 Vote passed . I shall start the release process

 On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
 wrote:
> +1 for both Java7 and Java8 !
>
> SUCCESS! [1:12:33.649607]
>
> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  wrote:
>>
>> The artifacts can be downloaded from:
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>
>> You can run the smoke tester directly with this command:
>> python3 -u dev-tools/scripts/smokeTestRelease.py
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>>
>>
>> +1
>> SUCCESS! [0:50:32.819792]
>>
>>
>> --
>> -
>> Noble Paul
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> Anshum Gupta



 --
 -
 Noble Paul
>>>
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



-- 
Regards,
Shalin Shekhar Mangar.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Noble Paul]

Release notes are here . Please feel free to edit them

https://wiki.apache.org/solr/ReleaseNote531

https://wiki.apache.org/lucene-java/ReleaseNote531

On Thu, Sep 24, 2015 at 5:15 PM, Shalin Shekhar Mangar
 wrote:
> Hi Noble,
>
> Have you created release notes for the 5.3.1 release? Can you share
> them with us?
>
> On Thu, Sep 24, 2015 at 4:58 PM, Noble Paul  wrote:
>> I'm about to send the announcement mail.
>>
>> lease take a look at http://lucene.us.apache.org to see if I have
>> missed anything
>>
>> --Noble
>>
>> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
>>> Vote passed . I shall start the release process
>>>
>>> On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
>>> wrote:
 +1 for both Java7 and Java8 !

 SUCCESS! [1:12:33.649607]

 On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  wrote:
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>
> You can run the smoke tester directly with this command:
> python3 -u dev-tools/scripts/smokeTestRelease.py
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>
>
> +1
> SUCCESS! [0:50:32.819792]
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



 --
 Anshum Gupta
>>>
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>
>>
>>
>> --
>> -
>> Noble Paul
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906300#comment-14906300
 ] 

Robert Muir commented on LUCENE-6813:
-

We also don't use them all the time: we don't want the possibility of hiding 
bugs with any mocking layers. 

And this stuff is complex... what we have "works for us"  but I would not say 
it is ready to be set as default for the jvm, that is my opinion.

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6066) CollapsingQParserPlugin + Elevation does not respects "fq" (filter query)

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906357#comment-14906357
 ] 

Joel Bernstein commented on SOLR-6066:
--

No, SOLR-6345 wasn't addressed in SOLR-6581.

There is some significant work being done on the CollapsingQparserPlugin in 
SOLR-6168. It doesn't address SOLR-6345 but maybe it makes sense to give that 
issue some thought while CollapsingQParser plugin is being reviewed. Best bet 
is to contact LucidWorks.

> CollapsingQParserPlugin + Elevation does not respects "fq" (filter query) 
> --
>
> Key: SOLR-6066
> URL: https://issues.apache.org/jira/browse/SOLR-6066
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 4.8
>Reporter: Herb Jiang
>Assignee: Joel Bernstein
> Fix For: 4.9
>
> Attachments: SOLR-6066.patch, SOLR-6066.patch, SOLR-6066.patch, 
> TestCollapseQParserPlugin.java
>
>
> QueryElevationComponent respects the "fq" parameter. But when use 
> CollapsingQParserPlugin with QueryElevationComponent, additional "fq" has no 
> effect.
> I use following test case to show this issue. (It will failed)
> {code:java}
> String[] doc = {"id","1", "term_s", "", "group_s", "group1", 
> "category_s", "cat2", "test_ti", "5", "test_tl", "10", "test_tf", "2000"};
> assertU(adoc(doc));
> assertU(commit());
> String[] doc1 = {"id","2", "term_s","", "group_s", "group1", 
> "category_s", "cat2", "test_ti", "50", "test_tl", "100", "test_tf", "200"};
> assertU(adoc(doc1));
> String[] doc2 = {"id","3", "term_s", "", "test_ti", "5000", 
> "test_tl", "100", "test_tf", "200"};
> assertU(adoc(doc2));
> assertU(commit());
> String[] doc3 = {"id","4", "term_s", "", "test_ti", "500", "test_tl", 
> "1000", "test_tf", "2000"};
> assertU(adoc(doc3));
> String[] doc4 = {"id","5", "term_s", "", "group_s", "group2", 
> "category_s", "cat1", "test_ti", "4", "test_tl", "10", "test_tf", "2000"};
> assertU(adoc(doc4));
> assertU(commit());
> String[] doc5 = {"id","6", "term_s","", "group_s", "group2", 
> "category_s", "cat1", "test_ti", "10", "test_tl", "100", "test_tf", "200"};
> assertU(adoc(doc5));
> assertU(commit());
> //Test additional filter query when using collapse
> params = new ModifiableSolrParams();
> params.add("q", "");
> params.add("fq", "{!collapse field=group_s}");
> params.add("fq", "category_s:cat1");
> params.add("defType", "edismax");
> params.add("bf", "field(test_ti)");
> params.add("qf", "term_s");
> params.add("qt", "/elevate");
> params.add("elevateIds", "2");
> assertQ(req(params), "*[count(//doc)=1]",
> "//result/doc[1]/float[@name='id'][.='6.0']");
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7883) MoreLikeThis is incompatible with facets

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906437#comment-14906437
 ] 

Tobias Kässmann commented on SOLR-7883:
---

We ran into the same exception with Solr 5.3 and in our case it was a wrongly 
configured Request handler.
We accidentally skip the prepare step of the FacetComponent by activate 
facetting in our own component afterwards.

> MoreLikeThis is incompatible with facets
> 
>
> Key: SOLR-7883
> URL: https://issues.apache.org/jira/browse/SOLR-7883
> Project: Solr
>  Issue Type: Bug
>  Components: faceting, MoreLikeThis
>Affects Versions: 5.2.1
> Environment: Arch Linux / OpenJDK 7.u85_2.6.1-1
>Reporter: Thomas Seidl
>
> When using the {{MoreLikeThis}} request handler, it doesn't seem possible to 
> also have facets. This worked in Solr 4, but seems to be broken now.
> Example:
> This works: {{?qt=mlt=id:item1=content}}
> This doesn't: {{?qt=mlt=id:item1=content=true}}
> (Yes, you don't even need to specify any facet fields/ranges/queries. The 
> {{q}} query just has to match an item.)
> While the latter will actually return the same result set as the former, the 
> HTTP status is 500 and the following error included in the response:
> {noformat}
> java.lang.NullPointerException
>   at 
> org.apache.solr.request.SimpleFacets.getHeatmapCounts(SimpleFacets.java:1753)
>   at 
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:289)
>   at 
> org.apache.solr.handler.MoreLikeThisHandler.handleRequestBody(MoreLikeThisHandler.java:233)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
>   at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>   at org.eclipse.jetty.server.Server.handle(Server.java:497)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>   at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6699) Integrate lat/lon BKD and spatial3d

[Terry Smith (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906465#comment-14906465
 ] 

Terry Smith commented on LUCENE-6699:
-

Thanks guys. I was hoping to squeeze those x,y,z values into a 64 bits instead 
of 96. I'm not a bit twiddler but I'll take a look at Nicholas' patch and see 
if I can adapt it.



> Integrate lat/lon BKD and spatial3d
> ---
>
> Key: LUCENE-6699
> URL: https://issues.apache.org/jira/browse/LUCENE-6699
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: Geo3DPacking.java, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch
>
>
> I'm opening this for discussion, because I'm not yet sure how to do
> this integration, because of my ignorance about spatial in general and
> spatial3d in particular :)
> Our BKD tree impl is very fast at doing lat/lon shape intersection
> (bbox, polygon, soon distance: LUCENE-6698) against previously indexed
> points.
> I think to integrate with spatial3d, we would first need to record
> lat/lon/z into doc values.  Somewhere I saw discussion about how we
> could stuff all 3 into a single long value with acceptable precision
> loss?  Or, we could use BinaryDocValues?  We need all 3 dims available
> to do the fast per-hit query time filtering.
> But, second: what do we index into the BKD tree?  Can we "just" index
> earth surface lat/lon, and then at query time is spatial3d able to
> give me an enclosing "surface lat/lon" bbox for a 3d shape?  Or
> ... must we index all 3 dimensions into the BKD tree (seems like this
> could be somewhat wasteful)?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Noble Paul]

Alexandre : I've added a known issues section

On Thu, Sep 24, 2015 at 7:27 PM, Noble Paul  wrote:
> Do we have a known issues section where we can add this
>
> On Thu, Sep 24, 2015 at 7:23 PM, Alexandre Rafalovitch
>  wrote:
>> I know SOLR-8073 has not made it into 5.3.1, but should we document a
>> workaround somewhere? (tl;dr: on Windows, startup fails when using
>> relative path to solr home; use absolute path).
>>
>> Regards,
>>Alex.
>> 
>> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
>> http://www.solr-start.com/
>>
>>
>> On 24 September 2015 at 08:57, Noble Paul  wrote:
>>> Release notes are here . Please feel free to edit them
>>>
>>> https://wiki.apache.org/solr/ReleaseNote531
>>>
>>> https://wiki.apache.org/lucene-java/ReleaseNote531
>>>
>>> On Thu, Sep 24, 2015 at 5:15 PM, Shalin Shekhar Mangar
>>>  wrote:
 Hi Noble,

 Have you created release notes for the 5.3.1 release? Can you share
 them with us?

 On Thu, Sep 24, 2015 at 4:58 PM, Noble Paul  wrote:
> I'm about to send the announcement mail.
>
> lease take a look at http://lucene.us.apache.org to see if I have
> missed anything
>
> --Noble
>
> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
>> Vote passed . I shall start the release process
>>
>> On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
>> wrote:
>>> +1 for both Java7 and Java8 !
>>>
>>> SUCCESS! [1:12:33.649607]
>>>
>>> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  
>>> wrote:

 The artifacts can be downloaded from:

 https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/

 You can run the smoke tester directly with this command:
 python3 -u dev-tools/scripts/smokeTestRelease.py

 https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/


 +1
 SUCCESS! [0:50:32.819792]


 --
 -
 Noble Paul

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>>>
>>>
>>>
>>> --
>>> Anshum Gupta
>>
>>
>>
>> --
>> -
>> Noble Paul
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



 --
 Regards,
 Shalin Shekhar Mangar.

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>>>
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> -
> Noble Paul



-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6815) Should DisjunctionScorer advance more lazily?

[Terry Smith (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906401#comment-14906401
 ] 

Terry Smith commented on LUCENE-6815:
-

Additionally, I don't see DisiPriorityQueue taking the cost of each scorer into 
account. I'd imagine that the scorer with highest cost is more likely to be a 
hit which would make this kind of lazy advancing even better.


> Should DisjunctionScorer advance more lazily?
> -
>
> Key: LUCENE-6815
> URL: https://issues.apache.org/jira/browse/LUCENE-6815
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
>
> Today if you call DisjunctionScorer.advance(X), it will try to advance all 
> sub scorers to X. However, if DisjunctionScorer is being intersected with 
> another scorer (which is almost always the case as we use BooleanScorer for 
> top-level disjunctions), we could stop as soon as we find one matching sub 
> scorer, and only advance the remaining sub scorers when freq() or score() is 
> called. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Alexandre Rafalovitch]

Great. Maybe adjust the text slightly to say:
"On Windows, Solr fails to start correctly when using relative path
with -s parameter."

This way, anybody not on Windows will not read past the coma :-)

Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/


On 24 September 2015 at 10:18, Noble Paul  wrote:
> Alexandre : I've added a known issues section
>
> On Thu, Sep 24, 2015 at 7:27 PM, Noble Paul  wrote:
>> Do we have a known issues section where we can add this
>>
>> On Thu, Sep 24, 2015 at 7:23 PM, Alexandre Rafalovitch
>>  wrote:
>>> I know SOLR-8073 has not made it into 5.3.1, but should we document a
>>> workaround somewhere? (tl;dr: on Windows, startup fails when using
>>> relative path to solr home; use absolute path).
>>>
>>> Regards,
>>>Alex.
>>> 
>>> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
>>> http://www.solr-start.com/
>>>
>>>
>>> On 24 September 2015 at 08:57, Noble Paul  wrote:
 Release notes are here . Please feel free to edit them

 https://wiki.apache.org/solr/ReleaseNote531

 https://wiki.apache.org/lucene-java/ReleaseNote531

 On Thu, Sep 24, 2015 at 5:15 PM, Shalin Shekhar Mangar
  wrote:
> Hi Noble,
>
> Have you created release notes for the 5.3.1 release? Can you share
> them with us?
>
> On Thu, Sep 24, 2015 at 4:58 PM, Noble Paul  wrote:
>> I'm about to send the announcement mail.
>>
>> lease take a look at http://lucene.us.apache.org to see if I have
>> missed anything
>>
>> --Noble
>>
>> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
>>> Vote passed . I shall start the release process
>>>
>>> On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
>>> wrote:
 +1 for both Java7 and Java8 !

 SUCCESS! [1:12:33.649607]

 On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  
 wrote:
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>
> You can run the smoke tester directly with this command:
> python3 -u dev-tools/scripts/smokeTestRelease.py
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
>
>
> +1
> SUCCESS! [0:50:32.819792]
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



 --
 Anshum Gupta
>>>
>>>
>>>
>>> --
>>> -
>>> Noble Paul
>>
>>
>>
>> --
>> -
>> Noble Paul
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



 --
 -
 Noble Paul

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>>
>>
>> --
>> -
>> Noble Paul
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6699) Integrate lat/lon BKD and spatial3d

[Nicholas Knize (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906486#comment-14906486
 ] 

Nicholas Knize commented on LUCENE-6699:


I started by packing all 3 values into a 64 bit long - in fact those MAGIC 
numbers are still there (MAGIC[7:12]). The problem with this is precision loss 
from compressing each value into 21 bits. The decoded values will give, at 
best, 3 decimal precision accuracy - which is ~110m at the equator. If you're 
fine with course grain accuracy then copy the BitUtil.{interleave | 
deinterleave} methods and use MAGIC[7:12]. The code is much simpler with the 
accuracy trade-off.

> Integrate lat/lon BKD and spatial3d
> ---
>
> Key: LUCENE-6699
> URL: https://issues.apache.org/jira/browse/LUCENE-6699
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: Geo3DPacking.java, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, 
> LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch
>
>
> I'm opening this for discussion, because I'm not yet sure how to do
> this integration, because of my ignorance about spatial in general and
> spatial3d in particular :)
> Our BKD tree impl is very fast at doing lat/lon shape intersection
> (bbox, polygon, soon distance: LUCENE-6698) against previously indexed
> points.
> I think to integrate with spatial3d, we would first need to record
> lat/lon/z into doc values.  Somewhere I saw discussion about how we
> could stuff all 3 into a single long value with acceptable precision
> loss?  Or, we could use BinaryDocValues?  We need all 3 dims available
> to do the fast per-hit query time filtering.
> But, second: what do we index into the BKD tree?  Can we "just" index
> earth surface lat/lon, and then at query time is spatial3d able to
> give me an enclosing "surface lat/lon" bbox for a 3d shape?  Or
> ... must we index all 3 dimensions into the BKD tree (seems like this
> could be somewhat wasteful)?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-5.x - Build # 329 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-5.x/329/

No tests ran.

Build Log:
[...truncated 52977 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/dist
 [copy] Copying 461 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 245 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.7 
JAVA_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
   [smoker] Java 1.8 
JAVA_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.1 MB in 0.01 sec (10.0 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-5.4.0-src.tgz...
   [smoker] 28.5 MB in 0.03 sec (833.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-5.4.0.tgz...
   [smoker] 66.2 MB in 0.08 sec (833.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-5.4.0.zip...
   [smoker] 76.5 MB in 0.09 sec (826.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-5.4.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.7...
   [smoker]   got 6128 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] test demo with 1.8...
   [smoker]   got 6128 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-5.4.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.7...
   [smoker]   got 6128 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] test demo with 1.8...
   [smoker]   got 6128 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-5.4.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 7 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.7...
   [smoker]   got 211 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] generate javadocs w/ Java 7...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 211 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] Releases that don't seem to be tested:
   [smoker]   5.3.1
   [smoker] Traceback (most recent call last):
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py",
 line 1449, in 
   [smoker] main()
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py",
 line 1394, in main
   [smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, 
c.is_signed, ' '.join(c.test_args))
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py",
 line 1432, in smokeTest
   [smoker] unpackAndVerify(java, 'lucene', tmpDir, 'lucene-%s-src.tgz' % 
version, svnRevision, version, testArgs, baseURL)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py",
 line 583, in unpackAndVerify
   [smoker] verifyUnpacked(java, project, artifact, unpackPath, 
svnRevision, version, testArgs, tmpDir, baseURL)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py",
 line 762, in verifyUnpacked
   [smoker] confirmAllReleasesAreTestedForBackCompat(unpackPath)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py",
 line 1387, in confirmAllReleasesAreTestedForBackCompat
   [smoker] raise RuntimeError('some releases are not tested by 
TestBackwardsCompatibility?')
   [smoker] RuntimeError: some releases are not tested by 

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Anshum Gupta]

Hi Noble,

We generally only announce Solr release (not Lucene) on the Solr - News
page. Think we should fix that headline there?
http://lucene.apache.org/solr/news.html

On Thu, Sep 24, 2015 at 4:28 AM, Noble Paul  wrote:

> I'm about to send the announcement mail.
>
> lease take a look at http://lucene.us.apache.org to see if I have
> missed anything
>
> --Noble
>
> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
> > Vote passed . I shall start the release process
> >
> > On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta 
> wrote:
> >> +1 for both Java7 and Java8 !
> >>
> >> SUCCESS! [1:12:33.649607]
> >>
> >> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul 
> wrote:
> >>>
> >>> The artifacts can be downloaded from:
> >>>
> >>>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
> >>>
> >>> You can run the smoke tester directly with this command:
> >>> python3 -u dev-tools/scripts/smokeTestRelease.py
> >>>
> >>>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
> >>>
> >>>
> >>> +1
> >>> SUCCESS! [0:50:32.819792]
> >>>
> >>>
> >>> --
> >>> -
> >>> Noble Paul
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >>> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>>
> >>
> >>
> >>
> >> --
> >> Anshum Gupta
> >
> >
> >
> > --
> > -
> > Noble Paul
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>


-- 
Anshum Gupta

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906547#comment-14906547
 ] 

Steve Rowe commented on SOLR-8091:
--

Thanks [~FranLukesh], I'll make the change to #33.

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906550#comment-14906550
 ] 

Michael McCandless commented on LUCENE-6814:


Commenting on the stuff you edited away because they are good confusions!

bq. I'm curious why there shouldn't there be some trimming in `end()` as well? 
Or is a `TokenStream` meant to be used only once (no multiple `reset()`, 
`incrementToken()`, `end()` on the same `TokenStream`)?

The {{TokenStream}} API is confusing :)

I started with {{end}} here too (it seemed correct) but it turns out {{close}} 
is also called (internally, in Lucene's {{IndexWriter}}) after all tokens are 
iterated for a single input, but {{close}} is called even on exception (but 
{{end}} is not necessarily I think).

The {{TokenStream}} instance is typically thread-private, and re-used (for that 
one thread) for analysis on future docs.

bq. Elasticsearch seems to never reinstantiate Tokenizers and just reuses them 
for each field in an index, though I may be wrong. Or elasticsearch is using 
TokenStream the wrong way?

ES using Lucene's {{Analyzer}} (well, {{DelegatingAnalyzerWrapper}} I think), 
which (by default) reuses the {{Tokenizer}} instance, per thread.

bq. It'd be great if this can get added to 4.10 so elasticsearch 1.x can pull 
it in too.

I think it's unlikely Lucene will have another 4.10.x release, and ES is 
releasing 2.0.0 (using Lucene 5.3.x) shortly.

Can you describe what impact you're seeing from this bug?  How many 
{{PatternTokenizer}} instances is ES keeping in your case, how large are your 
docs, etc.?  You could probably lower the ES bulk indexing thread pool size (if 
you don't in fact need so much concurrency) to reduce the impact of the bug ...

I think this bug means {{PatternTokenizer}} holds onto the max sized doc it 
ever saw in heap right?  Does {{StringBuilder}} ever reduce its allocated space 
by itself...

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_60) - Build # 5282 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/5282/
Java: 32bit/jdk1.8.0_60 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([71041E64357D6027:40BFA051904270F7]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:765)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:732)
at 
org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts(SpellCheckCollatorTest.java:530)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//lst[@name='spellcheck']/lst[@name='collations']/lst[@name='collation']/int[@name='hits'
 and 6 <= . and . <= 10]
xml response was: 

021918everyotherteststop:everyother14everyother

[jira] [Created] (SOLR-8090) Admin Interface Fails WCAG Contrast Ratio Tests

[Mike Mallett (JIRA)]

Mike Mallett created SOLR-8090:
--

 Summary: Admin Interface Fails WCAG Contrast Ratio Tests
 Key: SOLR-8090
 URL: https://issues.apache.org/jira/browse/SOLR-8090
 Project: Solr
  Issue Type: Improvement
  Components: UI
Affects Versions: 5.3
Reporter: Mike Mallett


The CSS for the admin UI in 5.3.1 has the following definition which affects 
help text in a variety of places:

#core-selector #has-no-cores span
{
  color: #c0c0c0;
  display: block;
}

This is displayed on a white background, making the text with colour #c0c0c0 
next to impossible to read. This fails WCAG accessability guidelines and can be 
checked using this Contract Checker: 
http://webaim.org/resources/contrastchecker/

Please adjust the CSS for the admin UI to display text in a more highly 
contrasted colour. Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[Francis Lukesh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906543#comment-14906543
 ] 

Francis Lukesh commented on SOLR-8091:
--

I support changing all gray text to #33, which passes both WCAG tests and 
doesn't sacrifice much of anything aesthetically.

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe reassigned SOLR-8091:


Assignee: Steve Rowe

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906552#comment-14906552
 ] 

Dawid Weiss commented on LUCENE-6813:
-

bq. We also don't use them all the time: we don't want the possibility of 
hiding bugs with any mocking layers.

I know, but since these are effectively decorators I think it could be done in 
a way in which you'd turn them on and off dynamically. The only difference 
would be that they'd be effectively global. I don't want to impose it on 
Lucene, I'm just saying it'd be a fun exercise in general to write such a layer.

The patch looks good to me.

> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906495#comment-14906495
 ] 

Michael McCandless commented on LUCENE-6813:


bq. Mike, your setter can also be package private

Oh good point, I'll do that.


> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[Mike Mallett (JIRA)]

Mike Mallett created SOLR-8091:
--

 Summary: Solr Website Fails WCAG Contrast Ratio
 Key: SOLR-8091
 URL: https://issues.apache.org/jira/browse/SOLR-8091
 Project: Solr
  Issue Type: Improvement
  Components: website
Reporter: Mike Mallett


The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
WCAG Accessibility Guidelines for Contrast Ratios.

The site places a lot of text with colour #959693 on a white background This 
fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this site: 
http://webaim.org/resources/contrastchecker/

Please use a more readable font colour for text on the Apache Solr website.

Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6813) OfflineSorter.sort shouldn't remove the output Path up front

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-6813:
---
Attachment: LUCENE-6813.patch

OK how about this new patch?

I made the new setter package private, and I removed the static class init,
and instead made it lazy init the first time someone asks for it.

And I renamed {{defaultTempDir()}} --> {{getDefaultTempDir()}}.


> OfflineSorter.sort shouldn't remove the output Path up front
> 
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch, LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like 
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906560#comment-14906560
 ] 

ASF subversion and git services commented on SOLR-8091:
---

Commit 1705093 from [~steve_rowe] in branch 'cms/trunk'
[ https://svn.apache.org/r1705093 ]

SOLR-8091: Darken Solr website base font color: #959693 -> #33

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Please vote for the 3rd release candidate for Lucene/Solr 5.3.1

[Steve Rowe]

Also, the download link on the Solr News page is 
, but for every other 
release we’ve used the redirect-to-latest link 
 - I think we 
should do the same for 5.3.1.

> On Sep 24, 2015, at 11:39 AM, Anshum Gupta  wrote:
> 
> Hi Noble,
> 
> We generally only announce Solr release (not Lucene) on the Solr - News page. 
> Think we should fix that headline there?
> http://lucene.apache.org/solr/news.html
> 
> On Thu, Sep 24, 2015 at 4:28 AM, Noble Paul  wrote:
> I'm about to send the announcement mail.
> 
> lease take a look at http://lucene.us.apache.org to see if I have
> missed anything
> 
> --Noble
> 
> On Wed, Sep 23, 2015 at 4:31 PM, Noble Paul  wrote:
> > Vote passed . I shall start the release process
> >
> > On Thu, Sep 17, 2015 at 10:29 PM, Anshum Gupta  
> > wrote:
> >> +1 for both Java7 and Java8 !
> >>
> >> SUCCESS! [1:12:33.649607]
> >>
> >> On Wed, Sep 16, 2015 at 4:22 PM, Noble Paul  wrote:
> >>>
> >>> The artifacts can be downloaded from:
> >>>
> >>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
> >>>
> >>> You can run the smoke tester directly with this command:
> >>> python3 -u dev-tools/scripts/smokeTestRelease.py
> >>>
> >>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.3.1-RC3-rev1703449/
> >>>
> >>>
> >>> +1
> >>> SUCCESS! [0:50:32.819792]
> >>>
> >>>
> >>> --
> >>> -
> >>> Noble Paul
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >>> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>>
> >>
> >>
> >>
> >> --
> >> Anshum Gupta
> >
> >
> >
> > --
> > -
> > Noble Paul
> 
> 
> 
> --
> -
> Noble Paul
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 
> 
> 
> 
> -- 
> Anshum Gupta


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906601#comment-14906601
 ] 

Uwe Schindler commented on LUCENE-6814:
---

Tokenstream consumer workflow is clearly defined:
https://lucene.apache.org/core/5_3_1/core/org/apache/lucene/analysis/TokenStream.html

The last step is close(). There is nothing confusing, just RTFM.

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906601#comment-14906601
 ] 

Uwe Schindler edited comment on LUCENE-6814 at 9/24/15 4:33 PM:


Tokenstream consumer workflow is clearly defined:
https://lucene.apache.org/core/5_3_1/core/org/apache/lucene/analysis/TokenStream.html

The last step is close(). There is nothing confusing, just RTFM.

close() is defined as "Releases resources associated with this stream.", and 
that's what we do here. end() has a different meaning. Its sole purpose is to 
get the "token" information after the very last token. At this time all 
resources still need to be hold, because the attributes must be accessible 
after this call. After close() the resources are freed. In fact, this does not 
make a difference for this tokenizer, but in general stuff like termattribute 
must still be in a valid state.


was (Author: thetaphi):
Tokenstream consumer workflow is clearly defined:
https://lucene.apache.org/core/5_3_1/core/org/apache/lucene/analysis/TokenStream.html

The last step is close(). There is nothing confusing, just RTFM.

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8092) The same faceting DocSet is generated in both the first and second phase of distributed search if there are facet refinements.

[Joel Bernstein (JIRA)]

Joel Bernstein created SOLR-8092:


 Summary: The same faceting DocSet is generated in both the first 
and second phase of distributed search if there are facet refinements.
 Key: SOLR-8092
 URL: https://issues.apache.org/jira/browse/SOLR-8092
 Project: Solr
  Issue Type: Improvement
Reporter: Joel Bernstein


Currently the facet DocSet generated during the first phase of the distributed 
search is not cached. This means that during the facet refinement phase the 
DocSet needs to be recreated. This involves re-issuing the entire query and 
re-building the DocSet. In some use cases that can be a high price to pay to 
calculate refinements.

This ticket aims to find a way to cache the DocSet generated from the first 
phase so it can be re-used in the second phase.

We may also want to add a way to turn off refinements as some use case might 
want to trade off performance for accuracy.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8092) The same faceting DocSet is generated in both the first and second phase of distributed search if there are facet refinements.

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-8092:
-
Priority: Minor  (was: Major)

> The same faceting DocSet is generated in both the first and second phase of 
> distributed search if there are facet refinements.
> --
>
> Key: SOLR-8092
> URL: https://issues.apache.org/jira/browse/SOLR-8092
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
>
> Currently the facet DocSet generated during the first phase of the 
> distributed search is not cached. This means that during the facet refinement 
> phase the DocSet needs to be recreated. This involves re-issuing the entire 
> query and re-building the DocSet. In some use cases that can be a high price 
> to pay to calculate refinements.
> This ticket aims to find a way to cache the DocSet generated from the first 
> phase so it can be re-used in the second phase.
> We may also want to add a way to turn off refinements as some use cases might 
> want to trade off performance for accuracy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8092) The same faceting DocSet is generated in both the first and second phase of distributed search if there are facet refinements.

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-8092:
-
Description: 
Currently the facet DocSet generated during the first phase of the distributed 
search is not cached. This means that during the facet refinement phase the 
DocSet needs to be recreated. This involves re-issuing the entire query and 
re-building the DocSet. In some use cases that can be a high price to pay to 
calculate refinements.

This ticket aims to find a way to cache the DocSet generated from the first 
phase so it can be re-used in the second phase.

We may also want to add a way to turn off refinements as some use cases might 
want to trade off performance for accuracy.


  was:
Currently the facet DocSet generated during the first phase of the distributed 
search is not cached. This means that during the facet refinement phase the 
DocSet needs to be recreated. This involves re-issuing the entire query and 
re-building the DocSet. In some use cases that can be a high price to pay to 
calculate refinements.

This ticket aims to find a way to cache the DocSet generated from the first 
phase so it can be re-used in the second phase.

We may also want to add a way to turn off refinements as some use case might 
want to trade off performance for accuracy.



> The same faceting DocSet is generated in both the first and second phase of 
> distributed search if there are facet refinements.
> --
>
> Key: SOLR-8092
> URL: https://issues.apache.org/jira/browse/SOLR-8092
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>
> Currently the facet DocSet generated during the first phase of the 
> distributed search is not cached. This means that during the facet refinement 
> phase the DocSet needs to be recreated. This involves re-issuing the entire 
> query and re-building the DocSet. In some use cases that can be a high price 
> to pay to calculate refinements.
> This ticket aims to find a way to cache the DocSet generated from the first 
> phase so it can be re-used in the second phase.
> We may also want to add a way to turn off refinements as some use cases might 
> want to trade off performance for accuracy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-8092) The same faceting DocSet is generated in both the first and second phase of distributed search if there are facet refinements.

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein reassigned SOLR-8092:


Assignee: Joel Bernstein

> The same faceting DocSet is generated in both the first and second phase of 
> distributed search if there are facet refinements.
> --
>
> Key: SOLR-8092
> URL: https://issues.apache.org/jira/browse/SOLR-8092
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>
> Currently the facet DocSet generated during the first phase of the 
> distributed search is not cached. This means that during the facet refinement 
> phase the DocSet needs to be recreated. This involves re-issuing the entire 
> query and re-building the DocSet. In some use cases that can be a high price 
> to pay to calculate refinements.
> This ticket aims to find a way to cache the DocSet generated from the first 
> phase so it can be re-used in the second phase.
> We may also want to add a way to turn off refinements as some use cases might 
> want to trade off performance for accuracy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[ANNOUNCE] Apache Lucene 5.3.1 released

[Noble Paul]

24 September 2015, Apache Solr™ 5.3.1 available


The Lucene PMC is pleased to announce the release of Apache Solr 5.3.1


Solr is the popular, blazing fast, open source NoSQL search platform

from the Apache Lucene project. Its major features include powerful

full-text search, hit highlighting, faceted search, dynamic

clustering, database integration, rich document (e.g., Word, PDF)

handling, and geospatial search. Solr is highly scalable, providing

fault tolerant distributed search and indexing, and powers the search

and navigation features of many of the world's largest internet sites.


This release contains various bug fixes and optimizations since the
5.3.0 release. The release is available for immediate download at:


  http://lucene.apache.org/solr/mirrors-solr-latest-redir.html


Please read CHANGES.txt for a full list of new features and changes:


  https://lucene.apache.org/solr/5_3_1/changes/Changes.html


Solr 5.3.1 includes these bug fixes.


 * security.json is not loaded on server start

 * RuleBasedAuthorization plugin does not respect the
collection-admin-edit permission

 * Fix VelocityResponseWriter template encoding issue. Templates must
be UTF-8 encoded

 * SimplePostTool (also bin/post) -filetypes "*" now works properly in
'web' mode

 * example/files update-script.js to be Java 7 and 8 compatible.

 * SolrJ could not make requests to handlers with '/admin/' prefix

 * Use of timeAllowed can cause incomplete filters to be cached and
incorrect results to be returned on subsequent requests

 * VelocityResponseWriter's $resource.get(key,baseName,locale) to use
specified locale.

 * Fix the exclusion filter so that collections that start with js,
css, img, tpl can be accessed.

 * Resolve XSS issue in Admin UI stats page


Known issues:

 * On Windows, bin/solr.cmd script fails to start correctly when using
relative path with -s parameter. Use absolute path as a workaround .
https://issues.apache.org/jira/browse/SOLR-8073


See the CHANGES.txt file included with the release for a full list of
changes and further details.


Please report any feedback to the mailing lists
(http://lucene.apache.org/solr/discussion.html)


Note: The Apache Software Foundation uses an extensive mirroring
network for distributing releases. It is possible that the mirror you
are using may not have replicated the release yet. If that is the
case, please try another mirror. This also goes for Maven access.

Noble Paul
on behalf of Lucene PMC

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906724#comment-14906724
 ] 

ASF subversion and git services commented on SOLR-8091:
---

Commit 1705119 from [~steve_rowe] in branch 'cms/trunk'
[ https://svn.apache.org/r1705119 ]

SOLR-8091: Darken sub-nav header items on the Solr Resources page when pinned 
at the top during scrolling: #999 -> #666

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Alex Chow (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906797#comment-14906797
 ] 

Alex Chow commented on LUCENE-6814:
---

{quote}
Tokenstream consumer workflow is clearly defined:
https://lucene.apache.org/core/5_3_1/core/org/apache/lucene/analysis/TokenStream.html
The last step is close(). There is nothing confusing, just RTFM.
{quote}

Yeah, just took a little bit to fully understand how things interact with reuse 
("expert mode").

bq. I think it's unlikely Lucene will have another 4.10.x release, and ES is 
releasing 2.0.0 (using Lucene 5.3.x) shortly.

That's pretty unfortunate. The tracker suggests that there are some patches 
slated for 4.10.5. Is 4.10 dead now?

I'm not able to find what the plan for 1.x is when 2.0 hits GA. 1.7 was cut in 
July, so that release is probably going to be supported through early 2017. 
I'll probably have to push for the PatternTokenizer fork to get it into 1.x?

bq. Can you describe what impact you're seeing from this bug? How many 
{{PatternTokenizer}} instances is ES keeping in your case, how large are your 
docs, etc.? You could probably lower the ES bulk indexing thread pool size (if 
you don't in fact need so much concurrency) to reduce the impact of the bug...

Our setup has 24 bulk indexing threads, and at peak we go through about 18 
tasks/s (8k documents indexed per bulk request) per node with 21 nodes and 168 
indices.

We've been making an effort to reduce heap sizes (from 22gb to 8gb using 3x 
nodes), and {{PatternTokenizer}} ends up retaining around 2gb before we get 
into a GC death spiral (CMS with instantiating occupancy=75%) but would 
otherwise grow a bit more. The biggest {{PatternTokenizer}} instances get to 
about 3-4mb. {{SegmentReaders}} occupy 3.5gb per node, so there's not too much 
space to work with unless we want to increase heap at least 1.5x. This pretty 
much destroys scaling horizontally and ends up being more... diagonally with 
volume depending on the data?

I'm pretty surprised nobody has really noticed this so far. Does everyone just 
like huge heaps (or just not use {{PatternAnalyzer}})?

bq. I think this bug means {{PatternTokenizer}} holds onto the max sized doc it 
ever saw in heap right? Does {{StringBuilder}} ever reduce its allocated space 
by itself...

I think this ends up growing to the max sized field. As far as I can tell 
{{trimToSize}} is the only for {{StringBuilder}} to shrink its buffer, and even 
then documentation suggests it isn't guaranteed (lots of "may")...

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[ANNOUNCE] Apache Solr 5.3.1 released

[Noble Paul]

24 September 2015, Apache Solr™ 5.3.1 available


The Lucene PMC is pleased to announce the release of Apache Solr 5.3.1


Solr is the popular, blazing fast, open source NoSQL search platform

from the Apache Lucene project. Its major features include powerful

full-text search, hit highlighting, faceted search, dynamic

clustering, database integration, rich document (e.g., Word, PDF)

handling, and geospatial search. Solr is highly scalable, providing

fault tolerant distributed search and indexing, and powers the search

and navigation features of many of the world's largest internet sites.


This release contains various bug fixes and optimizations since the
5.3.0 release. The release is available for immediate download at:


  http://lucene.apache.org/solr/mirrors-solr-latest-redir.html


Please read CHANGES.txt for a full list of new features and changes:


  https://lucene.apache.org/solr/5_3_1/changes/Changes.html


Solr 5.3.1 includes these bug fixes.


 * security.json is not loaded on server start

 * RuleBasedAuthorization plugin does not respect the
collection-admin-edit permission

 * Fix VelocityResponseWriter template encoding issue. Templates must
be UTF-8 encoded

 * SimplePostTool (also bin/post) -filetypes "*" now works properly in
'web' mode

 * example/files update-script.js to be Java 7 and 8 compatible.

 * SolrJ could not make requests to handlers with '/admin/' prefix

 * Use of timeAllowed can cause incomplete filters to be cached and
incorrect results to be returned on subsequent requests

 * VelocityResponseWriter's $resource.get(key,baseName,locale) to use
specified locale.

 * Fix the exclusion filter so that collections that start with js,
css, img, tpl can be accessed.

 * Resolve XSS issue in Admin UI stats page


Known issues:

 * On Windows, bin/solr.cmd script fails to start correctly when using
relative path with -s parameter. Use absolute path as a workaround .
https://issues.apache.org/jira/browse/SOLR-8073


See the CHANGES.txt file included with the release for a full list of
changes and further details.

Noble Paul
on behalf of Lucene PMC

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-5.3 - Build # 29 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-5.3/29/

No tests ran.

Build Log:
[...truncated 53041 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/lucene/build/smokeTestRelease/dist
 [copy] Copying 461 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 245 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.7 
JAVA_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
   [smoker] Java 1.8 
JAVA_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.1 MB in 0.01 sec (11.2 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-5.3.1-src.tgz...
   [smoker] 28.5 MB in 0.04 sec (679.2 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-5.3.1.tgz...
   [smoker] 65.6 MB in 0.09 sec (706.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-5.3.1.zip...
   [smoker] 75.9 MB in 0.11 sec (702.3 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-5.3.1.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.7...
   [smoker]   got 6059 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] test demo with 1.8...
   [smoker]   got 6059 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-5.3.1.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.7...
   [smoker]   got 6059 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] test demo with 1.8...
   [smoker]   got 6059 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-5.3.1-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 7 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.7...
   [smoker]   got 213 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] generate javadocs w/ Java 7...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 213 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] Releases that don't seem to be tested:
   [smoker]   5.3.1
   [smoker] Traceback (most recent call last):
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/dev-tools/scripts/smokeTestRelease.py",
 line 1449, in 
   [smoker] main()
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/dev-tools/scripts/smokeTestRelease.py",
 line 1394, in main
   [smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, 
c.is_signed, ' '.join(c.test_args))
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/dev-tools/scripts/smokeTestRelease.py",
 line 1432, in smokeTest
   [smoker] unpackAndVerify(java, 'lucene', tmpDir, 'lucene-%s-src.tgz' % 
version, svnRevision, version, testArgs, baseURL)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/dev-tools/scripts/smokeTestRelease.py",
 line 583, in unpackAndVerify
   [smoker] verifyUnpacked(java, project, artifact, unpackPath, 
svnRevision, version, testArgs, tmpDir, baseURL)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/dev-tools/scripts/smokeTestRelease.py",
 line 762, in verifyUnpacked
   [smoker] confirmAllReleasesAreTestedForBackCompat(unpackPath)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.3/dev-tools/scripts/smokeTestRelease.py",
 line 1387, in confirmAllReleasesAreTestedForBackCompat
   [smoker] raise RuntimeError('some releases are not tested by 
TestBackwardsCompatibility?')
   [smoker] RuntimeError: some releases are not tested by 

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906784#comment-14906784
 ] 

Steve Rowe commented on SOLR-8091:
--

Thanks [~nerdcorenet]!

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8088) Distributed grouping seems to require docValues in 5.x, didn't in 4.x

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906906#comment-14906906
 ] 

Shawn Heisey commented on SOLR-8088:


bq. Yet another problem caused by LUCENE-5666

That looks like all kinds of fun.  I saw what you said about that issue on the 
list thread talking about slow facets in 5.3.  I'm guessing that if you find a 
solution to the general problem, it will clean up this as well?

> Distributed grouping seems to require docValues in 5.x, didn't in 4.x
> -
>
> Key: SOLR-8088
> URL: https://issues.apache.org/jira/browse/SOLR-8088
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.2.1
>Reporter: Shawn Heisey
>
> I have a field in my index that is lowercased after the KeywordTokenizer.  I 
> wish to do grouping on this field.  It is a distributed index.
> This works fine in Solr 4.9.1 running on Java 8.
> When I try the distributed grouping request (with the same schema) on Solr 
> 5.2.1, it fails, with this exception:
> {code}
> java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for 
> field 'ip' (expected=SORTED). Use UninvertingReader or index with docvalues.
> {code}
> If I make the same request directly to one of the shards on 5.2.1, it works.  
> If I create a copyField to a field using StrField with docValues, the 
> distributed request works ... but then I lose the lowercasing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8030) Transaction log does not store the update chain used for updates

[Ludovic Boutros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ludovic Boutros updated SOLR-8030:
--
Attachment: SOLR-8030-test.patch

Update of the test.

I have a problem with this test, it creates incoherent replicas (On my mac).

This prevents the real update chain testing.

> Transaction log does not store the update chain used for updates
> 
>
> Key: SOLR-8030
> URL: https://issues.apache.org/jira/browse/SOLR-8030
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.3
>Reporter: Ludovic Boutros
> Attachments: SOLR-8030-test.patch, SOLR-8030-test.patch
>
>
> Transaction Log does not store the update chain used during updates.
> Therefore tLog uses the default update chain during log replay.
> If we implement custom update logic with multiple update chains, the log 
> replay could break this logic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906705#comment-14906705
 ] 

ASF subversion and git services commented on SOLR-8091:
---

Commit 1705118 from [~steve_rowe] in branch 'cms/trunk'
[ https://svn.apache.org/r1705118 ]

SOLR-8091: Lighten sub-nav header items on the Solr Resources page: #555/#000 
-> #666/#000 (otherwise hovering, which uses the darker color, doesn't look 
different enough to be noticeable)

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8093) `bin/solr create_core` syntax doesn't match documentation for '-d' option

[Mike Mallett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Mallett updated SOLR-8093:
---
Component/s: documentation

> `bin/solr create_core` syntax doesn't match documentation for '-d' option
> -
>
> Key: SOLR-8093
> URL: https://issues.apache.org/jira/browse/SOLR-8093
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 5.3
>Reporter: Mike Mallett
>
> I'm attempting to create a new Solr Core using an existing set of config 
> files. Specifically these are being taken from the Drupal module 
> search_api_solr-7.x-1.9 in this tarball: 
> http://ftp.drupal.org/files/projects/search_api_solr-7.x-1.9.tar.gz
> The config dir located in the tarball is: search_api_solr/solr-conf/5.x/
> This works fine, as I was able to copy this config directory in the shell 
> then use the Solr Admin UI to create the core. No errors were reported in 
> that methodology.
> However, I'm reading the help documentation included by executing the command 
> `bin/solr create_core -help` which states:
> "-d   Configuration directory to copy when creating the new core, 
> built-in options are:
>   basic_configs: Minimal Solr configuration
>   data_driven_schema_configs: Managed schema with field-guessing support 
> enabled
>   sample_techproducts_configs: Example configuration with many optional 
> features enabled to demonstrate the full power of Solr
>   If not specified, default is: data_driven_schema_configs
>   Alternatively, you can pass the path to your own configuration 
> directory instead of using one of the built-in configurations, such as: 
> bin/solr create_core -c mycore -d /tmp/myconfig"
> So I tried the following command:
> bin/solr create_core -c testcore -d 
> /opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x
> This resulted in:
> ERROR: Error CREATEing SolrCore 'testcore': Unable to create core [testcore] 
> Caused by: /var/solr5/data/testcore/data
> I was instructed on IRC that I am using the command incorrectly and that 
> valid options for '-d' are only "basic_configs", 
> "data_driven_schema_configs",  or "sample_techproducts_configs"
> I am unable to create a core from the command line, and I don't know what the 
> process should look like.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8093) `bin/solr create_core` syntax doesn't match documentation for '-d' option

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906897#comment-14906897
 ] 

Shawn Heisey commented on SOLR-8093:


I have placed a note in the docs regarding permissions and the create command:

https://cwiki.apache.org/confluence/display/solr/Solr+Start+Script+Reference#SolrStartScriptReference-Create

> `bin/solr create_core` syntax doesn't match documentation for '-d' option
> -
>
> Key: SOLR-8093
> URL: https://issues.apache.org/jira/browse/SOLR-8093
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 5.3
>Reporter: Mike Mallett
>
> I'm attempting to create a new Solr Core using an existing set of config 
> files. Specifically these are being taken from the Drupal module 
> search_api_solr-7.x-1.9 in this tarball: 
> http://ftp.drupal.org/files/projects/search_api_solr-7.x-1.9.tar.gz
> The config dir located in the tarball is: search_api_solr/solr-conf/5.x/
> This works fine, as I was able to copy this config directory in the shell 
> then use the Solr Admin UI to create the core. No errors were reported in 
> that methodology.
> However, I'm reading the help documentation included by executing the command 
> `bin/solr create_core -help` which states:
> "-d   Configuration directory to copy when creating the new core, 
> built-in options are:
>   basic_configs: Minimal Solr configuration
>   data_driven_schema_configs: Managed schema with field-guessing support 
> enabled
>   sample_techproducts_configs: Example configuration with many optional 
> features enabled to demonstrate the full power of Solr
>   If not specified, default is: data_driven_schema_configs
>   Alternatively, you can pass the path to your own configuration 
> directory instead of using one of the built-in configurations, such as: 
> bin/solr create_core -c mycore -d /tmp/myconfig"
> So I tried the following command:
> bin/solr create_core -c testcore -d 
> /opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x
> This resulted in:
> ERROR: Error CREATEing SolrCore 'testcore': Unable to create core [testcore] 
> Caused by: /var/solr5/data/testcore/data
> I was instructed on IRC that I am using the command incorrectly and that 
> valid options for '-d' are only "basic_configs", 
> "data_driven_schema_configs",  or "sample_techproducts_configs"
> I am unable to create a core from the command line, and I don't know what the 
> process should look like.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[ANNOUNCE] Apache Lucene 5.3.1 released

[Noble Paul]

24 September 2015, Apache Lucene™ 5.3.1 available

The Lucene PMC is pleased to announce the release of Apache Lucene 5.3.1

Apache Lucene is a high-performance, full-featured text search engine
library written entirely in Java. It is a technology suitable for nearly
any application that requires full-text search, especially cross-platform.

This release contains various bug fixes and optimizations since the
5.3.0 release. The release
is available for immediate download at:

  http://lucene.apache.org/core/mirrors-core-latest-redir.html

Please read CHANGES.txt for a full list of new features and changes:

  https://lucene.apache.org/core/5_3_1/changes/Changes.html

Lucene 5.3.1 includes 3 bug fixes:

 * Remove classloader hack in MorfologikFilter
 * UsageTrackingQueryCachingPolicy no longer caches trivial queries
like MatchAllDocsQuery
 * Fixed BoostingQuery to rewrite wrapped queries

Please report any feedback to the mailing lists
(http://lucene.apache.org/core/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases.  It is possible that the mirror you are using
may not have replicated the release yet.  If that is the case, please
try another mirror.  This also goes for Maven access.


Noble Paul on behalf of Lucene PMC

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8093) `bin/solr create_core` syntax doesn't match documentation for '-d' option

[Mike Mallett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906804#comment-14906804
 ] 

Mike Mallett commented on SOLR-8093:


In addition to a change in the documentation regarding the create_core command, 
I'd also like to know what the error message means. It appears that a directory 
(/var/solr5/data/testcore/data) is "causing" an error. What does this mean?

> `bin/solr create_core` syntax doesn't match documentation for '-d' option
> -
>
> Key: SOLR-8093
> URL: https://issues.apache.org/jira/browse/SOLR-8093
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.3
>Reporter: Mike Mallett
>
> I'm attempting to create a new Solr Core using an existing set of config 
> files. Specifically these are being taken from the Drupal module 
> search_api_solr-7.x-1.9 in this tarball: 
> http://ftp.drupal.org/files/projects/search_api_solr-7.x-1.9.tar.gz
> The config dir located in the tarball is: search_api_solr/solr-conf/5.x/
> This works fine, as I was able to copy this config directory in the shell 
> then use the Solr Admin UI to create the core. No errors were reported in 
> that methodology.
> However, I'm reading the help documentation included by executing the command 
> `bin/solr create_core -help` which states:
> "-d   Configuration directory to copy when creating the new core, 
> built-in options are:
>   basic_configs: Minimal Solr configuration
>   data_driven_schema_configs: Managed schema with field-guessing support 
> enabled
>   sample_techproducts_configs: Example configuration with many optional 
> features enabled to demonstrate the full power of Solr
>   If not specified, default is: data_driven_schema_configs
>   Alternatively, you can pass the path to your own configuration 
> directory instead of using one of the built-in configurations, such as: 
> bin/solr create_core -c mycore -d /tmp/myconfig"
> So I tried the following command:
> bin/solr create_core -c testcore -d 
> /opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x
> This resulted in:
> ERROR: Error CREATEing SolrCore 'testcore': Unable to create core [testcore] 
> Caused by: /var/solr5/data/testcore/data
> I was instructed on IRC that I am using the command incorrectly and that 
> valid options for '-d' are only "basic_configs", 
> "data_driven_schema_configs",  or "sample_techproducts_configs"
> I am unable to create a core from the command line, and I don't know what the 
> process should look like.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8094) HdfsUpdateLog should not replay buffered documents as a replacement to dropping them.

[Mark Miller (JIRA)]

Mark Miller created SOLR-8094:
-

 Summary: HdfsUpdateLog should not replay buffered documents as a 
replacement to dropping them.
 Key: SOLR-8094
 URL: https://issues.apache.org/jira/browse/SOLR-8094
 Project: Solr
  Issue Type: Bug
Reporter: Mark Miller
Assignee: Mark Miller


This can cause replicas to get out of sync because replication may have failed, 
but then if you replay you can setup peer sync to pass when it should not.

After talking to Yonik a bit, I tried doing everything the same as on local fs 
except skipping the truncate - because of error handling we already have, this 
appears to work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8093) `bin/solr create_core` syntax doesn't match documentation for '-d' option

[Mike Mallett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906857#comment-14906857
 ] 

Mike Mallett commented on SOLR-8093:


Through some troubleshooting on IRC we have discovered that my execution of the 
"create_core" commands as user root, I have created directories which are not 
writable by the jetty process running solr which is user "solr".

It seems these commands should be executed as another user (solr).

The documentation should reflect this.

> `bin/solr create_core` syntax doesn't match documentation for '-d' option
> -
>
> Key: SOLR-8093
> URL: https://issues.apache.org/jira/browse/SOLR-8093
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.3
>Reporter: Mike Mallett
>
> I'm attempting to create a new Solr Core using an existing set of config 
> files. Specifically these are being taken from the Drupal module 
> search_api_solr-7.x-1.9 in this tarball: 
> http://ftp.drupal.org/files/projects/search_api_solr-7.x-1.9.tar.gz
> The config dir located in the tarball is: search_api_solr/solr-conf/5.x/
> This works fine, as I was able to copy this config directory in the shell 
> then use the Solr Admin UI to create the core. No errors were reported in 
> that methodology.
> However, I'm reading the help documentation included by executing the command 
> `bin/solr create_core -help` which states:
> "-d   Configuration directory to copy when creating the new core, 
> built-in options are:
>   basic_configs: Minimal Solr configuration
>   data_driven_schema_configs: Managed schema with field-guessing support 
> enabled
>   sample_techproducts_configs: Example configuration with many optional 
> features enabled to demonstrate the full power of Solr
>   If not specified, default is: data_driven_schema_configs
>   Alternatively, you can pass the path to your own configuration 
> directory instead of using one of the built-in configurations, such as: 
> bin/solr create_core -c mycore -d /tmp/myconfig"
> So I tried the following command:
> bin/solr create_core -c testcore -d 
> /opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x
> This resulted in:
> ERROR: Error CREATEing SolrCore 'testcore': Unable to create core [testcore] 
> Caused by: /var/solr5/data/testcore/data
> I was instructed on IRC that I am using the command incorrectly and that 
> valid options for '-d' are only "basic_configs", 
> "data_driven_schema_configs",  or "sample_techproducts_configs"
> I am unable to create a core from the command line, and I don't know what the 
> process should look like.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-MacOSX (64bit/jdk1.8.0) - Build # 2702 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/2702/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestReplicaProperties.test

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:56918, 
http://127.0.0.1:56927, http://127.0.0.1:56904, http://127.0.0.1:56913, 
http://127.0.0.1:56922]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:56918, http://127.0.0.1:56927, 
http://127.0.0.1:56904, http://127.0.0.1:56913, http://127.0.0.1:56922]
at 
__randomizedtesting.SeedInfo.seed([D0B5585D42450E30:58E16787ECB963C8]:0)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at 
org.apache.solr.cloud.ReplicaPropertiesBase.doPropertyAction(ReplicaPropertiesBase.java:51)
at 
org.apache.solr.cloud.TestReplicaProperties.clusterAssignPropertyTest(TestReplicaProperties.java:183)
at 
org.apache.solr.cloud.TestReplicaProperties.test(TestReplicaProperties.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

Re: [ANNOUNCE] Apache Lucene 5.3.1 released

[Noble Paul]

Wrong title

On Thu, Sep 24, 2015 at 10:55 PM, Noble Paul  wrote:
> 24 September 2015, Apache Solr™ 5.3.1 available
>
>
> The Lucene PMC is pleased to announce the release of Apache Solr 5.3.1
>
>
> Solr is the popular, blazing fast, open source NoSQL search platform
>
> from the Apache Lucene project. Its major features include powerful
>
> full-text search, hit highlighting, faceted search, dynamic
>
> clustering, database integration, rich document (e.g., Word, PDF)
>
> handling, and geospatial search. Solr is highly scalable, providing
>
> fault tolerant distributed search and indexing, and powers the search
>
> and navigation features of many of the world's largest internet sites.
>
>
> This release contains various bug fixes and optimizations since the
> 5.3.0 release. The release is available for immediate download at:
>
>
>   http://lucene.apache.org/solr/mirrors-solr-latest-redir.html
>
>
> Please read CHANGES.txt for a full list of new features and changes:
>
>
>   https://lucene.apache.org/solr/5_3_1/changes/Changes.html
>
>
> Solr 5.3.1 includes these bug fixes.
>
>
>  * security.json is not loaded on server start
>
>  * RuleBasedAuthorization plugin does not respect the
> collection-admin-edit permission
>
>  * Fix VelocityResponseWriter template encoding issue. Templates must
> be UTF-8 encoded
>
>  * SimplePostTool (also bin/post) -filetypes "*" now works properly in
> 'web' mode
>
>  * example/files update-script.js to be Java 7 and 8 compatible.
>
>  * SolrJ could not make requests to handlers with '/admin/' prefix
>
>  * Use of timeAllowed can cause incomplete filters to be cached and
> incorrect results to be returned on subsequent requests
>
>  * VelocityResponseWriter's $resource.get(key,baseName,locale) to use
> specified locale.
>
>  * Fix the exclusion filter so that collections that start with js,
> css, img, tpl can be accessed.
>
>  * Resolve XSS issue in Admin UI stats page
>
>
> Known issues:
>
>  * On Windows, bin/solr.cmd script fails to start correctly when using
> relative path with -s parameter. Use absolute path as a workaround .
> https://issues.apache.org/jira/browse/SOLR-8073
>
>
> See the CHANGES.txt file included with the release for a full list of
> changes and further details.
>
>
> Please report any feedback to the mailing lists
> (http://lucene.apache.org/solr/discussion.html)
>
>
> Note: The Apache Software Foundation uses an extensive mirroring
> network for distributing releases. It is possible that the mirror you
> are using may not have replicated the release yet. If that is the
> case, please try another mirror. This also goes for Maven access.
>
> Noble Paul
> on behalf of Lucene PMC



-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906691#comment-14906691
 ] 

ASF subversion and git services commented on SOLR-8091:
---

Commit 1705115 from [~steve_rowe] in branch 'cms/trunk'
[ https://svn.apache.org/r1705115 ]

SOLR-8091: Make sub-bullets under feature descriptions darker: #D9A08E -> 
#CB9685

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8085) ChaosMonkey Safe Leader Test fail with shard inconsistency.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-8085:
--
Attachment: SOLR-8085.patch

Here is a patch I just ported to trunk that starts buffering before peer sync 
and before we publish as RECOVERING. 

With [~ysee...@gmail.com]'s fix and the couple I have, I've been able to hit 
200 HdfsChaosMonkey test runs without replica inconsistency on my 4.10.3 + 
backports based code for the first time.

> ChaosMonkey Safe Leader Test fail with shard inconsistency.
> ---
>
> Key: SOLR-8085
> URL: https://issues.apache.org/jira/browse/SOLR-8085
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
> Attachments: SOLR-8085.patch, SOLR-8085.patch, fail.150922_125320, 
> fail.150922_130608
>
>
> I've been discussing this fail I found with Yonik.
> The problem seems to be that a replica tries to recover and publishes 
> recovering - the attempt then fails, but docs are now coming in from the 
> leader. The replica tries to recover again and has gotten enough docs to pass 
> peery sync.
> I'm trying a possible solution now where we won't allow peer sync after a 
> recovery that is not successful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8094) HdfsUpdateLog should not replay buffered documents as a replacement to dropping them.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-8094:
--
Attachment: SOLR-8094.patch

Patch attached.

> HdfsUpdateLog should not replay buffered documents as a replacement to 
> dropping them.
> -
>
> Key: SOLR-8094
> URL: https://issues.apache.org/jira/browse/SOLR-8094
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Attachments: SOLR-8094.patch
>
>
> This can cause replicas to get out of sync because replication may have 
> failed, but then if you replay you can setup peer sync to pass when it should 
> not.
> After talking to Yonik a bit, I tried doing everything the same as on local 
> fs except skipping the truncate - because of error handling we already have, 
> this appears to work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8093) `bin/solr create_core` syntax doesn't match documentation for '-d' option

[Mike Mallett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906888#comment-14906888
 ] 

Mike Mallett commented on SOLR-8093:


The command used is correct and works properly if executed as the "solr" user

bin/solr create_core -c testcore -d 
/opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x

> `bin/solr create_core` syntax doesn't match documentation for '-d' option
> -
>
> Key: SOLR-8093
> URL: https://issues.apache.org/jira/browse/SOLR-8093
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.3
>Reporter: Mike Mallett
>
> I'm attempting to create a new Solr Core using an existing set of config 
> files. Specifically these are being taken from the Drupal module 
> search_api_solr-7.x-1.9 in this tarball: 
> http://ftp.drupal.org/files/projects/search_api_solr-7.x-1.9.tar.gz
> The config dir located in the tarball is: search_api_solr/solr-conf/5.x/
> This works fine, as I was able to copy this config directory in the shell 
> then use the Solr Admin UI to create the core. No errors were reported in 
> that methodology.
> However, I'm reading the help documentation included by executing the command 
> `bin/solr create_core -help` which states:
> "-d   Configuration directory to copy when creating the new core, 
> built-in options are:
>   basic_configs: Minimal Solr configuration
>   data_driven_schema_configs: Managed schema with field-guessing support 
> enabled
>   sample_techproducts_configs: Example configuration with many optional 
> features enabled to demonstrate the full power of Solr
>   If not specified, default is: data_driven_schema_configs
>   Alternatively, you can pass the path to your own configuration 
> directory instead of using one of the built-in configurations, such as: 
> bin/solr create_core -c mycore -d /tmp/myconfig"
> So I tried the following command:
> bin/solr create_core -c testcore -d 
> /opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x
> This resulted in:
> ERROR: Error CREATEing SolrCore 'testcore': Unable to create core [testcore] 
> Caused by: /var/solr5/data/testcore/data
> I was instructed on IRC that I am using the command incorrectly and that 
> valid options for '-d' are only "basic_configs", 
> "data_driven_schema_configs",  or "sample_techproducts_configs"
> I am unable to create a core from the command line, and I don't know what the 
> process should look like.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8093) `bin/solr create_core` syntax doesn't match documentation for '-d' option

[Mike Mallett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Mallett updated SOLR-8093:
---
Issue Type: Improvement  (was: Bug)

> `bin/solr create_core` syntax doesn't match documentation for '-d' option
> -
>
> Key: SOLR-8093
> URL: https://issues.apache.org/jira/browse/SOLR-8093
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 5.3
>Reporter: Mike Mallett
>
> I'm attempting to create a new Solr Core using an existing set of config 
> files. Specifically these are being taken from the Drupal module 
> search_api_solr-7.x-1.9 in this tarball: 
> http://ftp.drupal.org/files/projects/search_api_solr-7.x-1.9.tar.gz
> The config dir located in the tarball is: search_api_solr/solr-conf/5.x/
> This works fine, as I was able to copy this config directory in the shell 
> then use the Solr Admin UI to create the core. No errors were reported in 
> that methodology.
> However, I'm reading the help documentation included by executing the command 
> `bin/solr create_core -help` which states:
> "-d   Configuration directory to copy when creating the new core, 
> built-in options are:
>   basic_configs: Minimal Solr configuration
>   data_driven_schema_configs: Managed schema with field-guessing support 
> enabled
>   sample_techproducts_configs: Example configuration with many optional 
> features enabled to demonstrate the full power of Solr
>   If not specified, default is: data_driven_schema_configs
>   Alternatively, you can pass the path to your own configuration 
> directory instead of using one of the built-in configurations, such as: 
> bin/solr create_core -c mycore -d /tmp/myconfig"
> So I tried the following command:
> bin/solr create_core -c testcore -d 
> /opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x
> This resulted in:
> ERROR: Error CREATEing SolrCore 'testcore': Unable to create core [testcore] 
> Caused by: /var/solr5/data/testcore/data
> I was instructed on IRC that I am using the command incorrectly and that 
> valid options for '-d' are only "basic_configs", 
> "data_driven_schema_configs",  or "sample_techproducts_configs"
> I am unable to create a core from the command line, and I don't know what the 
> process should look like.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe resolved SOLR-8091.
--
Resolution: Fixed

I did a bulk change to #33, then tweaked a couple things that fell out of 
that:

* sub-bullets on the Features page looked out of place as dark gray under 
orange text, so I made them a grayer orange than the parent bullets
* the floating header on the Resources page uses font darkening on hover, and 
at #33, even darkening to full black (#00) wasn't a noticeable enough 
change when hovering, so I lightened it up to #66 (first I tried #55, 
which wasn't enough)

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8093) `bin/solr create_core` syntax doesn't match documentation for '-d' option

[Mike Mallett (JIRA)]

Mike Mallett created SOLR-8093:
--

 Summary: `bin/solr create_core` syntax doesn't match documentation 
for '-d' option
 Key: SOLR-8093
 URL: https://issues.apache.org/jira/browse/SOLR-8093
 Project: Solr
  Issue Type: Bug
Affects Versions: 5.3
Reporter: Mike Mallett


I'm attempting to create a new Solr Core using an existing set of config files. 
Specifically these are being taken from the Drupal module 
search_api_solr-7.x-1.9 in this tarball: 
http://ftp.drupal.org/files/projects/search_api_solr-7.x-1.9.tar.gz

The config dir located in the tarball is: search_api_solr/solr-conf/5.x/

This works fine, as I was able to copy this config directory in the shell then 
use the Solr Admin UI to create the core. No errors were reported in that 
methodology.

However, I'm reading the help documentation included by executing the command 
`bin/solr create_core -help` which states:

"-d   Configuration directory to copy when creating the new core, 
built-in options are:
  basic_configs: Minimal Solr configuration
  data_driven_schema_configs: Managed schema with field-guessing support 
enabled
  sample_techproducts_configs: Example configuration with many optional 
features enabled to demonstrate the full power of Solr

  If not specified, default is: data_driven_schema_configs

  Alternatively, you can pass the path to your own configuration directory 
instead of using one of the built-in configurations, such as: bin/solr 
create_core -c mycore -d /tmp/myconfig"

So I tried the following command:

bin/solr create_core -c testcore -d 
/opt/drupal/search_api_solr-7.x-1.9/solr-conf/5.x

This resulted in:

ERROR: Error CREATEing SolrCore 'testcore': Unable to create core [testcore] 
Caused by: /var/solr5/data/testcore/data

I was instructed on IRC that I am using the command incorrectly and that valid 
options for '-d' are only "basic_configs", "data_driven_schema_configs",  or 
"sample_techproducts_configs"

I am unable to create a core from the command line, and I don't know what the 
process should look like.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8085) ChaosMonkey Safe Leader Test fail with shard inconsistency.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906838#comment-14906838
 ] 

Mark Miller commented on SOLR-8085:
---

For the last comment I filed SOLR-8094.

> ChaosMonkey Safe Leader Test fail with shard inconsistency.
> ---
>
> Key: SOLR-8085
> URL: https://issues.apache.org/jira/browse/SOLR-8085
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
> Attachments: SOLR-8085.patch, fail.150922_125320, fail.150922_130608
>
>
> I've been discussing this fail I found with Yonik.
> The problem seems to be that a replica tries to recover and publishes 
> recovering - the attempt then fails, but docs are now coming in from the 
> leader. The replica tries to recover again and has gotten enough docs to pass 
> peery sync.
> I'm trying a possible solution now where we won't allow peer sync after a 
> recovery that is not successful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906963#comment-14906963
 ] 

ASF subversion and git services commented on SOLR-8091:
---

Commit 1705141 from [~steve_rowe] in branch 'cms/trunk'
[ https://svn.apache.org/r1705141 ]

SOLR-8091: Darken feature descriptions: #A13016 -> #33 (but leave the 
sub-bullet explanations at #A13016)

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Solaris (64bit/jdk1.8.0) - Build # 76 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Solaris/76/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([1C243A739ABC9FE3:F57E814B04250F4B]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:765)
at 
org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir(TestArbitraryIndexDir.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=*[count(//doc)=1]
xml response was: 

01


request was:q=id:2=standard=0=20=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:758)
... 40 more




Build Log:
[...truncated 9388 lines...]
   [junit4] Suite: 

[jira] [Commented] (LUCENE-6814) PatternTokenizer should free heap after it's done

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907036#comment-14907036
 ] 

Michael McCandless commented on LUCENE-6814:


bq.  Is 4.10 dead now?

It's not that it's dead, it's more that there would need to be a really really 
bad bug to warrant another 4.10.x release at this point.  We are already well 
into 5.x, and just released 5.3.1 bug fix today: 
http://lucene.apache.org/#24-september-2015-apache-lucene-531-and-apache-solr-531-available

bq. I'll probably have to push for the PatternTokenizer fork to get it into 1.x?

Yes, maybe ... or patch your ES locally with the change?

bq.  and PatternTokenizer ends up retaining around 2gb

Hmm, how many {{PatternTokenizer}} instances do you have?  With 24 bulk 
indexing threads you should (I think?) only have at most 24 instances * 4 MB 
max should be ~100 MB.

bq. Does everyone just like huge heaps (or just not use PatternAnalyzer)?

I suspect {{PatternTokenizer}} is not commonly used ...

There is a TODO in the code to fix this class to not hold an entire copy of the 
document ...

bq. I think this ends up growing to the max sized field. 

OK I'll put that in the CHANGES when I commit, and fix this issue title.

> PatternTokenizer should free heap after it's done
> -
>
> Key: LUCENE-6814
> URL: https://issues.apache.org/jira/browse/LUCENE-6814
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6814.patch, LUCENE-6814.patch
>
>
> Caught by Alex Chow in this Elasticsearch issue: 
> https://github.com/elastic/elasticsearch/issues/13721
> Today, PatternTokenizer reuses a single StringBuilder, but it doesn't free 
> its heap usage after tokenizing is done.  We can either stop reusing, or ask 
> it to {{.trimToSize}} when we are done ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8091) Solr Website Fails WCAG Contrast Ratio

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906951#comment-14906951
 ] 

ASF subversion and git services commented on SOLR-8091:
---

Commit 1705137 from [~steve_rowe] in branch 'cms/trunk'
[ https://svn.apache.org/r1705137 ]

SOLR-8091: Darken feature descriptions: #D9411E -> #A13016

> Solr Website Fails WCAG Contrast Ratio
> --
>
> Key: SOLR-8091
> URL: https://issues.apache.org/jira/browse/SOLR-8091
> Project: Solr
>  Issue Type: Improvement
>  Components: website
>Reporter: Mike Mallett
>Assignee: Steve Rowe
>
> The Solr website http://lucene.apache.org/solr/ has a lot of text which fails 
> WCAG Accessibility Guidelines for Contrast Ratios.
> The site places a lot of text with colour #959693 on a white background This 
> fails WCAG Accessibility Guidelines for Contrast Ratio as checked on this 
> site: http://webaim.org/resources/contrastchecker/
> Please use a more readable font colour for text on the Apache Solr website.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8095) Allow disabling HDFS Locality Metrics

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-8095:

Attachment: SOLR-8095.patch

> Allow disabling HDFS Locality Metrics
> -
>
> Key: SOLR-8095
> URL: https://issues.apache.org/jira/browse/SOLR-8095
> Project: Solr
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Mike Drob
>  Labels: metrics
> Fix For: Trunk
>
> Attachments: SOLR-8095.patch
>
>
> We added metrics, but not a way to configure/turn them off.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8095) Allow disabling HDFS Locality Metrics

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907242#comment-14907242
 ] 

Mark Miller commented on SOLR-8095:
---

Normally I'd worry about back compat, but probably okay in this case?

> Allow disabling HDFS Locality Metrics
> -
>
> Key: SOLR-8095
> URL: https://issues.apache.org/jira/browse/SOLR-8095
> Project: Solr
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Mike Drob
>  Labels: metrics
> Fix For: Trunk
>
> Attachments: SOLR-8095.patch
>
>
> We added metrics, but not a way to configure/turn them off.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906959#comment-14906959
 ] 

Mike Drob commented on SOLR-8075:
-

Patch LGTM.

On commit, can you add a comment to {{waitForReplicasToComeUp}} explaining what 
the return value means? It makes sense now, but in six months I'm pretty sure I 
won't have a clue.

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org