[jira] [Commented] (LUCENE-8951) Create issues@ and builds@ lists and update notifications
[ https://issues.apache.org/jira/browse/LUCENE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929317#comment-16929317 ] Steve Rowe commented on LUCENE-8951: bq. I will take care of ASF Jenkins after lunch. I am not sure about status of Steve Rowe and Elasticsearch Jenkins. My Jenkins doesn't email any lists. > Create issues@ and builds@ lists and update notifications > - > > Key: LUCENE-8951 > URL: https://issues.apache.org/jira/browse/LUCENE-8951 > Project: Lucene - Core > Issue Type: Task >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > > Issue to plan and execute decision from dev mailing list > [https://lists.apache.org/thread.html/762d72a9045642dc488dc7a2fd0a525707e5fa5671ac0648a3604c9b@%3Cdev.lucene.apache.org%3E] > # Create mailing lists as an announce only list (/) > # Subscribe all emails that will be allowed to post (/) > # Update websites with info about the new lists (/) > # Announce to dev@ list that the change will happen (/) > # Modify Jira and Github bots to post to issues@ list instead of dev@ > # Modify Jenkins (including Policeman and other) to post to builds@ > # Announce to dev@ list that the change is effective -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13746) Apache jenkins needs JVM 11 upgraded to at least 11.0.3 (SSL bugs)
[
https://issues.apache.org/jira/browse/SOLR-13746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926227#comment-16926227
]
Steve Rowe commented on SOLR-13746:
---
bq. Can/should we be filing an INFRA jira explicitly requesting upgraded JDKs
so it's clear there is a demonstrable need? (does such an issue already exist
already? can someone please link it here?)
I looked and didn't find such an issue, so I created one:
https://issues.apache.org/jira/browse/INFRA-19009
> Apache jenkins needs JVM 11 upgraded to at least 11.0.3 (SSL bugs)
> --
>
> Key: SOLR-13746
> URL: https://issues.apache.org/jira/browse/SOLR-13746
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Hoss Man
>Priority: Major
>
> I just realized that back in June, there was a misscommunication between
> myself & Uwe (and a lack of double checking on my part!) regarding upgrading
> the JVM versions on our jenkins machines...
> *
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/201906.mbox/%3calpine.DEB.2.11.1906181434350.23523@tray%3e]
> *
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/201906.mbox/%3C00b301d52918$d27b2f60$77718e20$@thetaphi.de%3E]
> ...Uwe only updated the JVMs on _his_ policeman jenkins machines - the JVM
> used on the _*apache*_ jenkins nodes is still (as of 2019-09-06)
> "11.0.1+13-LTS" ...
> [https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-Tests-master/3689/consoleText]
> {noformat}
> ...
> [java-info] java version "11.0.1"
> [java-info] Java(TM) SE Runtime Environment (11.0.1+13-LTS, Oracle
> Corporation)
> [java-info] Java HotSpot(TM) 64-Bit Server VM (11.0.1+13-LTS, Oracle
> Corporation)
> ...
> {noformat}
> This means that even after the changes made in SOLR-12988 to re-enable SSL
> testing on java11, all Apache jenkins 'master' builds, (including, AFAICT the
> yetus / 'Patch Review' builds) are still SKIPping thousands of tests that use
> SSL (either explicitly, or due to randomization) becauseof the logic in
> SSLTestConfig to detects bad JVM versions an prevent confusion/spurious
> failures.
> We really need to get the jenkins nodes updated to openjdk 11.0.3 or 11.0.4
> ASAP.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13746) Apache jenkins needs JVM 11 upgraded to at least 11.0.3 (SSL bugs)
[
https://issues.apache.org/jira/browse/SOLR-13746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925242#comment-16925242
]
Steve Rowe commented on SOLR-13746:
---
Uwe and I have login credentials on the two ASF Jenkins nodes, and we can sudo
as the jenkins user on those nodes. The {{/home/jenkins/tools/}} directory
contains symlinks to the installed JDKs, so theoretically, yes, Uwe and I could
overwrite those links. But AFAIK Infra manages these and other stuff in the
tools directory via Puppet, so manual change there could cause trouble. I
think we should get Infra involved, at a minimum to ask if we should be
managing JDKs on a self-serve basis.
>From the lucene1 node:
{noformat}
jenkins@lucene1-us-west:~$ ls -l /home/jenkins/tools/java/latest11
lrwxrwxrwx 1 root root 38 Nov 10 2018 /home/jenkins/tools/java/latest11 ->
/usr/local/asfpackages/java/jdk-11.0.1
jenkins@lucene1-us-west:~$ ls -ld /usr/local/asfpackages/java/*11*
drwxr-xr-x 8 root root 4096 Jul 19 18:40
/usr/local/asfpackages/java/jdk-11.0.1
-rw-r--r-- 1 root root 179758537 Nov 10 2018
/usr/local/asfpackages/java/jdk-11.0.1_linux-x64_bin.tar.gz
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+19
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+22
drwxr-xr-x 8 root root 4096 Jul 19 18:40
/usr/local/asfpackages/java/jdk-11-ea+28
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+8
{noformat}
I see the same thing ^^ on the lucene2 node. When I go to configure the JDK on
ASF Jenkins jobs, for Java11 I see a preset list that looks like the above.
FYI I upgraded the JDK used by my Jenkins's lucene-solr master branch jobs to
11.0.3 a while back.
> Apache jenkins needs JVM 11 upgraded to at least 11.0.3 (SSL bugs)
> --
>
> Key: SOLR-13746
> URL: https://issues.apache.org/jira/browse/SOLR-13746
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Hoss Man
>Priority: Major
>
> I just realized that back in June, there was a misscommunication between
> myself & Uwe (and a lack of double checking on my part!) regarding upgrading
> the JVM versions on our jenkins machines...
> *
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/201906.mbox/%3calpine.DEB.2.11.1906181434350.23523@tray%3e]
> *
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/201906.mbox/%3C00b301d52918$d27b2f60$77718e20$@thetaphi.de%3E]
> ...Uwe only updated the JVMs on _his_ policeman jenkins machines - the JVM
> used on the _*apache*_ jenkins nodes is still (as of 2019-09-06)
> "11.0.1+13-LTS" ...
> [https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-Tests-master/3689/consoleText]
> {noformat}
> ...
> [java-info] java version "11.0.1"
> [java-info] Java(TM) SE Runtime Environment (11.0.1+13-LTS, Oracle
> Corporation)
> [java-info] Java HotSpot(TM) 64-Bit Server VM (11.0.1+13-LTS, Oracle
> Corporation)
> ...
> {noformat}
> This means that even after the changes made in SOLR-12988 to re-enable SSL
> testing on java11, all Apache jenkins 'master' builds, (including, AFAICT the
> yetus / 'Patch Review' builds) are still SKIPping thousands of tests that use
> SSL (either explicitly, or due to randomization) becauseof the logic in
> SSLTestConfig to detects bad JVM versions an prevent confusion/spurious
> failures.
> We really need to get the jenkins nodes updated to openjdk 11.0.3 or 11.0.4
> ASAP.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13746) Apache jenkins needs JVM 11 upgraded to at least 11.0.3 (SSL bugs)
[
https://issues.apache.org/jira/browse/SOLR-13746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925242#comment-16925242
]
Steve Rowe edited comment on SOLR-13746 at 9/8/19 8:12 PM:
---
Uwe and I have login credentials on the two ASF Jenkins nodes dedicated to
Lucene/Solr, and we can sudo as the jenkins user on those nodes. The
{{/home/jenkins/tools/}} directory contains symlinks to the installed JDKs, so
theoretically, yes, Uwe and I could overwrite those links. But AFAIK Infra
manages these and other stuff in the tools directory via Puppet, so manual
change there could cause trouble. I think we should get Infra involved, at a
minimum to ask if we should be managing JDKs on a self-serve basis.
>From the lucene1 node:
{noformat}
jenkins@lucene1-us-west:~$ ls -l /home/jenkins/tools/java/latest11
lrwxrwxrwx 1 root root 38 Nov 10 2018 /home/jenkins/tools/java/latest11 ->
/usr/local/asfpackages/java/jdk-11.0.1
jenkins@lucene1-us-west:~$ ls -ld /usr/local/asfpackages/java/*11*
drwxr-xr-x 8 root root 4096 Jul 19 18:40
/usr/local/asfpackages/java/jdk-11.0.1
-rw-r--r-- 1 root root 179758537 Nov 10 2018
/usr/local/asfpackages/java/jdk-11.0.1_linux-x64_bin.tar.gz
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+19
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+22
drwxr-xr-x 8 root root 4096 Jul 19 18:40
/usr/local/asfpackages/java/jdk-11-ea+28
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+8
{noformat}
I see the same thing ^^ on the lucene2 node. When I go to configure the JDK on
ASF Jenkins jobs, for Java11 I see a preset list that looks like the above.
FYI I upgraded the JDK used by my Jenkins's lucene-solr master branch jobs to
11.0.3 a while back.
was (Author: steve_rowe):
Uwe and I have login credentials on the two ASF Jenkins nodes, and we can sudo
as the jenkins user on those nodes. The {{/home/jenkins/tools/}} directory
contains symlinks to the installed JDKs, so theoretically, yes, Uwe and I could
overwrite those links. But AFAIK Infra manages these and other stuff in the
tools directory via Puppet, so manual change there could cause trouble. I
think we should get Infra involved, at a minimum to ask if we should be
managing JDKs on a self-serve basis.
>From the lucene1 node:
{noformat}
jenkins@lucene1-us-west:~$ ls -l /home/jenkins/tools/java/latest11
lrwxrwxrwx 1 root root 38 Nov 10 2018 /home/jenkins/tools/java/latest11 ->
/usr/local/asfpackages/java/jdk-11.0.1
jenkins@lucene1-us-west:~$ ls -ld /usr/local/asfpackages/java/*11*
drwxr-xr-x 8 root root 4096 Jul 19 18:40
/usr/local/asfpackages/java/jdk-11.0.1
-rw-r--r-- 1 root root 179758537 Nov 10 2018
/usr/local/asfpackages/java/jdk-11.0.1_linux-x64_bin.tar.gz
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+19
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+22
drwxr-xr-x 8 root root 4096 Jul 19 18:40
/usr/local/asfpackages/java/jdk-11-ea+28
drwxr-xr-x 8 root root 4096 Jul 19 18:39
/usr/local/asfpackages/java/jdk-11-ea+8
{noformat}
I see the same thing ^^ on the lucene2 node. When I go to configure the JDK on
ASF Jenkins jobs, for Java11 I see a preset list that looks like the above.
FYI I upgraded the JDK used by my Jenkins's lucene-solr master branch jobs to
11.0.3 a while back.
> Apache jenkins needs JVM 11 upgraded to at least 11.0.3 (SSL bugs)
> --
>
> Key: SOLR-13746
> URL: https://issues.apache.org/jira/browse/SOLR-13746
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Hoss Man
>Priority: Major
>
> I just realized that back in June, there was a misscommunication between
> myself & Uwe (and a lack of double checking on my part!) regarding upgrading
> the JVM versions on our jenkins machines...
> *
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/201906.mbox/%3calpine.DEB.2.11.1906181434350.23523@tray%3e]
> *
> [http://mail-archives.apache.org/mod_mbox/lucene-dev/201906.mbox/%3C00b301d52918$d27b2f60$77718e20$@thetaphi.de%3E]
> ...Uwe only updated the JVMs on _his_ policeman jenkins machines - the JVM
> used on the _*apache*_ jenkins nodes is still (as of 2019-09-06)
> "11.0.1+13-LTS" ...
> [https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-Tests-master/3689/consoleText]
> {noformat}
> ...
> [java-info] java version "11.0.1"
> [java-info] Java(TM) SE Runtime Environment (11.0.1+13-LTS, Oracle
> Corporation)
> [java-info] Java HotSpot(TM) 64-Bit Server VM (11.0.1+13-LTS, Oracle
> Corporation)
> ...
> {noformat}
> This means that even after the changes made in SOLR-12988 to
[jira] [Commented] (SOLR-13593) Allow to specify analyzer components by their SPI names in schema definition
[
https://issues.apache.org/jira/browse/SOLR-13593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900067#comment-16900067
]
Steve Rowe commented on SOLR-13593:
---
bq. Both name & class seems error prone; can't we simply disallow this (throw
error) from the start?
+1
> Allow to specify analyzer components by their SPI names in schema definition
>
>
> Key: SOLR-13593
> URL: https://issues.apache.org/jira/browse/SOLR-13593
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
>Reporter: Tomoko Uchida
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Now each analysis factory has explicitely documented SPI name which is stored
> in the static "NAME" field (LUCENE-8778).
> Solr uses factories' simple class name in schema definition (like
> class="solr.WhitespaceTokenizerFactory"), but we should be able to also use
> more concise SPI names (like name="whitespace").
> e.g.:
> {code:xml}
>
>
>
> />
>
>
>
> {code}
> would be
> {code:xml}
>
>
>
>
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6468) Regression: StopFilterFactory doesn't work properly without deprecated enablePositionIncrements="false"
[
https://issues.apache.org/jira/browse/SOLR-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900057#comment-16900057
]
Steve Rowe commented on SOLR-6468:
--
Hi Alexander,
I'm not sure about the performance impact, you'd have to test to see how it
performs on your own data.
The only downside I know of: Since you're removing content prior to
tokenization, if the boundaries you use for MappingCharFilter are not the same
as those used in tokenization, or if your replacement string impacts
tokenization, you may see some differences from the behavior of your analysis
chain when using StopFilter. My recommendation: test using some real world
data.
> Regression: StopFilterFactory doesn't work properly without deprecated
> enablePositionIncrements="false"
> ---
>
> Key: SOLR-6468
> URL: https://issues.apache.org/jira/browse/SOLR-6468
> Project: Solr
> Issue Type: Bug
> Components: Schema and Analysis
>Affects Versions: 4.8.1, 4.9, 5.3.1, 6.6.2, 7.1
>Reporter: Alexander S.
>Priority: Major
> Attachments: FieldValue.png
>
>
> Setup:
> * Schema version is 1.5
> * Field config:
> {code}
> autoGeneratePhraseQueries="true">
>
>
> ignoreCase="true" />
>
>
>
> {code}
> * Stop words:
> {code}
> http
> https
> ftp
> www
> {code}
> So very simple. In the index I have:
> * twitter.com/testuser
> All these queries do match:
> * twitter.com/testuser
> * com/testuser
> * testuser
> But none of these does:
> * https://twitter.com/testuser
> * https://www.twitter.com/testuser
> * www.twitter.com/testuser
> Debug output shows:
> "parsedquery_toString": "+(url_words_ngram:\"? twitter com testuser\")"
> But we need:
> "parsedquery_toString": "+(url_words_ngram:\"twitter com testuser\")"
> Complete debug outputs:
> * a valid search:
> http://pastie.org/pastes/9500661/text?key=rgqj5ivlgsbk1jxsudx9za
> * an invalid search:
> http://pastie.org/pastes/9500662/text?key=b4zlh2oaxtikd8jvo5xaww
> The complete discussion and explanation of the problem is here:
> http://lucene.472066.n3.nabble.com/Help-with-StopFilterFactory-td4153839.html
> I didn't find a clear explanation how can we upgrade Solr, there's no any
> replacement or a workarround to this, so this is not just a major change but
> a major disrespect to all existing Solr users who are using this feature.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8937) Avoid agressive stemming on numbers in the FrenchMinimalStemmer
[ https://issues.apache.org/jira/browse/LUCENE-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895358#comment-16895358 ] Steve Rowe commented on LUCENE-8937: +1 for the change. > Avoid agressive stemming on numbers in the FrenchMinimalStemmer > --- > > Key: LUCENE-8937 > URL: https://issues.apache.org/jira/browse/LUCENE-8937 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Gallou >Priority: Minor > Attachments: 0001-adds-test-cases-on-french-minimal-stemmer.patch, > 0002-check-if-the-last-character-is-a-letter-before-remov.patch, > SOLR-8937.patch > > > Here is the discussion on the mailing list : > [http://mail-archives.apache.org/mod_mbox/lucene-java-user/201907.mbox/browser] > The light stemmer removes the last character of a word if the last two > characters are identical. > We can see that here: > > [https://github.com/apache/lucene-solr/blob/master/lucene/analysis/common/src/java/org/apache/lucene/analysis/fr/FrenchLightStemmer.java#L263] > In this light stemmer, there is a check to avoid altering the token if the > token is a number. > The minimal stemmer also removes the last character of a word if the last > two characters are identical. > We can see that here: > > [https://github.com/apache/lucene-solr/blob/master/lucene/analysis/common/src/java/org/apache/lucene/analysis/fr/FrenchMinimalStemmer.java#L77] > But in this minimal stemmer there is no check to see if the character is a > letter or not. > So when we have numeric tokens with the last two characters identical they > are altered. > For example "1234567899" will be stemmed as "123456789". > It could be great of it's not altered. > Here is the same issue for the LightStemmer : > https://issues.apache.org/jira/browse/LUCENE-4063 -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12532) Slop specified in query string is not preserved for certain phrase searches
[ https://issues.apache.org/jira/browse/SOLR-12532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893979#comment-16893979 ] Steve Rowe commented on SOLR-12532: --- Sure, I'll take a look in the next couple days. > Slop specified in query string is not preserved for certain phrase searches > --- > > Key: SOLR-12532 > URL: https://issues.apache.org/jira/browse/SOLR-12532 > Project: Solr > Issue Type: Bug > Components: query parsers >Affects Versions: 7.4 >Reporter: Brad Sumersford >Priority: Major > Attachments: SOLR-12532.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Note: This only impacts specific settings for the WordDelimiterGraphFilter as > detailed below. > When a phrase search is parsed by the SolrQueryParser, and the phrase search > results in a graph token stream, the resulting SpanNearQuery created does not > have the slop correctly set. > h4. Conditions > - Slop provided in query string (ex: ~2") > - WordDelimiterGraphFilterFactory with query time preserveOriginal and > generateWordParts > - query string includes a term that contains a word delimiter > h4. Example > Field: wdf_partspreserve > – WordDelimiterGraphFilterFactory > preserveOriginal="1" > generateWordParts="1" > Data: you just can't > Search: wdf_partspreserve:"you can't"~2 -> 0 Results > h4. Cause > The slop supplied by the query string is applied in > SolrQueryParserBase#getFieldQuery which will set the slop only for > PhraseQuery and MultiPhaseQuery. Since "can't" will be broken down into > multiple tokens analyzeGraphPhrase will be triggered when the Query is being > constructed which will return a SpanNearQuery instead of a (Multi)PhraseQuery. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8925) Test2BPostingsBytes org.apache.lucene.index.CorruptIndexException: docs out of order (490879719 <= 490879719 )
[
https://issues.apache.org/jira/browse/LUCENE-8925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888247#comment-16888247
]
Steve Rowe commented on LUCENE-8925:
I tested the repro line (after adding {{-Dtests.monster=true}}) at the tip of
branch_8x (commit {{eb75a60857deb96c55a2d79cdb4cdabf4a0fda1b}}) with {{openjdk
version "1.8.0_171"}}:
{noformat}
$ ant test -Dtestcase=Test2BPostingsBytes -Dtests.method=test
-Dtests.seed=1C14F78FC0AF1835 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=fr -Dtests.timezone=SystemV/AST4ADT -Dtests.asserts=true
-Dtests.file.encoding=UTF-8 -Dtests.monster=true
[...]
[junit4] Suite: org.apache.lucene.index.Test2BPostingsBytes
[junit4] HEARTBEAT J0 PID(16237@localhost): 2019-07-18T11:29:13, stalled for
71.2s at: Test2BPostingsBytes.test
[junit4] HEARTBEAT J0 PID(16237@localhost): 2019-07-18T11:30:13, stalled for
131s at: Test2BPostingsBytes.test
[junit4] HEARTBEAT J0 PID(16237@localhost): 2019-07-18T11:31:13, stalled for
191s at: Test2BPostingsBytes.test
[junit4] OK 222s | Test2BPostingsBytes.test
[junit4] Completed [1/1] in 222.41s, 1 test
[junit4]
[junit4] JVM J0: 0.77 .. 223.76 = 222.99s
[junit4] Execution time total: 3 minutes 43 seconds
[junit4] Tests summary: 1 suite, 1 test
[...]
BUILD SUCCESSFUL
Total time: 3 minutes 46 seconds
{noformat}
I also tested at the same commit as the OP
({{081e2ef2c05e017e87a2aef2a4f55067fbba5cb4}}), with the same result: {{BUILD
SUCCESSFUL}}. (The first time I tried this resulted in OOM, but succeeded on
the second try, after adding {{-Dtests.heapsize=30g}} to the cmdline, which may
be overkill here, but is what I use on my Jenkins jobs that run the monster
tests once a week.)
> Test2BPostingsBytes org.apache.lucene.index.CorruptIndexException: docs out
> of order (490879719 <= 490879719 )
> --
>
> Key: LUCENE-8925
> URL: https://issues.apache.org/jira/browse/LUCENE-8925
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.1.1
> Environment: RHEL-7.3 (ppc64le - Power9)
> kernel 3.10.0-957.21.3.el7.ppc64le
> 48G vm, 64 core
> java version "1.8.0_211"
> Java(TM) SE Runtime Environment (build 8.0.5.37 -
> pxl6480sr5fp37-20190618_01(SR5 FP37))
> IBM J9 VM (build 2.9, JRE 1.8.0 Linux ppc64le-64-Bit Compressed References
> 20190617_419755 (JIT enabled, AOT enabled)
> OpenJ9 - 354b31d
> OMR - 0437c69
> IBM - 4972efe)
> JCL - 20190606_01 based on Oracle jdk8u211-b25
>Reporter: Daniel Black
>Priority: Major
> Labels: test-failure
>
> 8x branch at commit 081e2ef2c05e017e87a2aef2a4f55067fbba5cb4
> while running {{ant -Dtests.filter=(@monster or @slow) and not(@awaitsfix)
> -Dtests.heapsize=4G -Dtests.jvms=64 test}}
> {noformat}
> 2> NOTE: reproduce with: ant test -Dtestcase=Test2BPostingsBytes
> -Dtests.method=test -Dtests.seed=1C14F78FC0AF1835 -Dtests.slow=true
> -Dtests.badapples=true -Dtests.locale=fr
> -Dtests.timezone=SystemV/AST4ADT -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> [23:54:00.627] ERROR111s J52 | Test2BPostingsBytes.test <<<
>> Throwable #1: org.apache.lucene.index.CorruptIndexException: docs out of
> order (490879719 <= 490879719 )
> (resource=MockIndexOutputWrapper(FSIndexOutput(path="/home/danielgb
> /lucene-solr/lucene/build/core/test/J52/temp/lucene.index.Test2BPostingsBytes_1C14F78FC0AF1835-001/2BPostingsBytes3-001/_0_Lucene50_0.doc")))
>> at
> __randomizedtesting.SeedInfo.seed([1C14F78FC0AF1835:9440C8556E5375CD]:0)
>> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsWriter.startDoc(Lucene50PostingsWriter.java:236)
>> at
> org.apache.lucene.codecs.PushPostingsWriterBase.writeTerm(PushPostingsWriterBase.java:148)
>> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.write(BlockTreeTermsWriter.java:865)
>> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.write(BlockTreeTermsWriter.java:344)
>> at
> org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
>> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:169)
>> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:245)
>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:140)
>> at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2988)
>> at org.apache.lucene.util.TestUtil.addIndexesSlowly(TestUtil.java:990)
>> at
> org.apache.lucene.index.Test2BPostingsBytes.test(Test2BPostingsBytes.java:127)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
>
[jira] [Commented] (LUCENE-8926) Test2BDocs.test2BDocs error java.lang.ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/LUCENE-8926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888124#comment-16888124
]
Steve Rowe commented on LUCENE-8926:
I tested the repo line (after adding {{-Dtests.monster=true}}) at the tip of
branch_8x (commit {{eb75a60857deb96c55a2d79cdb4cdabf4a0fda1b}}) with {{openjdk
version "1.8.0_171"}}:
{noformat}
$ ant test -Dtestcase=Test2BDocs -Dtests.method=test2BDocs
-Dtests.seed=ECB69064FDA0F2A6 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=zh -Dtests.timezone=Europe/Zurich -Dtests.asserts=true
-Dtests.file.encoding=UTF-8 -Dtests.monster=true
[...]
[junit4] Suite: org.apache.lucene.index.Test2BDocs
[junit4] 1> indexed: 0
[junit4] 1> indexed: 1000
[junit4] 1> indexed: 2000
[...]
[junit4] HEARTBEAT J0 PID(16332@localhost): 2019-07-18T11:40:46, stalled for
732s at: Test2BDocs.test2BDocs
[junit4] 1> indexed: 213000
[junit4] 1> indexed: 214000
[junit4] HEARTBEAT J0 PID(16332@localhost): 2019-07-18T11:41:46, stalled for
792s at: Test2BDocs.test2BDocs
[junit4] 1> verifying...
[junit4] HEARTBEAT J0 PID(16332@localhost): 2019-07-18T11:42:46, stalled for
852s at: Test2BDocs.test2BDocs
[junit4] HEARTBEAT J0 PID(16332@localhost): 2019-07-18T11:43:46, stalled for
912s at: Test2BDocs.test2BDocs
[junit4] 1> Skip count=372418 seconds=123
[junit4] OK 926s | Test2BDocs.test2BDocs
[junit4] Completed [1/1] in 926.79s, 1 test
[junit4]
[junit4] JVM J0: 0.45 .. 927.62 = 927.18s
[junit4] Execution time total: 15 minutes 27 seconds
[junit4] Tests summary: 1 suite, 1 test
[...]
BUILD SUCCESSFUL
Total time: 15 minutes 42 seconds
{noformat}
I also tested at the same commit as the OP
({{081e2ef2c05e017e87a2aef2a4f55067fbba5cb4}}), with the same result: {{BUILD
SUCCESSFUL}}.
> Test2BDocs.test2BDocs error java.lang.ArrayIndexOutOfBoundsException
>
>
> Key: LUCENE-8926
> URL: https://issues.apache.org/jira/browse/LUCENE-8926
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.1.1
>Reporter: Daniel Black
>Priority: Major
>
> {noformat}
> HEARTBEAT J2 PID(27364@bobby): 2019-07-16T01:43:37, stalled for 190s at:
> Test2BPoints.test2D
> 1> indexed: 0
> 1> indexed: 1000
> 1> indexed: 2000
> ...
> 1> indexed: 213000
> 1> indexed: 214000
> 1> verifying...
> 2> NOTE: reproduce with: ant test -Dtestcase=Test2BDocs
> -Dtests.method=test2BDocs -Dtests.seed=ECB69064FDA0F2A6 -Dtests.slow=true
> -Dtests.badapples=true -Dtests.locale=zh -Dtests.timezone=Europe/Zurich
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> [00:41:02.330] ERROR 3755s J1 | Test2BDocs.test2BDocs <<<
> > Throwable #1: java.lang.ArrayIndexOutOfBoundsException
> > at __randomizedtesting.SeedInfo.seed([ECB69064FDA0F2A6:2B65388480C84F43]:0)
> > at
> > org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockImpactsEverythingEnum.advance(Lucene50PostingsReader.java:1605)
> > at
> > org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockImpactsEverythingEnum.nextDoc(Lucene50PostingsReader.java:1583)
> > at org.apache.lucene.index.CheckIndex.checkFields(CheckIndex.java:1526)
> > at org.apache.lucene.index.CheckIndex.testPostings(CheckIndex.java:1871)
> > at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:724)
> > at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:280)
> > at
> > org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:862)
> > at org.apache.lucene.index.Test2BDocs.test2BDocs(Test2BDocs.java:127)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
> > at java.lang.reflect.Method.invoke(Method.java:508)
> > at
> > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> > at
> > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
> > at
> > com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
> > at
> > com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
> > at
> > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> > at
> > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> > at
> > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> > at
> >
[jira] [Updated] (SOLR-13582) Ref guide build script: always check for and download public keys used to verify RVM distribution signature
[
https://issues.apache.org/jira/browse/SOLR-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe updated SOLR-13582:
--
Summary: Ref guide build script: always check for and download public keys
used to verify RVM distribution signature (was: Ref guide build script: always
check for and download public keys used to verify RVM distributiion signature)
> Ref guide build script: always check for and download public keys used to
> verify RVM distribution signature
> ---
>
> Key: SOLR-13582
> URL: https://issues.apache.org/jira/browse/SOLR-13582
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
>
> The 8.x ref guide builds are running on a Jenkins node they've never run on
> before: "websites2". As a result, the public keys used to verify the RVM
> distribution are not on the machine, and when the RVM bootstrap encounters
> this problem, it fails the build.
> In the past I've modified the Jenkins job config to download the public keys,
> triggered a run to download the keys, and then commented out the key download
> lines in the job config's build script.
> This issue is intended to permanently solve the problem by adding the
> following to our build script
> ({{dev-tools/scripts/jenkins.build.ref.guide.sh}}), to avoid manual
> intervention when new nodes are added in the future:
> * If {{gpg}} is installed:
> ** If the keys needed to verify RVM distribution signature are not available:
> *** Download the keys
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13582) Ref guide build script: always check for and download public keys used to verify RVM distributiion signature
Steve Rowe created SOLR-13582:
-
Summary: Ref guide build script: always check for and download
public keys used to verify RVM distributiion signature
Key: SOLR-13582
URL: https://issues.apache.org/jira/browse/SOLR-13582
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Steve Rowe
Assignee: Steve Rowe
The 8.x ref guide builds are running on a Jenkins node they've never run on
before: "websites2". As a result, the public keys used to verify the RVM
distribution are not on the machine, and when the RVM bootstrap encounters this
problem, it fails the build.
In the past I've modified the Jenkins job config to download the public keys,
triggered a run to download the keys, and then commented out the key download
lines in the job config's build script.
This issue is intended to permanently solve the problem by adding the following
to our build script ({{dev-tools/scripts/jenkins.build.ref.guide.sh}}), to
avoid manual intervention when new nodes are added in the future:
* If {{gpg}} is installed:
** If the keys needed to verify RVM distribution signature are not available:
*** Download the keys
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13549) Maven build fails to build Solr core tests due to missing dependency
[ https://issues.apache.org/jira/browse/SOLR-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16864290#comment-16864290 ] Steve Rowe commented on SOLR-13549: --- See also https://issues.apache.org/jira/browse/SOLR-13434?focusedCommentId=16861794=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16861794 > Maven build fails to build Solr core tests due to missing dependency > > > Key: SOLR-13549 > URL: https://issues.apache.org/jira/browse/SOLR-13549 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Affects Versions: master (9.0), 8.2 >Reporter: Daniel Collins >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > If I run > ant get-maven-poms > mvn -f maven-build/pom.xml install -DskipTests > On master or branch_8x, it fails to build the Solr tests due to a dependency > on opentracing-mock. Eclipse builds fine (because it uses a single classpath > with everything in it), and not entirely sure how ant builds... > But the solr/core/ivy.xml doesn't have a dependency on opentracing-mock > (though it has all the other opentracing modules) > [https://github.com/apache/lucene-solr/pull/720] for the fix > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13434) OpenTracing support for Solr
[ https://issues.apache.org/jira/browse/SOLR-13434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862272#comment-16862272 ] Steve Rowe commented on SOLR-13434: --- The Maven build, as Dat says, is quite a mess :) IIRC, the POM templates include some hard-code test dependency stuff, because the custom Ant plugin that generates POMs from the Ant build doesn't fully grok test dependencies. I'll take a look later today. > OpenTracing support for Solr > > > Key: SOLR-13434 > URL: https://issues.apache.org/jira/browse/SOLR-13434 > Project: Solr > Issue Type: New Feature >Reporter: Shalin Shekhar Mangar >Assignee: Cao Manh Dat >Priority: Major > Fix For: master (9.0), 8.2 > > Attachments: SOLR-13434.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > [OpenTracing|https://opentracing.io/] is a vendor neutral API and > infrastructure for distributed tracing. Many OSS tracers just as Jaeger, > OpenZipkin, Apache SkyWalking as well as commercial tools support OpenTracing > APIs. Ideally, we can implement it once and have integrations for popular > tracers like we have with metrics and prometheus. > I'm aware of SOLR-9641 but HTrace has since retired from incubator for lack > of activity so this is a fresh attempt at solving this problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13448) UAX29 URL Email Tokenizer: Ref guide description of hyphen handling is wrong
Steve Rowe created SOLR-13448: - Summary: UAX29 URL Email Tokenizer: Ref guide description of hyphen handling is wrong Key: SOLR-13448 URL: https://issues.apache.org/jira/browse/SOLR-13448 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: documentation Affects Versions: 7.2 Reporter: Steve Rowe Assignee: Steve Rowe As reported on the Solr user mailing list by Tom Van Cuyck: The UAX29 URL Email Tokenizer is not working as expected. According to the documentation ( https://lucene.apache.org/solr/guide/7_2/tokenizers.html): "Words are split at hyphens, unless there is a number in the word, in which case the token is not split and the numbers and hyphen(s) are preserved." So I expect "ABC-123" to remain "ABC-123" However the term is split in 2 separate tokens "ABC" and "123". Same for "AB12-CD34" --> "AB12" and "CD34" etc... Is this behavior to be expected? Or is there a way to get the behavior I expect? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13448) UAX29 URL Email Tokenizer: Ref guide description of hyphen handling is wrong
[ https://issues.apache.org/jira/browse/SOLR-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834423#comment-16834423 ] Steve Rowe commented on SOLR-13448: --- The documentation is wrong. The quoted sentence was inherited from Classic Tokenizer's description. UAX 29 URL Email Tokenizer is a specialization of Standard Tokenizer, the 7.2 documentation for which says the following: Note that words are split at hyphens. The ref guide should be updated to use the above sentence. > UAX29 URL Email Tokenizer: Ref guide description of hyphen handling is wrong > > > Key: SOLR-13448 > URL: https://issues.apache.org/jira/browse/SOLR-13448 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Affects Versions: 7.2 >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Minor > > As reported on the Solr user mailing list by Tom Van Cuyck: > The UAX29 URL Email Tokenizer is not working as expected. > According to the documentation ( > https://lucene.apache.org/solr/guide/7_2/tokenizers.html): "Words are split > at hyphens, unless there is a number in the word, in which case the token > is not split and the numbers and hyphen(s) are preserved." > So I expect "ABC-123" to remain "ABC-123" > However the term is split in 2 separate tokens "ABC" and "123". > Same for "AB12-CD34" --> "AB12" and "CD34" etc... > Is this behavior to be expected? Or is there a way to get the behavior I > expect? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816335#comment-16816335
]
Steve Rowe commented on LUCENE-2562:
{{ant nightly-smoke}} succeeded for me:
bq.[smoker] SUCCESS! [0:37:23.510122]
Uwe is right - {{verifyPOMperBinaryArtifact()}} just makes sure that each
binary artifact *in the {{maven/}} directory* has a POM, and since Luke doesn't
put anything in there, no problem is detected.
+1 to merge/cherry-pick.
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Tomoko Uchida
>Priority: Major
> Labels: gsoc2014
> Fix For: 8.1, master (9.0)
>
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png,
> Luke-ALE-3.png, Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png,
> luke-javafx2.png, luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg,
> lukeALE-documents.png, screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816304#comment-16816304
]
Steve Rowe commented on LUCENE-2562:
[~Tomoko Uchida], I'm re-running {{ant nightly-smoke}} from the branch after
your latest commit - you should be able to do the same.
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Tomoko Uchida
>Priority: Major
> Labels: gsoc2014
> Fix For: 8.1, master (9.0)
>
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png,
> Luke-ALE-3.png, Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png,
> luke-javafx2.png, luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg,
> lukeALE-documents.png, screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816269#comment-16816269
]
Steve Rowe edited comment on LUCENE-2562 at 4/12/19 1:36 PM:
-
The smoke tester failed for a non-Maven-related reason:
{noformat}
[smoker] Test Lucene...
[smoker] verify sha512 digest
[smoker] unpack lucene-9.0.0.tgz...
[smoker] Traceback (most recent call last):
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1518, in
[smoker] main()
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1448, in main
[smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir,
c.is_signed, c.local_keys, ' '.join(c.test_args))
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1499, in smokeTest
[smoker] unpackAndVerify(java, 'lucene', tmpDir, artifact, gitRevision,
version, testArgs, baseURL)
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line 604,
in unpackAndVerify
[smoker] verifyUnpacked(java, project, artifact, unpackPath,
gitRevision, version, testArgs, tmpDir, baseURL)
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line 679,
in verifyUnpacked
[smoker] raise RuntimeError('%s: unexpected files/dirs in artifact %s:
%s' % (project, artifact, l))
[smoker] RuntimeError: lucene: unexpected files/dirs in artifact
lucene-9.0.0.tgz: ['luke']
{noformat}
Sorry, I don't have time right now to work on it, but I will have time this
weekend if nobody beats me to it.
FYI, I expect there will be a Maven-related problem with the smoke tester -
here's what I think will be the problem (from {{smokeTestRelease.py}}) -
probably just need to introduce an exception list:
{noformat}
def verifyPOMperBinaryArtifact(artifacts, version):
print('verify that each binary artifact has a deployed POM...')
reBinaryJarWar = re.compile(r'%s\.[jw]ar$' % re.escape(version))
for project in ('lucene', 'solr'):
for artifact in [a for a in artifacts[project] if reBinaryJarWar.search(a)]:
POM = artifact[:-4] + '.pom'
if POM not in artifacts[project]:
raise RuntimeError('missing: POM for %s' % artifact)
{noformat}
was (Author: steve_rowe):
The smoke tester failed for a non-Maven-related reason:
{noformat}
[smoker] Test Lucene...
[smoker] verify sha512 digest
[smoker] unpack lucene-9.0.0.tgz...
[smoker] Traceback (most recent call last):
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1518, in
[smoker] main()
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1448, in main
[smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir,
c.is_signed, c.local_keys, ' '.join(c.test_args))
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1499, in smokeTest
[smoker] unpackAndVerify(java, 'lucene', tmpDir, artifact, gitRevision,
version, testArgs, baseURL)
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line 604,
in unpackAndVerify
[smoker] verifyUnpacked(java, project, artifact, unpackPath,
gitRevision, version, testArgs, tmpDir, baseURL)
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line 679,
in verifyUnpacked
[smoker] raise RuntimeError('%s: unexpected files/dirs in artifact %s:
%s' % (project, artifact, l))
[smoker] RuntimeError: lucene: unexpected files/dirs in artifact
lucene-9.0.0.tgz: ['luke']
{noformat}
Sorry, I don't have time right now to work on it, but I will have time this
weekend if nobody beats me to it.
FYI, I expect there will be a Maven-related problem with the smoke tester -
here's what I think will be the problem (from {{smokeTestRelease.py}} -
probably just need to introduce an exception list:
{noformat}
def verifyPOMperBinaryArtifact(artifacts, version):
print('verify that each binary artifact has a deployed POM...')
reBinaryJarWar = re.compile(r'%s\.[jw]ar$' % re.escape(version))
for project in ('lucene', 'solr'):
for artifact in [a for a in artifacts[project] if reBinaryJarWar.search(a)]:
POM = artifact[:-4] + '.pom'
if POM not in artifacts[project]:
raise RuntimeError('missing: POM for %s' % artifact)
{noformat}
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Tomoko Uchida
>Priority: Major
>
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816269#comment-16816269
]
Steve Rowe commented on LUCENE-2562:
The smoke tester failed for a non-Maven-related reason:
{noformat}
[smoker] Test Lucene...
[smoker] verify sha512 digest
[smoker] unpack lucene-9.0.0.tgz...
[smoker] Traceback (most recent call last):
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1518, in
[smoker] main()
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1448, in main
[smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir,
c.is_signed, c.local_keys, ' '.join(c.test_args))
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line
1499, in smokeTest
[smoker] unpackAndVerify(java, 'lucene', tmpDir, artifact, gitRevision,
version, testArgs, baseURL)
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line 604,
in unpackAndVerify
[smoker] verifyUnpacked(java, project, artifact, unpackPath,
gitRevision, version, testArgs, tmpDir, baseURL)
[smoker] File
"/home/sarowe/git/lucene-solr/dev-tools/scripts/smokeTestRelease.py", line 679,
in verifyUnpacked
[smoker] raise RuntimeError('%s: unexpected files/dirs in artifact %s:
%s' % (project, artifact, l))
[smoker] RuntimeError: lucene: unexpected files/dirs in artifact
lucene-9.0.0.tgz: ['luke']
{noformat}
Sorry, I don't have time right now to work on it, but I will have time this
weekend if nobody beats me to it.
FYI, I expect there will be a Maven-related problem with the smoke tester -
here's what I think will be the problem (from {{smokeTestRelease.py}} -
probably just need to introduce an exception list:
{noformat}
def verifyPOMperBinaryArtifact(artifacts, version):
print('verify that each binary artifact has a deployed POM...')
reBinaryJarWar = re.compile(r'%s\.[jw]ar$' % re.escape(version))
for project in ('lucene', 'solr'):
for artifact in [a for a in artifacts[project] if reBinaryJarWar.search(a)]:
POM = artifact[:-4] + '.pom'
if POM not in artifacts[project]:
raise RuntimeError('missing: POM for %s' % artifact)
{noformat}
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Tomoko Uchida
>Priority: Major
> Labels: gsoc2014
> Fix For: 8.1, master (9.0)
>
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png,
> Luke-ALE-3.png, Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png,
> luke-javafx2.png, luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg,
> lukeALE-documents.png, screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816237#comment-16816237
]
Steve Rowe edited comment on LUCENE-2562 at 4/12/19 1:06 PM:
-
[~thetaphi]: I didn't think of running the smoke tester; I'm not sure
whether/how Maven artifact validation checks artifact correspondence, I'll run
it locally on the branch now that Tomoko has updated it.
[~Tomoko Uchida]: I saw that same problem, as did Kevin Risden [over on
SOLR-9515|https://jira.apache.org/jira/browse/SOLR-9515?focusedCommentId=16762835=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16762835].
AFAICT this happens when you have a local maven repo that you haven't used the
Lucene/Solr Maven build with, but which is populated by the Lucene/Solr Ant
build via the Maven Ant Tasks plugin. Earlier in the log I see:
{noformat}
[artifact:dependencies] Downloading:
com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom from
repository taglets at http://maven.geotoolkit.org/
[artifact:dependencies] Transferring 0K from taglets
[artifact:dependencies] [WARNING] *** CHECKSUM FAILED - Checksum failed on
download: local = 'c53d5de1975ce58462f226d7ed126e02d8f1f58b'; remote = '
[artifact:dependencies] 301' - RETRYING
{noformat}
So (for me anyway) what happened was that a Maven repository we don't specify
(probably specified in the jackcess POM hierarchy) returned an HTML page and a
301 HTTP code (permanently relocated), which is improperly interpreted by Maven
Ant Tasks as the appropriate artifact and saved as if it were the jackcess
parent POM. I worked around this by deleting the local repo's bad file, then
installing the correct POM manually:
{noformat}
rm
~/.m2/repository/com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom
curl -O
https://repo1.maven.org/maven2/com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom
mvn install:install-file -Dfile=openhms-parent-1.1.8.pom
-DgroupId=com.healthmarketscience -DartifactId=openhms-parent -Dversion=1.1.8
-Dpackaging=pom
{noformat}
Maybe there's a better way? (E.g. running the Maven build in this context?)
The above fixed the problem for me. (Maven Ant Tasks have been EOL'd for a few
years, so there will be no fix for this problem in that project.)
was (Author: steve_rowe):
[~thetaphi]: I didn't think of running the smoke tester; I'm not sure
whether/how Maven artifact validation looks at , I'll run it locally on the
branch now that Tomoko has updated it.
[~Tomoko Uchida]: I saw that same problem, as did Kevin Risden [over on
SOLR-9515|https://jira.apache.org/jira/browse/SOLR-9515?focusedCommentId=16762835=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16762835].
AFAICT this happens when you have a local maven repo that you haven't used the
Lucene/Solr Maven build with, but which is populated by the Lucene/Solr Ant
build via the Maven Ant Tasks plugin. Earlier in the log I see:
{noformat}
[artifact:dependencies] Downloading:
com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom from
repository taglets at http://maven.geotoolkit.org/
[artifact:dependencies] Transferring 0K from taglets
[artifact:dependencies] [WARNING] *** CHECKSUM FAILED - Checksum failed on
download: local = 'c53d5de1975ce58462f226d7ed126e02d8f1f58b'; remote = '
[artifact:dependencies] 301' - RETRYING
{noformat}
So (for me anyway) what happened was that a Maven repository we don't specify
(probably specified in the jackcess POM hierarchy) returned an HTML page and a
301 HTTP code (permanently relocated), which is improperly interpreted by Maven
Ant Tasks as the appropriate artifact and saved as if it were the jackcess
parent POM. I worked around this by deleting the local repo's bad file, then
installing the correct POM manually:
{noformat}
rm
~/.m2/repository/com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom
curl -O
https://repo1.maven.org/maven2/com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom
mvn install:install-file -Dfile=openhms-parent-1.1.8.pom
-DgroupId=com.healthmarketscience -DartifactId=openhms-parent -Dversion=1.1.8
-Dpackaging=pom
{noformat}
Maybe there's a better way? (E.g. running the Maven build in this context?)
The above fixed the problem for me. (Maven Ant Tasks have been EOL'd for a few
years, so there will be no fix for this problem in that project.)
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Tomoko Uchida
>Priority: Major
> Labels: gsoc2014
> Fix For: 8.1, master (9.0)
>
>
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816237#comment-16816237
]
Steve Rowe commented on LUCENE-2562:
[~thetaphi]: I didn't think of running the smoke tester; I'm not sure
whether/how Maven artifact validation looks at , I'll run it locally on the
branch now that Tomoko has updated it.
[~Tomoko Uchida]: I saw that same problem, as did Kevin Risden [over on
SOLR-9515|https://jira.apache.org/jira/browse/SOLR-9515?focusedCommentId=16762835=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16762835].
AFAICT this happens when you have a local maven repo that you haven't used the
Lucene/Solr Maven build with, but which is populated by the Lucene/Solr Ant
build via the Maven Ant Tasks plugin. Earlier in the log I see:
{noformat}
[artifact:dependencies] Downloading:
com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom from
repository taglets at http://maven.geotoolkit.org/
[artifact:dependencies] Transferring 0K from taglets
[artifact:dependencies] [WARNING] *** CHECKSUM FAILED - Checksum failed on
download: local = 'c53d5de1975ce58462f226d7ed126e02d8f1f58b'; remote = '
[artifact:dependencies] 301' - RETRYING
{noformat}
So (for me anyway) what happened was that a Maven repository we don't specify
(probably specified in the jackcess POM hierarchy) returned an HTML page and a
301 HTTP code (permanently relocated), which is improperly interpreted by Maven
Ant Tasks as the appropriate artifact and saved as if it were the jackcess
parent POM. I worked around this by deleting the local repo's bad file, then
installing the correct POM manually:
{noformat}
rm
~/.m2/repository/com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom
curl -O
https://repo1.maven.org/maven2/com/healthmarketscience/openhms-parent/1.1.8/openhms-parent-1.1.8.pom
mvn install:install-file -Dfile=openhms-parent-1.1.8.pom
-DgroupId=com.healthmarketscience -DartifactId=openhms-parent -Dversion=1.1.8
-Dpackaging=pom
{noformat}
Maybe there's a better way? (E.g. running the Maven build in this context?)
The above fixed the problem for me. (Maven Ant Tasks have been EOL'd for a few
years, so there will be no fix for this problem in that project.)
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Tomoko Uchida
>Priority: Major
> Labels: gsoc2014
> Fix For: 8.1, master (9.0)
>
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png,
> Luke-ALE-3.png, Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png,
> luke-javafx2.png, luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg,
> lukeALE-documents.png, screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815824#comment-16815824
]
Steve Rowe commented on LUCENE-2562:
Hi [~Tomoko Uchida],
The Luke module has Maven-related build problems. E.g. from
https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1306/ :
{noformat}
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/build.xml:446:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build.xml:433:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build.xml:413:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/common-build.xml:2269:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/common-build.xml:1727:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/common-build.xml:657:
Unable to initialize POM pom.xml: Could not find the model file
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/poms/lucene/luke/pom.xml'.
for project unknown
{noformat}
This is caused by the Luke module not being set up to produce Maven artifacts
(e.g. there is no POM template under {{dev-tools/maven/}}). I don't think
publishing binary/source/javadoc jars on Maven central would be useful, but the
build should be fixed.
The following addition to {{lucene/luke/build.xml}} fixes the problematic
targets ({{validate-maven-dependencies}} and {{generate-maven-artifacts}}) by
locally re-defining Maven-related targets to do nothing:
{noformat}
{noformat}
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Tomoko Uchida
>Priority: Major
> Labels: gsoc2014
> Fix For: 8.1, master (9.0)
>
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png,
> Luke-ALE-3.png, Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png,
> luke-javafx2.png, luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg,
> lukeALE-documents.png, screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8616) Maven build: Simplify the way transitive dependency resolution is disabled, and increase the minimum supported Maven from 2.2.1 to 3.2.1
[
https://issues.apache.org/jira/browse/LUCENE-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796487#comment-16796487
]
Steve Rowe commented on LUCENE-8616:
[~krisden]: it's ready to go (might have grown stale since...), but as I wrote
in a post above:
{quote}I won't commit until I get other opinions about the wisdom of doing so,
given the Ant support issues mentioned in my previous comment on this
issue.{quote}
There has been no activity on IVY-1580 since then, and a
[query|http://mail-archives.apache.org/mod_mbox/ant-dev/201903.mbox/%3ccak-d_hvwodnhqgctwhupa55dq9ypaoees_mrswlamcl3uiy...@mail.gmail.com%3e]
about Ivy 2.5.0 release timing on the ant-dev list was [responded
to|http://mail-archives.apache.org/mod_mbox/ant-dev/201903.mbox/%3cbb6f61de-b0fa-c9f1-d30b-3dcae070a...@apache.org%3e]
two weeks ago:
{quote}
The only thing that's stopping me from
releasing 2.5.0 is IVY-1580 issue. I will send out a mail this week to
dev list to explain what that issue is and what are our options on
getting past it. Once we have a decision, I'll go ahead with a release
proposal.
{quote}
> Maven build: Simplify the way transitive dependency resolution is disabled,
> and increase the minimum supported Maven from 2.2.1 to 3.2.1
>
>
> Key: LUCENE-8616
> URL: https://issues.apache.org/jira/browse/LUCENE-8616
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Attachments: LUCENE-8616.patch
>
>
> Right now the Maven build disables transitive dependency resolution for all
> dependencies by listing each dependency's dependencies in a nested
> {{}} section in the grandparent POM's {{}}
> section.
> As of Maven 3.2.1, it's possible to use a wildcard syntax in nested
> {{}} sections to exclude all transitive deps: MNG-2315. This
> would make the grandparent POM much smaller and make the intent much clearer.
> We would have to increase our minimum supported Maven (currently 2.2.1), but
> that seems reasonable, given that Maven 2.x has been EOL'd:
> https://maven.apache.org/maven-2.x-eol.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-8865) real-time get does not retrieve values from docValues
[ https://issues.apache.org/jira/browse/SOLR-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-8865: - Comment: was deleted (was: [https://www.rinkalarijitsingh.com]) > real-time get does not retrieve values from docValues > - > > Key: SOLR-8865 > URL: https://issues.apache.org/jira/browse/SOLR-8865 > Project: Solr > Issue Type: Bug >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Blocker > Fix For: 5.5.1, 5.6, 6.0 > > Attachments: SOLR-8865.patch, SOLR-8865.patch, SOLR-8865.patch, > SOLR-8865.patch, SOLR-8865.patch, SOLR-8865.patch > > > Uncovered during ad-hoc testing... the _version_ field, which has > stored=false docValues=true is not retrieved with realtime-get -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-8865) real-time get does not retrieve values from docValues
[ https://issues.apache.org/jira/browse/SOLR-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-8865: - Comment: was deleted (was: happpy holi[https://www.rinkalarijitsingh.com/2019/03/happy-holi-joyful-and-colourful-festival.html]) > real-time get does not retrieve values from docValues > - > > Key: SOLR-8865 > URL: https://issues.apache.org/jira/browse/SOLR-8865 > Project: Solr > Issue Type: Bug >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Blocker > Fix For: 5.5.1, 5.6, 6.0 > > Attachments: SOLR-8865.patch, SOLR-8865.patch, SOLR-8865.patch, > SOLR-8865.patch, SOLR-8865.patch, SOLR-8865.patch > > > Uncovered during ad-hoc testing... the _version_ field, which has > stored=false docValues=true is not retrieved with realtime-get -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762779#comment-16762779
]
Steve Rowe edited comment on SOLR-9515 at 2/7/19 3:44 PM:
--
Maven build usage instructions are at {{dev-tools/maven/README.maven}}. FYI
currently the Maven Jenkins jobs don't run tests under Maven because nobody
maintains that capability - the Maven test runner (surefire) is different from
RandomizedRunner, the Ant one (see LUCENE-4045); the Jenkins jobs just build
the Maven artifacts and populate the Apache Maven snapshot repo.
was (Author: steve_rowe):
Maven build usage instructions are at {{dev-tools/maven/README. maven}}. FYI
currently the Maven Jenkins jobs don't run tests under Maven because nobody
maintains that capability - the Maven test runner (surefire) is different from
RandomizedRunner, the Ant one (see LUCENE-4045); the Jenkins jobs just build
the Maven artifacts and populate the Apache Maven snapshot repo.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, 8.x, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762779#comment-16762779
]
Steve Rowe commented on SOLR-9515:
--
Maven build usage instructions are at {{dev-tools/maven/README. maven}}. FYI
currently the Maven Jenkins jobs don't run tests under Maven because nobody
maintains that capability - the Maven test runner (surefire) is different from
RandomizedRunner, the Ant one (see LUCENE-4045); the Jenkins jobs just build
the Maven artifacts and populate the Apache Maven snapshot repo.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, 8.x, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762377#comment-16762377
]
Steve Rowe commented on SOLR-9515:
--
The Maven build has been failing since this was committed because the
kerby-kerb test dependency can't be found, e.g. from
[https://builds.apache.org/job/Lucene-Solr-Maven-master/2483]:
{noformat}
[mvn] [INFO] Apache Solr Core tests
FAILED [1.525s]
[...]
[mvn] [INFO]
[mvn] [INFO] Error for project: Apache Solr Core tests (during install)
[mvn] [INFO]
[mvn] [INFO] Failed to resolve artifact.
[mvn]
[mvn] Missing:
[mvn] --
[mvn] 1) org.apache.kerby:kerby-kerb:jar:1.0.1
[mvn]
[mvn] Try downloading the file manually from the project website.
[mvn]
[mvn] Then, install it using the command:
[mvn] mvn install:install-file -DgroupId=org.apache.kerby
-DartifactId=kerby-kerb -Dversion=1.0.1 -Dpackaging=jar -Dfile=/path/to/file
[mvn]
[mvn] Alternatively, if you host your own repository you can deploy the
file there:
[mvn] mvn deploy:deploy-file -DgroupId=org.apache.kerby
-DartifactId=kerby-kerb -Dversion=1.0.1 -Dpackaging=jar -Dfile=/path/to/file
-Durl=[url] -DrepositoryId=[id]
[mvn]
[mvn] Path to dependency:
[mvn] 1) org.apache.solr:solr-core-tests:jar:9.0.0-SNAPSHOT
[mvn] 2) org.apache.kerby:kerby-kerb:jar:1.0.1
{noformat}
The reason: {{kerby-kerb}} is an aggregation pom, not a jar. See
https://search.maven.org/artifact/org.apache.kerby/kerby-kerb/1.0.1/pom : note
{{pom}}.
It can be safely removed from {{solr/core/ivy.xml}} (and from
{{lucene/ivy-versions.properties}}, since it's only used in the one place).
When I remove it, {{ant test}} succeeds for me under {{solr/core/}}.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, 8.x, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8659) Upgrade OpenNLP to 1.9.1
[ https://issues.apache.org/jira/browse/LUCENE-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8659: --- Fix Version/s: (was: trunk) master (9.0) > Upgrade OpenNLP to 1.9.1 > > > Key: LUCENE-8659 > URL: https://issues.apache.org/jira/browse/LUCENE-8659 > Project: Lucene - Core > Issue Type: Task > Components: modules/analysis >Reporter: Tommaso Teofili >Assignee: Tommaso Teofili >Priority: Major > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > > Since Apache OpenNLP 1.9.1 has been released it would be nice to upgrade > Lucene/Solr to use that. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8659) Upgrade OpenNLP to 1.9.1
[
https://issues.apache.org/jira/browse/LUCENE-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16753158#comment-16753158
]
Steve Rowe commented on LUCENE-8659:
[~teofili], you didn't remove the 1.9.0 jar's checksum and add the one for
1.9.1 (via {{ant jar-checksums}}), so Jenkins is complaining (from
[https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1243/]):
{noformat}
[smoker] check-licenses:
[smoker] [echo] License check under:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/lucene-9.0.0
[smoker] [licenses] MISSING sha1 checksum file for:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/lucene-9.0.0/analysis/opennlp/lib/opennlp-tools-1.9.1.jar
[smoker] [licenses] EXPECTED sha1 checksum file :
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/lucene-9.0.0/licenses/opennlp-tools-1.9.1.jar.sha1
[smoker] [licenses] Scanned 36 JAR file(s) for licenses (in 0.40s.), 1
error(s).
{noformat}
Also:
* I changed the fixVersion from "trunk" (svn-based, no longer used) to "master
(9.0)".
* Shouldn't this be backported to branch_7x & branch_8x ?
> Upgrade OpenNLP to 1.9.1
>
>
> Key: LUCENE-8659
> URL: https://issues.apache.org/jira/browse/LUCENE-8659
> Project: Lucene - Core
> Issue Type: Task
> Components: modules/analysis
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Since Apache OpenNLP 1.9.1 has been released it would be nice to upgrade
> Lucene/Solr to use that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13053) NodeAddedTrigger and NodeLostTrigger do not reserve added/removed time populated by restoreState
[
https://issues.apache.org/jira/browse/SOLR-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737451#comment-16737451
]
Steve Rowe commented on SOLR-13053:
---
[~caomanhdat], looks like you neglected to backport your commit today on this
issue to branch_8x?
> NodeAddedTrigger and NodeLostTrigger do not reserve added/removed time
> populated by restoreState
>
>
> Key: SOLR-13053
> URL: https://issues.apache.org/jira/browse/SOLR-13053
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Minor
> Attachments: SOLR-13053.patch
>
>
> Currently, NodeAddedTrigger and NodeLostTrigger do not reserve added/removed
> time populated by restoreState.
> I believe that this is the root cause of failures in
> {{TestSimTriggerIntegration.testNodeLostTriggerRestoreState}} and
> {{TestSimTriggerIntegration.testNodeAddedTriggerRestoreState}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12983) JavabinLoader should avoid creating String Objects and create UTF8CharSequence fields from byte[]
[ https://issues.apache.org/jira/browse/SOLR-12983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737443#comment-16737443 ] Steve Rowe commented on SOLR-12983: --- [~noble.paul], looks like you neglected to backport commits on this issue to branch_8x ? > JavabinLoader should avoid creating String Objects and create > UTF8CharSequence fields from byte[] > -- > > Key: SOLR-12983 > URL: https://issues.apache.org/jira/browse/SOLR-12983 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Major > Attachments: SOLR-12983.patch > > > Javabin stings already contain Strings in UTF8 byte[] format. String fields > can be created directly from those -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6993) Update UAX29URLEmailTokenizer TLDs to latest list, and upgrade all JFlex-based tokenizers to support Unicode 8.0
[ https://issues.apache.org/jira/browse/LUCENE-6993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe resolved LUCENE-6993. Resolution: Duplicate Assignee: Steve Rowe (was: Robert Muir) Superseded by LUCENE-8527. > Update UAX29URLEmailTokenizer TLDs to latest list, and upgrade all > JFlex-based tokenizers to support Unicode 8.0 > > > Key: LUCENE-6993 > URL: https://issues.apache.org/jira/browse/LUCENE-6993 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/analysis >Reporter: Mike Drob >Assignee: Steve Rowe >Priority: Major > Attachments: LUCENE-6993.patch, LUCENE-6993.patch, LUCENE-6993.patch, > LUCENE-6993.patch, LUCENE-6993.patch, LUCENE-6993.patch, LUCENE-6993.patch, > LUCENE-6993.patch > > > We did this once before in LUCENE-5357, but it might be time to update the > list of TLDs again. Comparing our old list with a new list indicates 800+ new > domains, so it would be nice to include them. > Also the JFlex tokenizer grammars should be upgraded to support Unicode 8.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe resolved LUCENE-8527. Resolution: Fixed Fix Version/s: master (9.0) 7.7 8.0 > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build, modules/analysis >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Minor > Fix For: 8.0, 7.7, master (9.0) > > Attachments: LUCENE-8527.patch, LUCENE-8527.patch, LUCENE-8527.patch > > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-2073) Document issues involved in building your index with one jdk version and then searching/updating with another
[
https://issues.apache.org/jira/browse/LUCENE-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe resolved LUCENE-2073.
Resolution: Fixed
> Document issues involved in building your index with one jdk version and then
> searching/updating with another
> -
>
> Key: LUCENE-2073
> URL: https://issues.apache.org/jira/browse/LUCENE-2073
> Project: Lucene - Core
> Issue Type: Task
> Components: core/index
>Reporter: Mark Miller
>Assignee: Robert Muir
>Priority: Major
> Fix For: 6.0, 4.9
>
> Attachments: LUCENE-2073.patch, LUCENE-2073.patch, LUCENE-2073.patch,
> LUCENE-2073.patch, stdAnalyzerTest.java
>
>
> I think this needs to go in something of a permenant spot - this isn't a one
> time release type issues - its going to present over multiple release.
> {quote}
> If there is nothing we can do here, then we just have to do the best we can -
> such as a prominent notice alerting that if you transition JVM's between
> building and searching the index and you are using or doing X, things will
> break.
> We should put this in a spot that is always pretty visible - perhaps even a
> new readme file titlted something like IndexBackwardCompatibility or
> something, to which we can add other tips and gotchyas as they come up. Or
> MaintainingIndicesAcrossVersions, or
> FancyWhateverGetsYourAttentionAboutUpgradingStuff. Or a permanent
> entry/sticky entry at the top of Changes.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6102) regenerate bug
[
https://issues.apache.org/jira/browse/LUCENE-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe resolved LUCENE-6102.
Resolution: Not A Problem
> regenerate bug
> --
>
> Key: LUCENE-6102
> URL: https://issues.apache.org/jira/browse/LUCENE-6102
> Project: Lucene - Core
> Issue Type: Bug
> Components: general/build
>Affects Versions: 4.10.2
> Environment: PYTHON 2.6
> JDK 1.7.0.45
> ANT 1.9.2
>Reporter: Martin Gainty
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
>
>
>
>
>
> Running ${python.exe} on htmlentity.py
>
>
>
>
>file="src/java/org/apache/lucene/analysis/charfilter/HTMLCharacterEntities.jflex"
> encoding="UTF-8"/>
>
>
>
>
>classpath="%ANT_HOME%/lib"/>
> Run jflex-HTMLCharacterEntities
>
> value="%LUCENE_HOME%/analysis/common/src/java/org/apache/lucene/analysis/standard"
> />
>
> file="%LUCENE_HOME%/analysis/common/src/java/org/apache/lucene/charfilter/HTMLCharacterEntities.jflex"
> outdir="%LUCENE_HOME%/analysis/common/src/java/org/apache/lucene/charfilter"
> nobak="off" />
>
>
> when I regen'ed HTMLCharacterEntities.jflex I wanted to use jflex to generate
> the HtmlCharacterEntitiesImpl.java file the above target
> jflex-HTMLCharacterEntities will gen that file for you
> Nota Bene; take note of the bat/sh that properly sets PYTHONHOME and
> PYTHONPATH so Python can locate subordinate .pyc components
> I also had to tweak Gerwin Kleins JFlex library from 2003 to take inputFile
> and outputDir parameters
> Erik/Michael: does the original regenerate work for you?
> Martin Gainty 8 Dec 2014
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732330#comment-16732330 ] Steve Rowe commented on LUCENE-8527: bq. \[W\]ith the default skeleton, JFlex 1.7.0 generates scanners that misbehaves when given a spoon-feeding reader (i.e. a reader that returns at least one char but fewer than the requested number of chars) \[\] I'll make a JFlex issue for this bug. I created an issue: https://github.com/jflex-de/jflex/issues/538 bq. \[I\]nvocations of JFlex's Ant target inherit options used by previous invocations in the same Ant session. I'll make a JFlex issue for this bug. There has been an ongoing effort to fix this, see: https://github.com/jflex-de/jflex/pull/258 , likely will be included in next JFlex release. > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build, modules/analysis >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Minor > Attachments: LUCENE-8527.patch, LUCENE-8527.patch > > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8627) TestSearchAfter.testQueries() failure
Steve Rowe created LUCENE-8627:
--
Summary: TestSearchAfter.testQueries() failure
Key: LUCENE-8627
URL: https://issues.apache.org/jira/browse/LUCENE-8627
Project: Lucene - Core
Issue Type: Bug
Reporter: Steve Rowe
>From [https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1735],
>reproduces 10/10 iterations for me:
{noformat}
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSearchAfter
-Dtests.method=testQueries -Dtests.seed=C0154BFB98528228 -Dtests.multiplier=2
-Dtests.nightly=true -Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
-Dtests.locale=sr-ME -Dtests.timezone=Africa/Timbuktu -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.34s J2 | TestSearchAfter.testQueries <<<
[junit4]> Throwable #1: java.lang.AssertionError: expected:<1082> but
was:<1000>
[junit4]>at
__randomizedtesting.SeedInfo.seed([C0154BFB98528228:9C9B8720823B3786]:0)
[junit4]>at
org.apache.lucene.search.TestSearchAfter.assertPage(TestSearchAfter.java:268)
[junit4]>at
org.apache.lucene.search.TestSearchAfter.assertQuery(TestSearchAfter.java:260)
[junit4]>at
org.apache.lucene.search.TestSearchAfter.testQueries(TestSearchAfter.java:183)
[junit4]>at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=CheapBastard,
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@6f9a6df3),
locale=sr-ME, timezone=Africa/Timbuktu
[junit4] 2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation
1.8.0_191 (64-bit)/cpus=4,threads=1,free=174481080,total=519045120
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8616) Maven build: Simplify the way transitive dependency resolution is disabled, and increase the minimum supported Maven from 2.2.1 to 3.2.1
[
https://issues.apache.org/jira/browse/LUCENE-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726196#comment-16726196
]
Steve Rowe edited comment on LUCENE-8616 at 12/20/18 8:25 PM:
--
About the size of the grandparent POM generated with the patch: it's 75%
smaller: 101KB vs. 386KB; and 3K lines vs. 11K lines.
was (Author: steve_rowe):
About the size of the grandparent POM generated with the patched Maven build:
it's 75% smaller: 101KB vs. 386KB; and 3K lines vs. 11K lines.
> Maven build: Simplify the way transitive dependency resolution is disabled,
> and increase the minimum supported Maven from 2.2.1 to 3.2.1
>
>
> Key: LUCENE-8616
> URL: https://issues.apache.org/jira/browse/LUCENE-8616
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Attachments: LUCENE-8616.patch
>
>
> Right now the Maven build disables transitive dependency resolution for all
> dependencies by listing each dependency's dependencies in a nested
> {{}} section in the grandparent POM's {{}}
> section.
> As of Maven 3.2.1, it's possible to use a wildcard syntax in nested
> {{}} sections to exclude all transitive deps: MNG-2315. This
> would make the grandparent POM much smaller and make the intent much clearer.
> We would have to increase our minimum supported Maven (currently 2.2.1), but
> that seems reasonable, given that Maven 2.x has been EOL'd:
> https://maven.apache.org/maven-2.x-eol.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8616) Maven build: Simplify the way transitive dependency resolution is disabled, and increase the minimum supported Maven from 2.2.1 to 3.2.1
[
https://issues.apache.org/jira/browse/LUCENE-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726196#comment-16726196
]
Steve Rowe commented on LUCENE-8616:
About the size of the grandparent POM generated with the patched Maven build:
it's 75% smaller: 101KB vs. 386KB; and 3K lines vs. 11K lines.
> Maven build: Simplify the way transitive dependency resolution is disabled,
> and increase the minimum supported Maven from 2.2.1 to 3.2.1
>
>
> Key: LUCENE-8616
> URL: https://issues.apache.org/jira/browse/LUCENE-8616
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Attachments: LUCENE-8616.patch
>
>
> Right now the Maven build disables transitive dependency resolution for all
> dependencies by listing each dependency's dependencies in a nested
> {{}} section in the grandparent POM's {{}}
> section.
> As of Maven 3.2.1, it's possible to use a wildcard syntax in nested
> {{}} sections to exclude all transitive deps: MNG-2315. This
> would make the grandparent POM much smaller and make the intent much clearer.
> We would have to increase our minimum supported Maven (currently 2.2.1), but
> that seems reasonable, given that Maven 2.x has been EOL'd:
> https://maven.apache.org/maven-2.x-eol.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8616) Maven build: Simplify the way transitive dependency resolution is disabled, and increase the minimum supported Maven from 2.2.1 to 3.2.1
[
https://issues.apache.org/jira/browse/LUCENE-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726194#comment-16726194
]
Steve Rowe edited comment on LUCENE-8616 at 12/20/18 8:20 PM:
--
Attaching patch that increases minimum supported Maven from 2.2.1 to 3.2.1, and
that switches exclusions to wildcards in the grandparent POM's
{{}} section.
I tested using Maven 3.5.3 on macOS.
Maven compilation works, and all Lucene tests pass. None of the Solr modules
pass all their tests, but that's also true of the unpatched Maven build, for me
anyway.
I had to upgrade the {{maven-remote-resources-plugin}} version, from (default)
1.4 to 1.6.0, because 1.4 apparently ignores wildcard exclusions.
I looked at differences between the generated grandparent POMs, and as
expected, all dependencies' {{}} sections are converted to
wildcards. One other notable difference: in the unpatched Maven build's
grandparent POM, 82/259 dependencies have zero exclusions (and hence no
{{}} section), while with the patch every dependency, including
these 82, includes wildcard exclusions. I looked at a random handful of these,
and they legitimately have no compile-time non-optional dependencies.
Declaring them with an {{exclusions}} section (as in the patched Maven build)
looks better though, since its presence makes explicit the intent to disable
transitive dependency resolution.
I also looked at the difference in output between the patched and the unpatched
Maven build for {{mvn dependency:tree}}, and there was only one difference
(repeated for each module with this dependency) - the *unpatched* Maven build
shows {{jdk.tools:jdk.tools}} as a transitive dependency of
{{hadoop-annotations}}:
{noformat}
[INFO] +- org.apache.hadoop:hadoop-annotations:jar:2.7.4:compile
[INFO] | \- jdk.tools:jdk.tools:jar:1.8:system
{noformat}
^^ looks like a failure of the dependency discovery mechanism used by the
{{mvndeps}} task (via the Ivy API); the patched Maven build does not include
this transitive dependency, so that would be an improvement.
This patch also includes a fix for a WARNING in the unpatched build:
{noformat}
[WARNING] Some problems were encountered while building the effective model for
org.apache.solr:solr-test-framework:jar:8.0.0-SNAPSHOT
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
be unique: org.apache.lucene:lucene-test-framework:jar -> duplicate declaration
of version (?) @ org.apache.solr:solr-test-framework:[unknown-version],
/Users/sarowe/git/lucene-solr-3/maven-build/solr/test-framework/pom.xml, line
131, column 17
{noformat}
^^ This is caused by the {{lucene-test-framework}} dependency being listed
twice in the generated POM for {{solr-test-framework}}, once from the POM
template (to order it at the top of the dependency list, since it must precede
the {{lucene-core}} dependency) and another mention from the dependency
discovery mechanism. The included fix just adjusts the regex for the logic to
exclude some internal dependencies, so that it matches properly in this case
and causes {{lucene-test-framework}} not to be added by the dependency
discovery mechanism.
I won't commit until I get other opinions about the wisdom of doing so, given
the Ant support issues mentioned in my previous comment on this issue.
was (Author: steve_rowe):
Attaching patch that increases minimum supported Maven from 2.2.1 to 3.2.1, and
that switches exclusions to wildcards in the grandparent POM's
{{}} section.
I tested using Maven 3.5.3 on macOS.
Maven compilation works, and all Lucene tests pass. None of the Solr modules
pass all their tests, but that's also true of the unpatched Maven build, for me
anyway.
I had to upgrade the {{maven-remote-resources-plugin}} version, from (default)
1.4 to 1.6.0, because 1.4 apparently ignores wildcard exclusions.
I looked at differences between the generated grandparent POMs, and as
expected, all dependencies' {{}} sections are converted to
wildcards. One other notable difference: in the unpatched Maven build's
grandparent POM, 82/259 dependencies have zero exclusions (and hence no
{{}} section), while with the patch every dependency, including
these 82, includes wildcard exclusions. I looked at a random handful of these,
and they legitimately have no compile-time non-optional dependencies.
Declaring them with an {{exclusions}} section (as in the patched Maven build)
looks better though, since its presence makes explicit the intent to disable
transitive dependency resolution.
I also looked at the difference in output between the patched and the unpatched
Maven build for {{mvn dependency:tree}}, and there was only one difference
(repeated for each module with this dependency) - the *unpatched* Maven build
shows {{jdk.tools:jdk.tools}} as a transitive dependency of
{{hadoop-annotations}}:
{noformat}
[INFO] +-
[jira] [Commented] (LUCENE-8616) Maven build: Simplify the way transitive dependency resolution is disabled, and increase the minimum supported Maven from 2.2.1 to 3.2.1
[
https://issues.apache.org/jira/browse/LUCENE-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726140#comment-16726140
]
Steve Rowe commented on LUCENE-8616:
Related: Elasticsearch had wildcard exclusions for a while and then put back
their explicit exclusions, because 3rd-party tools like Ant didn't understand
wildcard exclusions, see https://github.com/elastic/elasticsearch/pull/24879
Ivy 2.5.0-rc1, released 8 months ago, includes wildcard exclusions support
(IVY-1531). There has been no discussion on the Ant dev mailing list about a
final 2.5.0 release, but I found an ivy-user@ post from an Ant dev (
https://lists.apache.org/thread.html/e911b9ea3b8c415de2927f3762ba0866465d95ee75aed6c6b336276a@%3Civy-user.ant.apache.org%3E
) that says:
{quote}
2.5.0-rc1 has a major issue that's being tracked as part of IVY-1580. I have
been working on it but haven't had enough time to get a right solution for it.
A fix for that would be necessary to release rc2 of Final of 2.5.0. I don't
have an exact time frame for the release, but I'll send a note on the dev list
once I have something.
{quote}
Unfortunately there has been no activity on IVY-1580 since then, and
Elasticsearch's latest POMs still have explicit exclusions.
Should we delay implementing wildcard exclusions until after Ivy 2.5.0 final
has been released?
> Maven build: Simplify the way transitive dependency resolution is disabled,
> and increase the minimum supported Maven from 2.2.1 to 3.2.1
>
>
> Key: LUCENE-8616
> URL: https://issues.apache.org/jira/browse/LUCENE-8616
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
>
> Right now the Maven build disables transitive dependency resolution for all
> dependencies by listing each dependency's dependencies in a nested
> {{}} section in the grandparent POM's {{}}
> section.
> As of Maven 3.2.1, it's possible to use a wildcard syntax in nested
> {{}} sections to exclude all transitive deps: MNG-2315. This
> would make the grandparent POM much smaller and make the intent much clearer.
> We would have to increase our minimum supported Maven (currently 2.2.1), but
> that seems reasonable, given that Maven 2.x has been EOL'd:
> https://maven.apache.org/maven-2.x-eol.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8611) Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency
[
https://issues.apache.org/jira/browse/LUCENE-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726087#comment-16726087
]
Steve Rowe commented on LUCENE-8611:
{quote}
Whilst we are in related territory, I did start SOLR-11579 about a year
ago,which was my attempt to bring all the plug-ins "up to date" for (at the
time!) Java 9.
I really need to update those patches for Java 11, but maybe that item should
merge into LUCENE-8616?
{quote}
As I commented over on LUCENE-8616, I think it makes sense to keep the maven
plugin upgrade effort separate, on SOLR-11579.
> Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency
> --
>
> Key: LUCENE-8611
> URL: https://issues.apache.org/jira/browse/LUCENE-8611
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
> Fix For: 7.7
>
> Attachments: LUCENE-8611-fix-maven.patch, LUCENE-8611.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8616) Maven build: Simplify the way transitive dependency resolution is disabled, and increase the minimum supported Maven from 2.2.1 to 3.2.1
[
https://issues.apache.org/jira/browse/LUCENE-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726080#comment-16726080
]
Steve Rowe commented on LUCENE-8616:
Thanks [~dancollins], I'll add SOLR-11579 as a linked issue here.
I had considered upgrading all maven plugins as part of this ticket, but since
you already have an open issue, let's do it over there. I do think it makes
sense to delay that work until this issue has been finished though, since IIRC
the Versions Maven Plugin will only suggest upgrades that are compatible with
the minimum Maven version, which will be raised by this issue.
> Maven build: Simplify the way transitive dependency resolution is disabled,
> and increase the minimum supported Maven from 2.2.1 to 3.2.1
>
>
> Key: LUCENE-8616
> URL: https://issues.apache.org/jira/browse/LUCENE-8616
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
>
> Right now the Maven build disables transitive dependency resolution for all
> dependencies by listing each dependency's dependencies in a nested
> {{}} section in the grandparent POM's {{}}
> section.
> As of Maven 3.2.1, it's possible to use a wildcard syntax in nested
> {{}} sections to exclude all transitive deps: MNG-2315. This
> would make the grandparent POM much smaller and make the intent much clearer.
> We would have to increase our minimum supported Maven (currently 2.2.1), but
> that seems reasonable, given that Maven 2.x has been EOL'd:
> https://maven.apache.org/maven-2.x-eol.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8615) Can LatLonShape's tessellator create more search-efficient triangles?
[ https://issues.apache.org/jira/browse/LUCENE-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8615: --- Attachment: image.png > Can LatLonShape's tessellator create more search-efficient triangles? > - > > Key: LUCENE-8615 > URL: https://issues.apache.org/jira/browse/LUCENE-8615 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Attachments: 2-tessellations.png, re-tessellate-triangle.png > > > The triangular mesh produced by LatLonShape's Tessellator creates reasonable > numbers of triangles, which is helpful for indexing speed. However I'm > wondering that there are conditions when it might be beneficial to run > tessellation slightly differently in order to create triangles that are more > search-friendly. Given that we only index the minimum bounding rectangle for > each triangle, we always check for intersection between the query and the > triangle if the query intersects with the MBR of the triangle. So the smaller > the area of the triangle compared to its MBR, the higher the likeliness to > have false positive when querying. > For instance see the following shape, there are two ways that it can be > tessellated into two triangles. LatLonShape's Tessellator is going to return > either of them depending on which point is listed first in the polygon. Yet > the first one is more efficient that the second one: with the second one, > both triangles have roughly the same MBR (which is also the MBR of the > polygon), so both triangles will need to be checked all the time whenever the > query intersects with this shared MBR. On the other hand, with the first way, > both MBRs are smaller and don't overlap, which makes it more likely that only > one triangle needs to be checked at query time. > !2-tessellations.png! > Another example is the following polygon. It can be tessellated into a single > triangle. Yet at times it might be a better idea create more triangles so > that the overall area of MBRs is smaller and queries are less likely to run > into false positives. > !re-tessellate-triangle.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8615) Can LatLonShape's tessellator create more search-efficient triangles?
[ https://issues.apache.org/jira/browse/LUCENE-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8615: --- Attachment: (was: image.png) > Can LatLonShape's tessellator create more search-efficient triangles? > - > > Key: LUCENE-8615 > URL: https://issues.apache.org/jira/browse/LUCENE-8615 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Attachments: 2-tessellations.png, re-tessellate-triangle.png > > > The triangular mesh produced by LatLonShape's Tessellator creates reasonable > numbers of triangles, which is helpful for indexing speed. However I'm > wondering that there are conditions when it might be beneficial to run > tessellation slightly differently in order to create triangles that are more > search-friendly. Given that we only index the minimum bounding rectangle for > each triangle, we always check for intersection between the query and the > triangle if the query intersects with the MBR of the triangle. So the smaller > the area of the triangle compared to its MBR, the higher the likeliness to > have false positive when querying. > For instance see the following shape, there are two ways that it can be > tessellated into two triangles. LatLonShape's Tessellator is going to return > either of them depending on which point is listed first in the polygon. Yet > the first one is more efficient that the second one: with the second one, > both triangles have roughly the same MBR (which is also the MBR of the > polygon), so both triangles will need to be checked all the time whenever the > query intersects with this shared MBR. On the other hand, with the first way, > both MBRs are smaller and don't overlap, which makes it more likely that only > one triangle needs to be checked at query time. > !2-tessellations.png! > Another example is the following polygon. It can be tessellated into a single > triangle. Yet at times it might be a better idea create more triangles so > that the overall area of MBRs is smaller and queries are less likely to run > into false positives. > !re-tessellate-triangle.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8611) Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency
[
https://issues.apache.org/jira/browse/LUCENE-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725911#comment-16725911
]
Steve Rowe commented on LUCENE-8611:
bq. I'm also thinking that it really makes sense to require a newer version of
Maven – plugins in those old versions don't support newer Java anyway and I
don't think there are many benefits of staying with such an old version if a
new one is available.
+1: LUCENE-8616
bq. I had an alternate fix which was to explicitly add hamcrest dependencies to
all the various ivy.xml files which now have a dependency on it
Thanks Daniel for noticing and working on a fix. I considered this ^^, but
then as other modules start to need hamcrest-core, there would be more build
failures like this, since the Ant build, used by the vast majority of devs,
works without the explicit dependency. I considered adding the dep to *all*
modules, but at that point, the intent was clear to me: all users of
{{lucene-test-framework}} should be able to depend on all its functionality
without having to declare extra stuff, which is exactly what transitive
dependency does.
bq. Does this patch apply to both branch_7x and master?
Yes, but I've just committed it both places.
> Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency
> --
>
> Key: LUCENE-8611
> URL: https://issues.apache.org/jira/browse/LUCENE-8611
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
> Fix For: 7.7
>
> Attachments: LUCENE-8611-fix-maven.patch, LUCENE-8611.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8616) Maven build: Simplify the way transitive dependency resolution is disabled, and increase the minimum supported Maven from 2.2.1 to 3.2.1
Steve Rowe created LUCENE-8616:
--
Summary: Maven build: Simplify the way transitive dependency
resolution is disabled, and increase the minimum supported Maven from 2.2.1 to
3.2.1
Key: LUCENE-8616
URL: https://issues.apache.org/jira/browse/LUCENE-8616
Project: Lucene - Core
Issue Type: Improvement
Components: general/build
Reporter: Steve Rowe
Assignee: Steve Rowe
Right now the Maven build disables transitive dependency resolution for all
dependencies by listing each dependency's dependencies in a nested
{{}} section in the grandparent POM's {{}}
section.
As of Maven 3.2.1, it's possible to use a wildcard syntax in nested
{{}} sections to exclude all transitive deps: MNG-2315. This would
make the grandparent POM much smaller and make the intent much clearer.
We would have to increase our minimum supported Maven (currently 2.2.1), but
that seems reasonable, given that Maven 2.x has been EOL'd:
https://maven.apache.org/maven-2.x-eol.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8611) Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency
[
https://issues.apache.org/jira/browse/LUCENE-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725446#comment-16725446
]
Steve Rowe commented on LUCENE-8611:
The addition of the {{hamcrest-core}} dependency to the
{{lucene-test-framework}} module under this issue has broken test compilation
under the Maven build, which -- unlike the Ant build -- enforces that all
modules depend intransitively on that module (along with all others).
I've attached a patch that fixes the problem. The grandparent pom, which hosts
the {{}} section, has each module's dependencies rendered
intransitive by listing each of its dependencies in a nested {{}}
section. The attached patch modifies the custom {{mvndeps}} Ant task's
{{GetMavenDependenciesTask}} to allow modules to have transitive dependencies,
by NOT writing out the aforementioned nested {{}} section in the
grandparent pom; only the {{lucene-test-framework}} module is granted this
ability in the patch.
The patch fixes Maven compilation. If there are no objections, I'll commit
tomorrow.
BTW as of Maven 3.2.1, it's possible to use a wildcard syntax to exclude all
transitive deps: MNG-2315. That would make the grandparent pom much smaller
and make the intent much clearer. We would have to increase our minimum
supported Maven (currently 2.2.1), but that seems reasonable, given that Maven
2.x has been EOL'd: https://maven.apache.org/maven-2.x-eol.html
> Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency
> --
>
> Key: LUCENE-8611
> URL: https://issues.apache.org/jira/browse/LUCENE-8611
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
> Fix For: 7.7
>
> Attachments: LUCENE-8611-fix-maven.patch, LUCENE-8611.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8611) Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency
[ https://issues.apache.org/jira/browse/LUCENE-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8611: --- Attachment: LUCENE-8611-fix-maven.patch > Update randomizedtesting to 2.7.2, JUnit to 4.12, add hamcrest-core dependency > -- > > Key: LUCENE-8611 > URL: https://issues.apache.org/jira/browse/LUCENE-8611 > Project: Lucene - Core > Issue Type: Task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: 7.7 > > Attachments: LUCENE-8611-fix-maven.patch, LUCENE-8611.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13060) Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job timeout, causing Jenkins to kill JVMs, causing dump files to be created that fill all disk
[
https://issues.apache.org/jira/browse/SOLR-13060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722231#comment-16722231
]
Steve Rowe commented on SOLR-13060:
---
{quote}
bq. Until this is solved, I don't think it makes sense to run hdfs tests at all
– they will hang and fill up disk space on jenkins.
+1, I'll apply the @AwaitsFix annotation.
{quote}
FYI I only applied the annotation to the three test suites mentioned in this
issue's description, since in all 9 or so instances I've seen, only those tests
have displayed the problem. (There are another 25 or so HDFS test suites that
I did not annotate.)
> Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job
> timeout, causing Jenkins to kill JVMs, causing dump files to be created that
> fill all disk space, causing failure of all following jobs on the same node
> -
>
> Key: SOLR-13060
> URL: https://issues.apache.org/jira/browse/SOLR-13060
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Steve Rowe
>Priority: Major
> Attachments:
> junit4-J0-20181210_065854_4175881849742830327151.spill.part1.gz
>
>
> The 3 tests that are affected:
> * HdfsAutoAddReplicasIntegrationTest
> * HdfsCollectionsAPIDistributedZkTest
> * MoveReplicaHDFSTest
> Instances from the dev list:
> 12/1:
> https://lists.apache.org/thread.html/e04ad0f9113e15f77393ccc26e3505e3090783b1d61bd1c7ff03d33e@%3Cdev.lucene.apache.org%3E
> 12/5:
> https://lists.apache.org/thread.html/d78c99255abfb5134803c2b77664c1a039d741f92d6e6fcbcc66cd14@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/92ad03795ae60b1e94859d49c07740ca303f997ae2532e6f079acfb4@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/26aace512bce0b51c4157e67ac3120f93a99905b40040bee26472097@%3Cdev.lucene.apache.org%3E
> 12/11:
> https://lists.apache.org/thread.html/33558a8dd292fd966a7f476bf345b66905d99f7eb9779a4d17b7ec97@%3Cdev.lucene.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13060) Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job timeout, causing Jenkins to kill JVMs, causing dump files to be created that fill all disk
[
https://issues.apache.org/jira/browse/SOLR-13060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722216#comment-16722216
]
Steve Rowe commented on SOLR-13060:
---
bq. Just FYI: the upgrade of randomizedtesting does fix the suite timeout
problem (I just tested it on by running SOLR-13074 with a suite timeout of 10
seconds...).
Awesome, thanks!
bq. I think one hour is very generous for the sysout loop in SOLR-13074, so
it'll be enough to fill the disk anyway.
All the examples I've seen have HEARTBEAT messages runing for 40k-50k seconds,
an order of magnitude higher, which is why I set it to an hour.
bq. I'll work on truncating sysouts up to at most 1 gig, test it on that
SOLR-13074, then maybe to fix the underlying cause of leaking threads.
Great!
bq. Until this is solved, I don't think it makes sense to run hdfs tests at all
– they will hang and fill up disk space on jenkins.
+1, I'll apply the {{@AwaitsFix}} annotation.
> Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job
> timeout, causing Jenkins to kill JVMs, causing dump files to be created that
> fill all disk space, causing failure of all following jobs on the same node
> -
>
> Key: SOLR-13060
> URL: https://issues.apache.org/jira/browse/SOLR-13060
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Steve Rowe
>Priority: Major
> Attachments:
> junit4-J0-20181210_065854_4175881849742830327151.spill.part1.gz
>
>
> The 3 tests that are affected:
> * HdfsAutoAddReplicasIntegrationTest
> * HdfsCollectionsAPIDistributedZkTest
> * MoveReplicaHDFSTest
> Instances from the dev list:
> 12/1:
> https://lists.apache.org/thread.html/e04ad0f9113e15f77393ccc26e3505e3090783b1d61bd1c7ff03d33e@%3Cdev.lucene.apache.org%3E
> 12/5:
> https://lists.apache.org/thread.html/d78c99255abfb5134803c2b77664c1a039d741f92d6e6fcbcc66cd14@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/92ad03795ae60b1e94859d49c07740ca303f997ae2532e6f079acfb4@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/26aace512bce0b51c4157e67ac3120f93a99905b40040bee26472097@%3Cdev.lucene.apache.org%3E
> 12/11:
> https://lists.apache.org/thread.html/33558a8dd292fd966a7f476bf345b66905d99f7eb9779a4d17b7ec97@%3Cdev.lucene.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13060) Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job timeout, causing Jenkins to kill JVMs, causing dump files to be created that fill all disk
[
https://issues.apache.org/jira/browse/SOLR-13060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720256#comment-16720256
]
Steve Rowe commented on SOLR-13060:
---
Same thing happened again, AFAICT *after* my suite timeout commit, on
https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1721/ - two test
suites continued to print HEARTBEAT messages long after the hour (3600s)
timeout I set on them. [~dweiss] do you understand what's happening?
>From .../1721/consoleText:
{noformat}
Checking out Revision ef2f0cd88c6eb4b662aea06eaeb3b933288b23eb (refs/remotes/
origin/master)
[...]
[junit4] HEARTBEAT J2 PID(17526@lucene1-us-west): 2018-12-13T05:07:17,
stalled for 48000s at: MoveReplicaHDFSTest (suite)
[junit4] HEARTBEAT J0 PID(17515@lucene1-us-west): 2018-12-13T05:07:17,
stalled for 49652s at: HdfsCollectionsAPIDistributedZkTest (suite)
{noformat}
>From {{git log}}:
{noformat}
commit ef2f0cd88c6eb4b662aea06eaeb3b933288b23eb
Author: Jan Høydahl
Date: Wed Dec 12 11:33:32 2018 +0100
[...]
commit ec1bd0da2f784a39fbe5dc21d78349c41bfdaec2
Author: Steve Rowe
Date: Tue Dec 11 18:49:06 2018 -0800
{noformat}
>From {{MoveReplicaHDFSTest}} and {{HdfsCollectionsAPIDistributedZkTest}} at
>commit {{ef2f0c}}:
{code:java}
@ThreadLeakFilters(defaultFilters = true, filters = {
BadHdfsThreadsFilter.class, // hdfs currently leaks thread(s)
MoveReplicaHDFSTest.ForkJoinThreadsFilter.class
})
@Nightly // test is too long for non nightly
@TimeoutSuite(millis = TimeUnits.HOUR)
public class MoveReplicaHDFSTest extends MoveReplicaTest {
{code}
{code:java}
@Slow
@Nightly
@ThreadLeakFilters(defaultFilters = true, filters = {
BadHdfsThreadsFilter.class // hdfs currently leaks thread(s)
})
@TimeoutSuite(millis = TimeUnits.HOUR)
//commented 23-AUG-2018
@LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028;)
// 12-Jun-2018
public class HdfsCollectionsAPIDistributedZkTest extends
CollectionsAPIDistributedZkTest {
{code}
> Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job
> timeout, causing Jenkins to kill JVMs, causing dump files to be created that
> fill all disk space, causing failure of all following jobs on the same node
> -
>
> Key: SOLR-13060
> URL: https://issues.apache.org/jira/browse/SOLR-13060
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Steve Rowe
>Priority: Major
> Attachments:
> junit4-J0-20181210_065854_4175881849742830327151.spill.part1.gz
>
>
> The 3 tests that are affected:
> * HdfsAutoAddReplicasIntegrationTest
> * HdfsCollectionsAPIDistributedZkTest
> * MoveReplicaHDFSTest
> Instances from the dev list:
> 12/1:
> https://lists.apache.org/thread.html/e04ad0f9113e15f77393ccc26e3505e3090783b1d61bd1c7ff03d33e@%3Cdev.lucene.apache.org%3E
> 12/5:
> https://lists.apache.org/thread.html/d78c99255abfb5134803c2b77664c1a039d741f92d6e6fcbcc66cd14@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/92ad03795ae60b1e94859d49c07740ca303f997ae2532e6f079acfb4@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/26aace512bce0b51c4157e67ac3120f93a99905b40040bee26472097@%3Cdev.lucene.apache.org%3E
> 12/11:
> https://lists.apache.org/thread.html/33558a8dd292fd966a7f476bf345b66905d99f7eb9779a4d17b7ec97@%3Cdev.lucene.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13060) Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job timeout, causing Jenkins to kill JVMs, causing dump files to be created that fill all disk
[
https://issues.apache.org/jira/browse/SOLR-13060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718414#comment-16718414
]
Steve Rowe commented on SOLR-13060:
---
As I mentioned I would on the dev list, I've committed suite timeouts (set to
one hour) on these three tests.
[~hossman] wrote on the dev list:
{quote}
There was a logging change that miller made on master ~Nov29 (and
backported to 7x a few days later) that i just dialed down from INFO to
DEBUG ... was causing an INFO log message once a second from every solr
core, which can add up in long running tests.
might explain the recent up tick...
de65b45fca00445d5dde8f25ee5bc67f791a84f7
a61c9f2c22362692b47e4d38dd32953f3a7156d1
{quote}
But the repeating log snippet in my previous comment is at WARN level, so I
don't think ^^ applies (at least in the case of the one test I took this from)?
[~dawid.weiss] wrote on the dev list:
{quote}
Unless the logging system is writing to raw io descriptors (or is
initialized earlier than test rules, which I don't think it should),
you could also use the:
@TestRuleLimitSysouts.Limit(bytes = )
to limit the amount of logs to a sensibly high limit.
{quote}
Good info, I'll wait to do this until the problem resurfaces.
> Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job
> timeout, causing Jenkins to kill JVMs, causing dump files to be created that
> fill all disk space, causing failure of all following jobs on the same node
> -
>
> Key: SOLR-13060
> URL: https://issues.apache.org/jira/browse/SOLR-13060
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Steve Rowe
>Priority: Major
> Attachments:
> junit4-J0-20181210_065854_4175881849742830327151.spill.part1.gz
>
>
> The 3 tests that are affected:
> * HdfsAutoAddReplicasIntegrationTest
> * HdfsCollectionsAPIDistributedZkTest
> * MoveReplicaHDFSTest
> Instances from the dev list:
> 12/1:
> https://lists.apache.org/thread.html/e04ad0f9113e15f77393ccc26e3505e3090783b1d61bd1c7ff03d33e@%3Cdev.lucene.apache.org%3E
> 12/5:
> https://lists.apache.org/thread.html/d78c99255abfb5134803c2b77664c1a039d741f92d6e6fcbcc66cd14@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/92ad03795ae60b1e94859d49c07740ca303f997ae2532e6f079acfb4@%3Cdev.lucene.apache.org%3E
> 12/8:
> https://lists.apache.org/thread.html/26aace512bce0b51c4157e67ac3120f93a99905b40040bee26472097@%3Cdev.lucene.apache.org%3E
> 12/11:
> https://lists.apache.org/thread.html/33558a8dd292fd966a7f476bf345b66905d99f7eb9779a4d17b7ec97@%3Cdev.lucene.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13060) Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job timeout, causing Jenkins to kill JVMs, causing dump files to be created that fill all disk sp
Steve Rowe created SOLR-13060: - Summary: Some Nightly HDFS tests never terminate on ASF Jenkins, triggering whole-job timeout, causing Jenkins to kill JVMs, causing dump files to be created that fill all disk space, causing failure of all following jobs on the same node Key: SOLR-13060 URL: https://issues.apache.org/jira/browse/SOLR-13060 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: Tests Reporter: Steve Rowe The 3 tests that are affected: * HdfsAutoAddReplicasIntegrationTest * HdfsCollectionsAPIDistributedZkTest * MoveReplicaHDFSTest Instances from the dev list: 12/1: https://lists.apache.org/thread.html/e04ad0f9113e15f77393ccc26e3505e3090783b1d61bd1c7ff03d33e@%3Cdev.lucene.apache.org%3E 12/5: https://lists.apache.org/thread.html/d78c99255abfb5134803c2b77664c1a039d741f92d6e6fcbcc66cd14@%3Cdev.lucene.apache.org%3E 12/8: https://lists.apache.org/thread.html/92ad03795ae60b1e94859d49c07740ca303f997ae2532e6f079acfb4@%3Cdev.lucene.apache.org%3E 12/8: https://lists.apache.org/thread.html/26aace512bce0b51c4157e67ac3120f93a99905b40040bee26472097@%3Cdev.lucene.apache.org%3E 12/11: https://lists.apache.org/thread.html/33558a8dd292fd966a7f476bf345b66905d99f7eb9779a4d17b7ec97@%3Cdev.lucene.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8527: --- Attachment: LUCENE-8527.patch > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build, modules/analysis >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Minor > Attachments: LUCENE-8527.patch > > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8527: --- Attachment: (was: LUCENE-8527.patch) > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build, modules/analysis >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Minor > Attachments: LUCENE-8527.patch > > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8527) Upgrade JFlex to 1.7.0
[
https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714302#comment-16714302
]
Steve Rowe edited comment on LUCENE-8527 at 12/10/18 5:20 AM:
--
Patch, passes most Lucene/Solr tests (see below), including the test built with
Unicode 9.0's word break test data: {{WordBreakTestUnicode_9_0_0}}.
{quote}So the stuff in the ICU tokenizer uses some properties to tag the "stuff
between breaks" as emoji token type versus something else. I looked at latest
jflex, it seems it would need those props?
{quote}
Yes, JFlex 1.7.0 doesn't have the Emoji props it needs to properly tokenize and
type as emoji, since these props' definitions are not included with
release-specific data. For Lucene's use it should be possible to script pulling
in Unicode data to augment the scanner specs, which would allow proper emoji
tokenization/typing to work. (I've made a note to add these properties to
future JFlex releases.)
Failing tests with the patch:
{{ant test -Dtestcase=TestStandardAnalyzer
-Dtests.method=testRandomHugeStringsGraphAfter -Dtests.seed=B33609C22A50A253
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-VE
-Dtests.timezone=Africa/Blantyre -Dtests.asserts=true
-Dtests.file.encoding=UTF-8}}
{{ant test -Dtestcase=TestStandardAnalyzer -Dtests.method=testRandomHugeStrings
-Dtests.seed=DA01A0705C379738 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=ru-RU -Dtests.timezone=Europe/Sarajevo -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1}}
In both ^^ of these cases,
{{BaseTokenStreamTestCase.checkAnalysisConsistency()}} fails with unexpected
tokenization after randomly choosing to use a spoon-feed reader wrapper:
{{MockReaderWrapper}}. If I disable the wrapping with those seeds, the tests
pass. I'll work on making a simplified test case demonstrating the problem; I'm
not sure what's going wrong.
was (Author: steve_rowe):
Patch, passes most Lucene/Solr tests (see below), including the test built with
Unicode 9.0's word break test data: {{WordBreakTestUnicode_9_0_0}}.
{quote}So the stuff in the ICU tokenizer uses some properties to tag the "stuff
between breaks" as emoji token type versus something else. I looked at latest
jflex, it seems it would need those props?
{quote}
Yes, JFlex 1.7.0 doesn't have the Emoji props it needs to properly tokenize and
type as emoji, since these props' definitions are not included with
release-specific data. For Lucene's use it should be possible to script pulling
in Unicode data to augment the scanner specs, which would allow proper emoji
tokenization/typing to work. (I've make a note to add these properties to
future JFlex releases.)
Failing tests with the patch:
{{ant test -Dtestcase=TestStandardAnalyzer
-Dtests.method=testRandomHugeStringsGraphAfter -Dtests.seed=B33609C22A50A253
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-VE
-Dtests.timezone=Africa/Blantyre -Dtests.asserts=true
-Dtests.file.encoding=UTF-8}}
{{ant test -Dtestcase=TestStandardAnalyzer -Dtests.method=testRandomHugeStrings
-Dtests.seed=DA01A0705C379738 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=ru-RU -Dtests.timezone=Europe/Sarajevo -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1}}
In both ^^ of these cases,
{{BaseTokenStreamTestCase.checkAnalysisConsistency()}} fails with unexpected
tokenization after randomly choosing to use a spoon-feed reader wrapper:
{{MockReaderWrapper}}. If I disable the wrapping with those seeds, the tests
pass. I'll work on making a simplified test case demonstrating the problem; I'm
not sure what's going wrong.
> Upgrade JFlex to 1.7.0
> --
>
> Key: LUCENE-8527
> URL: https://issues.apache.org/jira/browse/LUCENE-8527
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build, modules/analysis
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Attachments: LUCENE-8527.patch
>
>
> JFlex 1.7.0, supporting Unicode 9.0, was released recently:
> [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8527) Upgrade JFlex to 1.7.0
[
https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714304#comment-16714304
]
Steve Rowe commented on LUCENE-8527:
FYI the patch does not include generated files, since that would make it much
larger. Run {{ant jflex}} in {{lucene/core}} and {{lucene/analysis/common}} to
do the generation.
> Upgrade JFlex to 1.7.0
> --
>
> Key: LUCENE-8527
> URL: https://issues.apache.org/jira/browse/LUCENE-8527
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build, modules/analysis
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Attachments: LUCENE-8527.patch
>
>
> JFlex 1.7.0, supporting Unicode 9.0, was released recently:
> [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8527: --- Attachment: (was: LUCENE-8527.patch) > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build, modules/analysis >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Minor > Attachments: LUCENE-8527.patch > > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8527: --- Attachment: LUCENE-8527.patch > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build, modules/analysis >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Minor > Attachments: LUCENE-8527.patch > > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (LUCENE-8527) Upgrade JFlex to 1.7.0
[
https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe reassigned LUCENE-8527:
--
Assignee: Steve Rowe
Attachment: LUCENE-8527.patch
Patch, passes most Lucene/Solr tests (see below), including the test built with
Unicode 9.0's word break test data: {{WordBreakTestUnicode_9_0_0}}.
{quote}So the stuff in the ICU tokenizer uses some properties to tag the "stuff
between breaks" as emoji token type versus something else. I looked at latest
jflex, it seems it would need those props?
{quote}
Yes, JFlex 1.7.0 doesn't have the Emoji props it needs to properly tokenize and
type as emoji, since these props' definitions are not included with
release-specific data. For Lucene's use it should be possible to script pulling
in Unicode data to augment the scanner specs, which would allow proper emoji
tokenization/typing to work. (I've make a note to add these properties to
future JFlex releases.)
Failing tests with the patch:
{{ant test -Dtestcase=TestStandardAnalyzer
-Dtests.method=testRandomHugeStringsGraphAfter -Dtests.seed=B33609C22A50A253
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-VE
-Dtests.timezone=Africa/Blantyre -Dtests.asserts=true
-Dtests.file.encoding=UTF-8}}
{{ant test -Dtestcase=TestStandardAnalyzer -Dtests.method=testRandomHugeStrings
-Dtests.seed=DA01A0705C379738 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=ru-RU -Dtests.timezone=Europe/Sarajevo -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1}}
In both ^^ of these cases,
{{BaseTokenStreamTestCase.checkAnalysisConsistency()}} fails with unexpected
tokenization after randomly choosing to use a spoon-feed reader wrapper:
{{MockReaderWrapper}}. If I disable the wrapping with those seeds, the tests
pass. I'll work on making a simplified test case demonstrating the problem; I'm
not sure what's going wrong.
> Upgrade JFlex to 1.7.0
> --
>
> Key: LUCENE-8527
> URL: https://issues.apache.org/jira/browse/LUCENE-8527
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build, modules/analysis
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Attachments: LUCENE-8527.patch
>
>
> JFlex 1.7.0, supporting Unicode 9.0, was released recently:
> [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8527) Upgrade JFlex to 1.7.0
[
https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713437#comment-16713437
]
Steve Rowe commented on LUCENE-8527:
[~rcmuir ] mentioned on LUCENE-8125 that StandardTokenizer should give such
sequences the {{}} token type - see the logic in the {{icu}} module's
{{BreakIteratorWrapper}}.
JFlex 1.7.0 supports Unicode 9.0, which, if I'm interpreting the discussion at
http://www.unicode.org/L2/L2016/16315r-handling-seg-emoji.pdf properly, does
not (fully) include Emoji sequence support (though customized rules that would
do that properly in Unicode 9.0 are listed in that doc).
Should we include the (post-9.0) customized rules for Unicode 9.0?
> Upgrade JFlex to 1.7.0
> --
>
> Key: LUCENE-8527
> URL: https://issues.apache.org/jira/browse/LUCENE-8527
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build, modules/analysis
>Reporter: Steve Rowe
>Priority: Minor
>
> JFlex 1.7.0, supporting Unicode 9.0, was released recently:
> [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8527) Upgrade JFlex to 1.7.0
[
https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713437#comment-16713437
]
Steve Rowe edited comment on LUCENE-8527 at 12/8/18 12:22 AM:
--
[~rcmuir ] mentioned on LUCENE-8125 that StandardTokenizer should give Emoji
sequences the {{}} token type - see the logic in the {{icu}} module's
{{BreakIteratorWrapper}}.
JFlex 1.7.0 supports Unicode 9.0, which, if I'm interpreting the discussion at
http://www.unicode.org/L2/L2016/16315r-handling-seg-emoji.pdf properly, does
not (fully) include Emoji sequence support (though customized rules that would
do that properly in Unicode 9.0 are listed in that doc).
Should we include the (post-9.0) customized rules for Unicode 9.0?
was (Author: steve_rowe):
[~rcmuir ] mentioned on LUCENE-8125 that StandardTokenizer should give such
sequences the {{}} token type - see the logic in the {{icu}} module's
{{BreakIteratorWrapper}}.
JFlex 1.7.0 supports Unicode 9.0, which, if I'm interpreting the discussion at
http://www.unicode.org/L2/L2016/16315r-handling-seg-emoji.pdf properly, does
not (fully) include Emoji sequence support (though customized rules that would
do that properly in Unicode 9.0 are listed in that doc).
Should we include the (post-9.0) customized rules for Unicode 9.0?
> Upgrade JFlex to 1.7.0
> --
>
> Key: LUCENE-8527
> URL: https://issues.apache.org/jira/browse/LUCENE-8527
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build, modules/analysis
>Reporter: Steve Rowe
>Priority: Minor
>
> JFlex 1.7.0, supporting Unicode 9.0, was released recently:
> [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13027) Harden LeaderTragicEventTest.
[
https://issues.apache.org/jira/browse/SOLR-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713336#comment-16713336
]
Steve Rowe commented on SOLR-13027:
---
This seed reproduces for me 10/10 iterations on Java8, from
[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23304/]:
{noformat}
Checking out Revision aaa64d7015998f28aaffac031c4032abf73bebd6
(refs/remotes/origin/master)
[...]
[java-info] java version "10.0.1"
[java-info] OpenJDK Runtime Environment (10.0.1+10, Oracle Corporation)
[java-info] OpenJDK 64-Bit Server VM (10.0.1+10, Oracle Corporation)
[java-info] Test args: [-XX:+UseCompressedOops -XX:+UseSerialGC]
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=LeaderTragicEventTest -Dtests.method=test
-Dtests.seed=482611237BA22E39 -Dtests.multiplier=3 -Dtests.slow=true
-Dtests.locale=fo-FO -Dtests.timezone=Asia/Kolkata -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 47.8s J1 | LeaderTragicEventTest.test <<<
[junit4]> Throwable #1: java.lang.AssertionError: Timeout waiting for
new replica become leader
[junit4]> Timeout waiting to see state for collection=collection1
:DocCollection(collection1//collections/collection1/state.json/5)={
[junit4]> "pullReplicas":"0",
[junit4]> "replicationFactor":"2",
[junit4]> "shards":{"shard1":{
[junit4]> "range":"8000-7fff",
[junit4]> "state":"active",
[junit4]> "replicas":{
[junit4]> "core_node3":{
[junit4]> "core":"collection1_shard1_replica_n1",
[junit4]> "base_url":"http://127.0.0.1:39173/solr;,
[junit4]> "node_name":"127.0.0.1:39173_solr",
[junit4]> "state":"active",
[junit4]> "type":"NRT",
[junit4]> "force_set_state":"false",
[junit4]> "leader":"true"},
[junit4]> "core_node4":{
[junit4]> "core":"collection1_shard1_replica_n2",
[junit4]> "base_url":"http://127.0.0.1:40623/solr;,
[junit4]> "node_name":"127.0.0.1:40623_solr",
[junit4]> "state":"active",
[junit4]> "type":"NRT",
[junit4]> "force_set_state":"false",
[junit4]> "router":{"name":"compositeId"},
[junit4]> "maxShardsPerNode":"1",
[junit4]> "autoAddReplicas":"false",
[junit4]> "nrtReplicas":"2",
[junit4]> "tlogReplicas":"0"}
[junit4]> Live Nodes: [127.0.0.1:39173_solr, 127.0.0.1:40623_solr]
[junit4]> Last available state:
DocCollection(collection1//collections/collection1/state.json/5)={
[junit4]> "pullReplicas":"0",
[junit4]> "replicationFactor":"2",
[junit4]> "shards":{"shard1":{
[junit4]> "range":"8000-7fff",
[junit4]> "state":"active",
[junit4]> "replicas":{
[junit4]> "core_node3":{
[junit4]> "core":"collection1_shard1_replica_n1",
[junit4]> "base_url":"http://127.0.0.1:39173/solr;,
[junit4]> "node_name":"127.0.0.1:39173_solr",
[junit4]> "state":"active",
[junit4]> "type":"NRT",
[junit4]> "force_set_state":"false",
[junit4]> "leader":"true"},
[junit4]> "core_node4":{
[junit4]> "core":"collection1_shard1_replica_n2",
[junit4]> "base_url":"http://127.0.0.1:40623/solr;,
[junit4]> "node_name":"127.0.0.1:40623_solr",
[junit4]> "state":"active",
[junit4]> "type":"NRT",
[junit4]> "force_set_state":"false",
[junit4]> "router":{"name":"compositeId"},
[junit4]> "maxShardsPerNode":"1",
[junit4]> "autoAddReplicas":"false",
[junit4]> "nrtReplicas":"2",
[junit4]> "tlogReplicas":"0"}
[junit4]>at
__randomizedtesting.SeedInfo.seed([482611237BA22E39:C0722EF9D55E43C1]:0)
[junit4]>at
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:289)
[junit4]>at
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:267)
[junit4]>at
org.apache.solr.cloud.LeaderTragicEventTest.test(LeaderTragicEventTest.java:84)
[junit4]>at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4]>at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4]>at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4]>at
java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4]>at java.base/java.lang.Thread.run(Thread.java:844)
[jira] [Updated] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8527: --- Component/s: modules/analysis general/build > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build, modules/analysis >Reporter: Steve Rowe >Priority: Minor > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8527: --- Issue Type: Improvement (was: Bug) > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Steve Rowe >Priority: Major > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8527) Upgrade JFlex to 1.7.0
[ https://issues.apache.org/jira/browse/LUCENE-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated LUCENE-8527: --- Priority: Minor (was: Major) > Upgrade JFlex to 1.7.0 > -- > > Key: LUCENE-8527 > URL: https://issues.apache.org/jira/browse/LUCENE-8527 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Steve Rowe >Priority: Minor > > JFlex 1.7.0, supporting Unicode 9.0, was released recently: > [http://jflex.de/changelog.html#jflex-1.7.0]. We should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-13015) ChaosMonkeySafeLeaderWithPullReplicasTest failure: NumberFormatException: null
[
https://issues.apache.org/jira/browse/SOLR-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe resolved SOLR-13015.
---
Resolution: Duplicate
Duplicate of SOLR-11094, I'll post the failure over there.
> ChaosMonkeySafeLeaderWithPullReplicasTest failure: NumberFormatException: null
> --
>
> Key: SOLR-13015
> URL: https://issues.apache.org/jira/browse/SOLR-13015
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Steve Rowe
>Priority: Major
>
> From [https://builds.apache.org/job/Lucene-Solr-Tests-7.6/14/], reproduces
> for me on master:
> {noformat}
>[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=ChaosMonkeySafeLeaderWithPullReplicasTest -Dtests.method=test
> -Dtests.seed=48F9B2C979613211 -Dtests.multiplier=2 -Dtests.slow=true
> -Dtests.locale=en-IN -Dtests.timezone=Asia/Seoul -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1
>[junit4] ERROR 40.3s J0 | ChaosMonkeySafeLeaderWithPullReplicasTest.test
> <<<
>[junit4]> Throwable #1: java.lang.NumberFormatException: null
>[junit4]> at
> __randomizedtesting.SeedInfo.seed([48F9B2C979613211:C0AD8D13D79D5FE9]:0)
>[junit4]> at java.lang.Long.parseLong(Long.java:552)
>[junit4]> at java.lang.Long.parseLong(Long.java:631)
>[junit4]> at
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForReplicationFromReplicas(AbstractFullDistribZkTestBase.java:2133)
>[junit4]> at
> org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:211)
>[junit4]> at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
>[junit4]> at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
>[junit4]> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11094) NumberFormatException in ChaosMonkeySafeLeaderWithPullReplicasTest
[
https://issues.apache.org/jira/browse/SOLR-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698257#comment-16698257
]
Steve Rowe commented on SOLR-11094:
---
>From [https://builds.apache.org/job/Lucene-Solr-Tests-7.6/14/], reproduces for
>me on master:
{noformat}
Checking out Revision a31ac7f01121dc4d6fd02104f444a7224912aa89
(refs/remotes/origin/branch_7_6)
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ChaosMonkeySafeLeaderWithPullReplicasTest -Dtests.method=test
-Dtests.seed=48F9B2C979613211 -Dtests.multiplier=2 -Dtests.slow=true
-Dtests.locale=en-IN -Dtests.timezone=Asia/Seoul -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 40.3s J0 | ChaosMonkeySafeLeaderWithPullReplicasTest.test
<<<
[junit4]> Throwable #1: java.lang.NumberFormatException: null
[junit4]>at
__randomizedtesting.SeedInfo.seed([48F9B2C979613211:C0AD8D13D79D5FE9]:0)
[junit4]>at java.lang.Long.parseLong(Long.java:552)
[junit4]>at java.lang.Long.parseLong(Long.java:631)
[junit4]>at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForReplicationFromReplicas(AbstractFullDistribZkTestBase.java:2133)
[junit4]>at
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:211)
[junit4]>at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
[junit4]>at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
> NumberFormatException in ChaosMonkeySafeLeaderWithPullReplicasTest
> --
>
> Key: SOLR-11094
> URL: https://issues.apache.org/jira/browse/SOLR-11094
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Tomás Fernández Löbbe
>Priority: Major
> Attachments: failure.txt
>
>
> This happens in the {{waitForReplicationFromReplicas}} method, the leader
> responds with "null" when asked for the index commit time. Not sure if this
> is an expected response or if there is some bug with the replication handler
> that causes this.
> Stack trace:
> {noformat}
> Error Message:
> null
> Stack Trace:
> java.lang.NumberFormatException: null
> at
> __randomizedtesting.SeedInfo.seed([71A6789FE7B5226A:F9F2474549494F92]:0)
> at java.lang.Long.parseLong(Long.java:552)
> at java.lang.Long.parseLong(Long.java:631)
> at
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForReplicationFromReplicas(AbstractFullDistribZkTestBase.java:2121)
> at
> org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
> at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
> at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
>
[jira] [Created] (SOLR-13015) ChaosMonkeySafeLeaderWithPullReplicasTest failure: NumberFormatException: null
Steve Rowe created SOLR-13015:
-
Summary: ChaosMonkeySafeLeaderWithPullReplicasTest failure:
NumberFormatException: null
Key: SOLR-13015
URL: https://issues.apache.org/jira/browse/SOLR-13015
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Steve Rowe
>From [https://builds.apache.org/job/Lucene-Solr-Tests-7.6/14/], reproduces for
>me on master:
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ChaosMonkeySafeLeaderWithPullReplicasTest -Dtests.method=test
-Dtests.seed=48F9B2C979613211 -Dtests.multiplier=2 -Dtests.slow=true
-Dtests.locale=en-IN -Dtests.timezone=Asia/Seoul -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 40.3s J0 | ChaosMonkeySafeLeaderWithPullReplicasTest.test
<<<
[junit4]> Throwable #1: java.lang.NumberFormatException: null
[junit4]>at
__randomizedtesting.SeedInfo.seed([48F9B2C979613211:C0AD8D13D79D5FE9]:0)
[junit4]>at java.lang.Long.parseLong(Long.java:552)
[junit4]>at java.lang.Long.parseLong(Long.java:631)
[junit4]>at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForReplicationFromReplicas(AbstractFullDistribZkTestBase.java:2133)
[junit4]>at
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:211)
[junit4]>at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
[junit4]>at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8517) TestRandomChains.testRandomChainsWithLargeStrings failure
[
https://issues.apache.org/jira/browse/LUCENE-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689623#comment-16689623
]
Steve Rowe commented on LUCENE-8517:
Another reproducing seed, though it only fails for me if I run the whole suite,
i.e. remove {{-Dtests.method=testRandomChainsWithLargeStrings}} from the
cmdline - maybe this test method is affected by other methods somehow? From
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/377]:
{noformat}
[junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
[junit4] 2> Exception from random analyzer:
[junit4] 2> charfilters=
[junit4] 2> tokenizer=
[junit4] 2>
org.apache.lucene.analysis.MockTokenizer(org.apache.lucene.util.AttributeFactory$1@9c912349,
initial state: 0
[junit4] 2> state 0 [reject]:
[junit4] 2> a -> 1
[junit4] 2> b -> 2
[junit4] 2> f -> 3
[junit4] 2> i -> 4
[junit4] 2> n -> 5
[junit4] 2> o -> 6
[junit4] 2> s -> 7
[junit4] 2> t -> 8
[junit4] 2> w -> 9
[junit4] 2> state 1 [accept]:
[junit4] 2> n -> 10
[junit4] 2> r -> 11
[junit4] 2> s -> 12
[junit4] 2> t -> 13
[junit4] 2> state 2 [reject]:
[junit4] 2> e -> 14
[junit4] 2> u -> 15
[junit4] 2> y -> 16
[junit4] 2> state 3 [reject]:
[junit4] 2> o -> 17
[junit4] 2> state 4 [reject]:
[junit4] 2> f -> 18
[junit4] 2> n -> 19
[junit4] 2> s -> 20
[junit4] 2> t -> 21
[junit4] 2> state 5 [reject]:
[junit4] 2> o -> 22
[junit4] 2> state 6 [reject]:
[junit4] 2> f -> 23
[junit4] 2> n -> 24
[junit4] 2> r -> 25
[junit4] 2> state 7 [reject]:
[junit4] 2> u -> 26
[junit4] 2> state 8 [reject]:
[junit4] 2> h -> 27
[junit4] 2> o -> 28
[junit4] 2> state 9 [reject]:
[junit4] 2> a -> 29
[junit4] 2> i -> 30
[junit4] 2> state 10 [accept]:
[junit4] 2> d -> 31
[junit4] 2> state 11 [reject]:
[junit4] 2> e -> 32
[junit4] 2> state 12 [accept]:
[junit4] 2> state 13 [accept]:
[junit4] 2> state 14 [accept]:
[junit4] 2> state 15 [reject]:
[junit4] 2> t -> 33
[junit4] 2> state 16 [accept]:
[junit4] 2> state 17 [reject]:
[junit4] 2> r -> 34
[junit4] 2> state 18 [accept]:
[junit4] 2> state 19 [accept]:
[junit4] 2> t -> 35
[junit4] 2> state 20 [accept]:
[junit4] 2> state 21 [accept]:
[junit4] 2> state 22 [accept]:
[junit4] 2> t -> 36
[junit4] 2> state 23 [accept]:
[junit4] 2> state 24 [accept]:
[junit4] 2> state 25 [accept]:
[junit4] 2> state 26 [reject]:
[junit4] 2> c -> 37
[junit4] 2> state 27 [reject]:
[junit4] 2> a -> 38
[junit4] 2> e -> 39
[junit4] 2> i -> 40
[junit4] 2> state 28 [accept]:
[junit4] 2> state 29 [reject]:
[junit4] 2> s -> 41
[junit4] 2> state 30 [reject]:
[junit4] 2> l -> 42
[junit4] 2> t -> 43
[junit4] 2> state 31 [accept]:
[junit4] 2> state 32 [accept]:
[junit4] 2> state 33 [accept]:
[junit4] 2> state 34 [accept]:
[junit4] 2> state 35 [reject]:
[junit4] 2> o -> 44
[junit4] 2> state 36 [accept]:
[junit4] 2> state 37 [reject]:
[junit4] 2> h -> 45
[junit4] 2> state 38 [reject]:
[junit4] 2> t -> 46
[junit4] 2> state 39 [accept]:
[junit4] 2> i -> 47
[junit4] 2> n -> 48
[junit4] 2> r -> 49
[junit4] 2> s -> 50
[junit4] 2> y -> 51
[junit4] 2> state 40 [reject]:
[junit4] 2> s -> 52
[junit4] 2> state 41 [accept]:
[junit4] 2> state 42 [reject]:
[junit4] 2> l -> 53
[junit4] 2> state 43 [reject]:
[junit4] 2> h -> 54
[junit4] 2> state 44 [accept]:
[junit4] 2> state 45 [accept]:
[junit4] 2> state 46 [accept]:
[junit4] 2> state 47 [reject]:
[junit4] 2> r -> 55
[junit4] 2> state 48 [accept]:
[junit4] 2> state 49 [reject]:
[junit4] 2> e -> 56
[junit4] 2> state 50 [reject]:
[junit4] 2> e -> 57
[junit4] 2> state 51 [accept]:
[junit4] 2> state 52 [accept]:
[junit4] 2> state 53 [accept]:
[junit4] 2> state 54 [accept]:
[junit4] 2> state 55 [accept]:
[junit4] 2> state 56 [accept]:
[junit4] 2> state 57 [accept]:
[junit4] 2> , true)
[junit4] 2> filters=
[junit4] 2>
org.apache.lucene.analysis.shingle.ShingleFilter(ValidatingTokenFilter@13de14e
term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1)
[junit4] 2>
Conditional:org.apache.lucene.analysis.shingle.FixedShingleFilter(OneTimeWrapper@2c0047b9
term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,
2)
[junit4] 2>
org.apache.lucene.analysis.miscellaneous.DateRecognizerFilter(ValidatingTokenFilter@7846ee89
[jira] [Commented] (LUCENE-8517) TestRandomChains.testRandomChainsWithLargeStrings failure
[
https://issues.apache.org/jira/browse/LUCENE-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689602#comment-16689602
]
Steve Rowe commented on LUCENE-8517:
Thanks [~sokolov], have at it!
> TestRandomChains.testRandomChainsWithLargeStrings failure
> -
>
> Key: LUCENE-8517
> URL: https://issues.apache.org/jira/browse/LUCENE-8517
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Reporter: Steve Rowe
>Priority: Major
>
> From
> [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2828/consoleText],
> reproduces for me on Java8:
> {noformat}
> Checking out Revision 216f10026b86627750e133fe24ce6a750c470695
> (refs/remotes/origin/branch_7x)
> [...]
> [java-info] java version "10.0.1"
> [java-info] OpenJDK Runtime Environment (10.0.1+10, Oracle Corporation)
> [java-info] OpenJDK 64-Bit Server VM (10.0.1+10, Oracle Corporation)
> [java-info] Test args: [-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC]
> [...]
>[junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
>[junit4] 2> Exception from random analyzer:
>[junit4] 2> charfilters=
>[junit4] 2>
> org.apache.lucene.analysis.charfilter.MappingCharFilter(org.apache.lucene.analysis.charfilter.NormalizeCharMap@3ef95503,
> java.io.StringReader@70dde633)
>[junit4] 2>
> org.apache.lucene.analysis.fa.PersianCharFilter(org.apache.lucene.analysis.charfilter.MappingCharFilter@12423b20)
>[junit4] 2> tokenizer=
>[junit4] 2> org.apache.lucene.analysis.th.ThaiTokenizer()
>[junit4] 2> filters=
>[junit4] 2>
> org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter(ValidatingTokenFilter@7914bba7
>
> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,
> org.apache.lucene.analysis.compound.hyphenation.HyphenationTree@abd7bca)
>[junit4] 2>
> Conditional:org.apache.lucene.analysis.MockGraphTokenFilter(java.util.Random@56348091,
> OneTimeWrapper@aa1c073
> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1)
>[junit4] 2>
> Conditional:org.apache.lucene.analysis.shingle.FixedShingleFilter(OneTimeWrapper@4cf58fce
>
> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,
> 4, , )
>[junit4] 2>
> org.apache.lucene.analysis.pt.PortugueseLightStemFilter(ValidatingTokenFilter@3a915324
>
> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,keyword=false)
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRandomChains
> -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=92344C536D4E00F4
> -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-ZW
> -Dtests.timezone=Atlantic/Faroe -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>[junit4] ERROR 0.46s J2 |
> TestRandomChains.testRandomChainsWithLargeStrings <<<
>[junit4]> Throwable #1: java.lang.IllegalStateException: stage 3:
> inconsistent startOffset at pos=0: 0 vs 5; token=effort
>[junit4]> at
> __randomizedtesting.SeedInfo.seed([92344C536D4E00F4:F86FF34234002007]:0)
>[junit4]> at
> org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:109)
>[junit4]> at
> org.apache.lucene.analysis.pt.PortugueseLightStemFilter.incrementToken(PortugueseLightStemFilter.java:48)
>[junit4]> at
> org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:68)
>[junit4]> at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkResetException(BaseTokenStreamTestCase.java:441)
>[junit4]> at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
>[junit4]> at
> org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:897)
>[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:564)
>[junit4]> at java.base/java.lang.Thread.run(Thread.java:844)
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
> {dummy=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))},
> docValues:{}, maxPointsInLeafNode=214,
[jira] [Issue Comment Deleted] (SOLR-12861) Add Solr factory for new ByteBuffersDirectory
[ https://issues.apache.org/jira/browse/SOLR-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-12861: -- Comment: was deleted (was: +1 for your upgrade note additions for ByteBuffersDirectoryFactory, thanks [~ctargett]. The [configuration section you link to|https://builds.apache.org/view/L/view/Lucene/job/Solr-reference-guide-7.6/javadoc/datadir-and-directoryfactory-in-solrconfig.html] still mentions RAMDirectoryFactory and not ByteBuffersDirectoryFactory, but I think that's okay for now, given [~dweiss]'s intent to more broadly switch from RAMDirectory to ByteBuffersDirectory once branch_8x has been cut.) > Add Solr factory for new ByteBuffersDirectory > - > > Key: SOLR-12861 > URL: https://issues.apache.org/jira/browse/SOLR-12861 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Major > Fix For: 7.6, master (8.0) > > Attachments: SOLR-12861.patch > > > LUCENE-8438 and sub-tasks introduced ByteBuffersDirectory, a RAMDirectory > replacement. Solr needs a factory class in order to expose it to end-users. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12927) Ref Guide: Upgrade Notes for 7.6
[ https://issues.apache.org/jira/browse/SOLR-12927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685931#comment-16685931 ] Steve Rowe edited comment on SOLR-12927 at 11/14/18 12:20 AM: -- (Moving this comment from SOLR-12861 where I mistakenly first put it...): +1 for your upgrade note additions for ByteBuffersDirectoryFactory, thanks [~ctargett]. The [configuration section you link to|https://builds.apache.org/view/L/view/Lucene/job/Solr-reference-guide-7.6/javadoc/datadir-and-directoryfactory-in-solrconfig.html] still mentions RAMDirectoryFactory and not ByteBuffersDirectoryFactory, but I think that's okay for now, given [~dweiss]'s intent to more broadly switch from RAMDirectory to ByteBuffersDirectory once branch_8x has been cut. was (Author: steve_rowe): (Moving this comment from SOLR-12861 where I mistakenly first put it...): +1 for your upgrade note additions for ByteBuffersDirectoryFactory, thanks [~ctargett]. The configuration section you link to still mentions RAMDirectoryFactory and not ByteBuffersDirectoryFactory, but I think that's okay for now, given [~dweiss]'s intent to more broadly switch from RAMDirectory to ByteBuffersDirectory once branch_8x has been cut. > Ref Guide: Upgrade Notes for 7.6 > > > Key: SOLR-12927 > URL: https://issues.apache.org/jira/browse/SOLR-12927 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Blocker > Fix For: 7.6 > > Attachments: SOLR-12927.patch > > > Add Upgrade Notes from CHANGES and any other relevant changes worth > mentioning. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12927) Ref Guide: Upgrade Notes for 7.6
[ https://issues.apache.org/jira/browse/SOLR-12927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685931#comment-16685931 ] Steve Rowe commented on SOLR-12927: --- (Moving this comment from SOLR-12861 where I mistakenly first put it...): +1 for your upgrade note additions for ByteBuffersDirectoryFactory, thanks [~ctargett]. The configuration section you link to still mentions RAMDirectoryFactory and not ByteBuffersDirectoryFactory, but I think that's okay for now, given [~dweiss]'s intent to more broadly switch from RAMDirectory to ByteBuffersDirectory once branch_8x has been cut. > Ref Guide: Upgrade Notes for 7.6 > > > Key: SOLR-12927 > URL: https://issues.apache.org/jira/browse/SOLR-12927 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Blocker > Fix For: 7.6 > > Attachments: SOLR-12927.patch > > > Add Upgrade Notes from CHANGES and any other relevant changes worth > mentioning. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12861) Add Solr factory for new ByteBuffersDirectory
[ https://issues.apache.org/jira/browse/SOLR-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685622#comment-16685622 ] Steve Rowe commented on SOLR-12861: --- +1 for your upgrade note additions for ByteBuffersDirectoryFactory, thanks [~ctargett]. The [configuration section you link to|https://builds.apache.org/view/L/view/Lucene/job/Solr-reference-guide-7.6/javadoc/datadir-and-directoryfactory-in-solrconfig.html] still mentions RAMDirectoryFactory and not ByteBuffersDirectoryFactory, but I think that's okay for now, given [~dweiss]'s intent to more broadly switch from RAMDirectory to ByteBuffersDirectory once branch_8x has been cut. > Add Solr factory for new ByteBuffersDirectory > - > > Key: SOLR-12861 > URL: https://issues.apache.org/jira/browse/SOLR-12861 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Major > Fix For: 7.6, master (8.0) > > Attachments: SOLR-12861.patch > > > LUCENE-8438 and sub-tasks introduced ByteBuffersDirectory, a RAMDirectory > replacement. Solr needs a factory class in order to expose it to end-users. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12759) Disable ExtractingRequestHandlerTest on JDK 11
[ https://issues.apache.org/jira/browse/SOLR-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681980#comment-16681980 ] Steve Rowe commented on SOLR-12759: --- Oops, re-resolved before mentioning: I think this should be backported to the newly cut branch_7_6 (given the fixVersion of 7.6). > Disable ExtractingRequestHandlerTest on JDK 11 > -- > > Key: SOLR-12759 > URL: https://issues.apache.org/jira/browse/SOLR-12759 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - Solr Cell (Tika extraction) > Environment: JDK 11 and Tika 1.x >Reporter: David Smiley >Assignee: David Smiley >Priority: Minor > Fix For: 7.6 > > > ExtractingRequestHandlerTest has failed on a JDK 11 RC due to two conspiring > problems: (A) Tika 1.x sometimes calls Date.toString() when extracting > metadata (unreleased 2.x will fix this), (B) JDK 11 RC has a bug in some > locales like Arabic in which a Date.toString() will have a timezone offset > using its locale's characters for the digits instead of using EN_US. > I'll add an "assume" check so we don't see failures about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-12759) Disable ExtractingRequestHandlerTest on JDK 11
[ https://issues.apache.org/jira/browse/SOLR-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe resolved SOLR-12759. --- Resolution: Fixed > Disable ExtractingRequestHandlerTest on JDK 11 > -- > > Key: SOLR-12759 > URL: https://issues.apache.org/jira/browse/SOLR-12759 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - Solr Cell (Tika extraction) > Environment: JDK 11 and Tika 1.x >Reporter: David Smiley >Assignee: David Smiley >Priority: Minor > Fix For: 7.6 > > > ExtractingRequestHandlerTest has failed on a JDK 11 RC due to two conspiring > problems: (A) Tika 1.x sometimes calls Date.toString() when extracting > metadata (unreleased 2.x will fix this), (B) JDK 11 RC has a bug in some > locales like Arabic in which a Date.toString() will have a timezone offset > using its locale's characters for the digits instead of using EN_US. > I'll add an "assume" check so we don't see failures about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12759) Disable ExtractingRequestHandlerTest on JDK 11
[
https://issues.apache.org/jira/browse/SOLR-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681868#comment-16681868
]
Steve Rowe commented on SOLR-12759:
---
FYI for me on MacOS with Oracle 1.8.0_112 and AdoptOpenJDK 11+28, the below
program lists {{ChST}} as the only short display name containing a lowercase
letter:
{code:java}
import java.util.TimeZone;
import java.util.Locale;
public class ListAllTimeZoneDisplayNames {
public static void main(String[] args) {
for (String id : TimeZone.getAvailableIDs()) {
String displayName = TimeZone.getTimeZone(id).getDisplayName(false,
TimeZone.SHORT, Locale.US);
if (displayName.matches(".*[a-z].*")) {
System.out.println(id + ": " + displayName);
}
}
}
}
{code}
Produces:
{noformat}
Pacific/Guam: ChST
Pacific/Saipan: ChST
{noformat}
FWIW Oracle Java 7.0.71 also includes {{America/Metlakatla: MeST}})
> Disable ExtractingRequestHandlerTest on JDK 11
> --
>
> Key: SOLR-12759
> URL: https://issues.apache.org/jira/browse/SOLR-12759
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - Solr Cell (Tika extraction)
> Environment: JDK 11 and Tika 1.x
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Fix For: 7.6
>
>
> ExtractingRequestHandlerTest has failed on a JDK 11 RC due to two conspiring
> problems: (A) Tika 1.x sometimes calls Date.toString() when extracting
> metadata (unreleased 2.x will fix this), (B) JDK 11 RC has a bug in some
> locales like Arabic in which a Date.toString() will have a timezone offset
> using its locale's characters for the digits instead of using EN_US.
> I'll add an "assume" check so we don't see failures about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12759) Disable ExtractingRequestHandlerTest on JDK 11
[
https://issues.apache.org/jira/browse/SOLR-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681857#comment-16681857
]
Steve Rowe edited comment on SOLR-12759 at 11/9/18 7:20 PM:
Looks like the regex needs another adjustment, maybe relax it from
{{\[A-Z\]\{3,\}...}} to {{\[A-Za-z\]\{3,\}...}} to allow for case-insensitive
matching, which would match the currently problematic {{ChST}} locale below?
See e.g. [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/899]:
{noformat}
Checking out Revision d214f968d765e5c30c8782c5545c38d9aef487fe
(refs/remotes/origin/branch_7x)
[...]
[java-info] java version "1.8.0_191"
[java-info] Java(TM) SE Runtime Environment (1.8.0_191-b12, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (25.191-b12, Oracle Corporation)
[java-info] Test args: [-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC]
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ExtractingRequestHandlerTest -Dtests.seed=B4BB8D072ABBC41E
-Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=Pacific/Guam
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ExtractingRequestHandlerTest (suite) <<<
[junit4]> Throwable #1: java.lang.AssertionError: Is some other JVM
affected? Or bad regex? TzDisplayName: ChST
[junit4]>at
__randomizedtesting.SeedInfo.seed([B4BB8D072ABBC41E]:0)
[junit4]>at
org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:50)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
was (Author: steve_rowe):
Looks like the regex needs another adjustment, maybe relax it from
"[A-Z]{3,}([+-]\\d\\d(:\\d\\d)?)?}} to "[A-Za-z]{3,}([+-]\\d\\d(:\\d\\d)?)?" to
allow for case-insensitive matching, which would match the currently
problematic {{ChST}} locale below?
See e.g. [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/899]:
{noformat}
Checking out Revision d214f968d765e5c30c8782c5545c38d9aef487fe
(refs/remotes/origin/branch_7x)
[...]
[java-info] java version "1.8.0_191"
[java-info] Java(TM) SE Runtime Environment (1.8.0_191-b12, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (25.191-b12, Oracle Corporation)
[java-info] Test args: [-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC]
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ExtractingRequestHandlerTest -Dtests.seed=B4BB8D072ABBC41E
-Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=Pacific/Guam
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ExtractingRequestHandlerTest (suite) <<<
[junit4]> Throwable #1: java.lang.AssertionError: Is some other JVM
affected? Or bad regex? TzDisplayName: ChST
[junit4]>at
__randomizedtesting.SeedInfo.seed([B4BB8D072ABBC41E]:0)
[junit4]>at
org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:50)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
> Disable ExtractingRequestHandlerTest on JDK 11
> --
>
> Key: SOLR-12759
> URL: https://issues.apache.org/jira/browse/SOLR-12759
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - Solr Cell (Tika extraction)
> Environment: JDK 11 and Tika 1.x
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Fix For: 7.6
>
>
> ExtractingRequestHandlerTest has failed on a JDK 11 RC due to two conspiring
> problems: (A) Tika 1.x sometimes calls Date.toString() when extracting
> metadata (unreleased 2.x will fix this), (B) JDK 11 RC has a bug in some
> locales like Arabic in which a Date.toString() will have a timezone offset
> using its locale's characters for the digits instead of using EN_US.
> I'll add an "assume" check so we don't see failures about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12759) Disable ExtractingRequestHandlerTest on JDK 11
[
https://issues.apache.org/jira/browse/SOLR-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681857#comment-16681857
]
Steve Rowe edited comment on SOLR-12759 at 11/9/18 7:18 PM:
Looks like the regex needs another adjustment, maybe relax it from
"[A-Z]{3,}([+-]\\d\\d(:\\d\\d)?)?}} to "[A-Za-z]{3,}([+-]\\d\\d(:\\d\\d)?)?" to
allow for case-insensitive matching, which would match the currently
problematic {{ChST}} locale below?
See e.g. [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/899]:
{noformat}
Checking out Revision d214f968d765e5c30c8782c5545c38d9aef487fe
(refs/remotes/origin/branch_7x)
[...]
[java-info] java version "1.8.0_191"
[java-info] Java(TM) SE Runtime Environment (1.8.0_191-b12, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (25.191-b12, Oracle Corporation)
[java-info] Test args: [-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC]
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ExtractingRequestHandlerTest -Dtests.seed=B4BB8D072ABBC41E
-Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=Pacific/Guam
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ExtractingRequestHandlerTest (suite) <<<
[junit4]> Throwable #1: java.lang.AssertionError: Is some other JVM
affected? Or bad regex? TzDisplayName: ChST
[junit4]>at
__randomizedtesting.SeedInfo.seed([B4BB8D072ABBC41E]:0)
[junit4]>at
org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:50)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
was (Author: steve_rowe):
Looks like the regex needs another adjustment, maybe relax it from
{{\[A-Z\]\{3,\}(\[+-\]\\d\\d(:\\d\\d)?)?}} to
{{[A-Za-z]{3,}([+-]\\d\\d(:\\d\\d)?)?}} to allow for case-insensitive matching,
which would match the currently problematic {{ChST}} locale below?
See e.g. [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/899]:
{noformat}
Checking out Revision d214f968d765e5c30c8782c5545c38d9aef487fe
(refs/remotes/origin/branch_7x)
[...]
[java-info] java version "1.8.0_191"
[java-info] Java(TM) SE Runtime Environment (1.8.0_191-b12, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (25.191-b12, Oracle Corporation)
[java-info] Test args: [-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC]
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ExtractingRequestHandlerTest -Dtests.seed=B4BB8D072ABBC41E
-Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=Pacific/Guam
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ExtractingRequestHandlerTest (suite) <<<
[junit4]> Throwable #1: java.lang.AssertionError: Is some other JVM
affected? Or bad regex? TzDisplayName: ChST
[junit4]>at
__randomizedtesting.SeedInfo.seed([B4BB8D072ABBC41E]:0)
[junit4]>at
org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:50)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
> Disable ExtractingRequestHandlerTest on JDK 11
> --
>
> Key: SOLR-12759
> URL: https://issues.apache.org/jira/browse/SOLR-12759
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - Solr Cell (Tika extraction)
> Environment: JDK 11 and Tika 1.x
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Fix For: 7.6
>
>
> ExtractingRequestHandlerTest has failed on a JDK 11 RC due to two conspiring
> problems: (A) Tika 1.x sometimes calls Date.toString() when extracting
> metadata (unreleased 2.x will fix this), (B) JDK 11 RC has a bug in some
> locales like Arabic in which a Date.toString() will have a timezone offset
> using its locale's characters for the digits instead of using EN_US.
> I'll add an "assume" check so we don't see failures about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-12759) Disable ExtractingRequestHandlerTest on JDK 11
[
https://issues.apache.org/jira/browse/SOLR-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe reopened SOLR-12759:
---
Looks like the regex needs another adjustment, maybe relax it from
{{\[A-Z\]\{3,\}(\[+-\]\\d\\d(:\\d\\d)?)?}} to
{{[A-Za-z]{3,}([+-]\\d\\d(:\\d\\d)?)?}} to allow for case-insensitive matching,
which would match the currently problematic {{ChST}} locale below?
See e.g. [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/899]:
{noformat}
Checking out Revision d214f968d765e5c30c8782c5545c38d9aef487fe
(refs/remotes/origin/branch_7x)
[...]
[java-info] java version "1.8.0_191"
[java-info] Java(TM) SE Runtime Environment (1.8.0_191-b12, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (25.191-b12, Oracle Corporation)
[java-info] Test args: [-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC]
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ExtractingRequestHandlerTest -Dtests.seed=B4BB8D072ABBC41E
-Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=Pacific/Guam
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ExtractingRequestHandlerTest (suite) <<<
[junit4]> Throwable #1: java.lang.AssertionError: Is some other JVM
affected? Or bad regex? TzDisplayName: ChST
[junit4]>at
__randomizedtesting.SeedInfo.seed([B4BB8D072ABBC41E]:0)
[junit4]>at
org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:50)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
> Disable ExtractingRequestHandlerTest on JDK 11
> --
>
> Key: SOLR-12759
> URL: https://issues.apache.org/jira/browse/SOLR-12759
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - Solr Cell (Tika extraction)
> Environment: JDK 11 and Tika 1.x
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Fix For: 7.6
>
>
> ExtractingRequestHandlerTest has failed on a JDK 11 RC due to two conspiring
> problems: (A) Tika 1.x sometimes calls Date.toString() when extracting
> metadata (unreleased 2.x will fix this), (B) JDK 11 RC has a bug in some
> locales like Arabic in which a Date.toString() will have a timezone offset
> using its locale's characters for the digits instead of using EN_US.
> I'll add an "assume" check so we don't see failures about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8560) TestByteBuffersDirectory.testSeekPastEOF() failures with ByteArrayIndexInput
[
https://issues.apache.org/jira/browse/LUCENE-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681789#comment-16681789
]
Steve Rowe commented on LUCENE-8560:
+1 to nuke ByteArrayIndexInput [~dweiss], in both cases users are not involved
in choice of exact implementation.
Probably the {{ByteBuffersDirectory.OUTPUT_AS_BYTE_ARRAY}} option could be
deprecated, and in the meantime its impl could be forwarded to the
{{OUTPUT_AS_ONE_BUFFER}} impl?
> TestByteBuffersDirectory.testSeekPastEOF() failures with ByteArrayIndexInput
>
>
> Key: LUCENE-8560
> URL: https://issues.apache.org/jira/browse/LUCENE-8560
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Reporter: Steve Rowe
>Assignee: Dawid Weiss
>Priority: Minor
> Attachments: LUCENE-8560.patch
>
>
> Two reproducing seeds below. In both cases:
> * the {{IndexInput}} implementation is {{ByteArrayIndexInput}}
> * seeking to exactly EOF does not throw an exception
> * {{ByteArrayIndexInput.readByte()}} throws AIOOBE instead of the expected
> EOFException
> From [https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4903]:
> {noformat}
> Checking out Revision 856e28d8cf07cc34bc1361784bf00e7aceb3af97
> (refs/remotes/origin/master)
> [...]
>[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestByteBuffersDirectory -Dtests.method=testSeekPastEOF
> -Dtests.seed=BDFA8CEDB7C93AC1 -Dtests.slow=true -Dtests.locale=sr-RS
> -Dtests.timezone=Europe/Astrakhan -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 0.00s J0 | TestByteBuffersDirectory.testSeekPastEOF
> {impl=byte array (heap)} <<<
>[junit4]> Throwable #1: junit.framework.AssertionFailedError:
> Unexpected exception type, expected EOFException but got
> java.lang.ArrayIndexOutOfBoundsException: 1770
>[junit4]> at
> __randomizedtesting.SeedInfo.seed([BDFA8CEDB7C93AC1:5DBC4714B74C4450]:0)
>[junit4]> at
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2680)
>[junit4]> at
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2669)
>[junit4]> at
> org.apache.lucene.store.BaseDirectoryTestCase.testSeekPastEOF(BaseDirectoryTestCase.java:516)
>[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:564)
>[junit4]> at java.base/java.lang.Thread.run(Thread.java:844)
>[junit4]> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1770
>[junit4]> at
> org.apache.lucene.store.ByteArrayIndexInput.readByte(ByteArrayIndexInput.java:145)
>[junit4]> at
> org.apache.lucene.store.BaseDirectoryTestCase.lambda$testSeekPastEOF$12(BaseDirectoryTestCase.java:518)
>[junit4]> at
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2675)
>[junit4]> ... 37 more
> [...]
>[junit4] 2> NOTE: test params are: codec=Lucene80,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2c972cf9),
> locale=sr-RS, timezone=Europe/Astrakhan
>[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 9
> (64-bit)/cpus=3,threads=1,free=157933784,total=235929600
> {noformat}
> Also (older) from
> [https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1645]:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestByteBuffersDirectory -Dtests.method=testSeekPastEOF
> -Dtests.seed=90B07B6267E63464 -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
> -Dtests.locale=es-PR -Dtests.timezone=Australia/Currie -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> [junit4] FAILURE 0.01s J1 | TestByteBuffersDirectory.testSeekPastEOF
> {impl=byte array (heap)} <<<
> [junit4]> Throwable #1: junit.framework.AssertionFailedError:
> Unexpected exception type, expected EOFException but got
> java.lang.ArrayIndexOutOfBoundsException: 1881
> [junit4]> at
> __randomizedtesting.SeedInfo.seed([90B07B6267E63464:70F6B09B67634AF5]:0)
> [junit4]> at
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2683)
> [junit4]> at
>
[jira] [Created] (LUCENE-8560) TestByteBuffersDirectory.testSeekPastEOF() failures with ByteArrayIndexInput
Steve Rowe created LUCENE-8560:
--
Summary: TestByteBuffersDirectory.testSeekPastEOF() failures with
ByteArrayIndexInput
Key: LUCENE-8560
URL: https://issues.apache.org/jira/browse/LUCENE-8560
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Reporter: Steve Rowe
Assignee: Steve Rowe
Two reproducing seeds below. In both cases:
* the {{IndexInput}} implementation is {{ByteArrayIndexInput}}
* seeking to exactly EOF does not throw an exception
* {{ByteArrayIndexInput.readByte()}} throws AIOOBE instead of the expected
EOFException
>From [https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4903]:
{noformat}
Checking out Revision 856e28d8cf07cc34bc1361784bf00e7aceb3af97
(refs/remotes/origin/master)
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestByteBuffersDirectory -Dtests.method=testSeekPastEOF
-Dtests.seed=BDFA8CEDB7C93AC1 -Dtests.slow=true -Dtests.locale=sr-RS
-Dtests.timezone=Europe/Astrakhan -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.00s J0 | TestByteBuffersDirectory.testSeekPastEOF
{impl=byte array (heap)} <<<
[junit4]> Throwable #1: junit.framework.AssertionFailedError: Unexpected
exception type, expected EOFException but got
java.lang.ArrayIndexOutOfBoundsException: 1770
[junit4]>at
__randomizedtesting.SeedInfo.seed([BDFA8CEDB7C93AC1:5DBC4714B74C4450]:0)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2680)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2669)
[junit4]>at
org.apache.lucene.store.BaseDirectoryTestCase.testSeekPastEOF(BaseDirectoryTestCase.java:516)
[junit4]>at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4]>at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4]>at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4]>at
java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4]>at java.base/java.lang.Thread.run(Thread.java:844)
[junit4]> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1770
[junit4]>at
org.apache.lucene.store.ByteArrayIndexInput.readByte(ByteArrayIndexInput.java:145)
[junit4]>at
org.apache.lucene.store.BaseDirectoryTestCase.lambda$testSeekPastEOF$12(BaseDirectoryTestCase.java:518)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2675)
[junit4]>... 37 more
[...]
[junit4] 2> NOTE: test params are: codec=Lucene80,
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2c972cf9),
locale=sr-RS, timezone=Europe/Astrakhan
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 9
(64-bit)/cpus=3,threads=1,free=157933784,total=235929600
{noformat}
Also (older) from
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1645]:
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestByteBuffersDirectory -Dtests.method=testSeekPastEOF
-Dtests.seed=90B07B6267E63464 -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
-Dtests.locale=es-PR -Dtests.timezone=Australia/Currie -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.01s J1 | TestByteBuffersDirectory.testSeekPastEOF
{impl=byte array (heap)} <<<
[junit4]> Throwable #1: junit.framework.AssertionFailedError: Unexpected
exception type, expected EOFException but got
java.lang.ArrayIndexOutOfBoundsException: 1881
[junit4]> at
__randomizedtesting.SeedInfo.seed([90B07B6267E63464:70F6B09B67634AF5]:0)
[junit4]> at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2683)
[junit4]> at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2672)
[junit4]> at
org.apache.lucene.store.BaseDirectoryTestCase.testSeekPastEOF(BaseDirectoryTestCase.java:516)
[junit4]> at java.lang.Thread.run(Thread.java:748)
[junit4]> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1881
[junit4]> at
org.apache.lucene.store.ByteArrayIndexInput.readByte(ByteArrayIndexInput.java:145)
[junit4]> at
org.apache.lucene.store.BaseDirectoryTestCase.lambda$testSeekPastEOF$12(BaseDirectoryTestCase.java:518)
[junit4]> at
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2678)
[junit4]> ... 38 more
[...]
[junit4] 2> NOTE: test params are:
[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms
[
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675267#comment-16675267
]
Steve Rowe commented on SOLR-12243:
---
bq. If we are fine here, I can commit this. No problem anymore! I just did not
get any response, so I was waiting. I will run tests and do it later this
evening. I am currently travelling. At least this is a good bugfix. Everything
else here is future work (like improving the Lucene queries for the
ordered/unordered cases).
[~thetaphi]: I'll take it.
> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch,
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt,
> schema.xml, solrconfig.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
>
>
>
> edismax
> 0.4
> title^100
> title~20^5000
> title~11
> title~22^1000
> text
>
> 3-1 6-3 930%
> *:*
> 25
>
>
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin" against the
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin
> dose" or pf3:"aspirin dose ?"
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms
[
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe reassigned SOLR-12243:
-
Assignee: Steve Rowe (was: Uwe Schindler)
> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Steve Rowe
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch,
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt,
> schema.xml, solrconfig.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
>
>
>
> edismax
> 0.4
> title^100
> title~20^5000
> title~11
> title~22^1000
> text
>
> 3-1 6-3 930%
> *:*
> 25
>
>
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin" against the
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin
> dose" or pf3:"aspirin dose ?"
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms
[
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675205#comment-16675205
]
Steve Rowe commented on SOLR-12243:
---
[~ehaubert], I was holding off committing, waiting for your response to my
earlier question, which I repeat below:
-
Elizabeth, I like the idea of adding something along the lines of the text you
suggested to the ref guide. I made a few tweaks (described below) - please let
me know if you think this is okay:
{quote}
h3. Synonyms expansion in phrase queries with slop
When a phrase query with slop (e.g. {{pf}} with {{ps}}) triggers synonym
expansions, a separate clause will be generated for each combination of
synonyms. For example, with configured synonyms {{dog,canine}} and
{{cat,feline}}, the query {{"dog chased cat"}} will generate the following
phrase query clauses:
* {{"dog chased cat"}}
* {{"canine chased cat"}}
* {{"dog chased feline"}}
* {{"canine chased feline"}}{quote}
My changes: this situation happens with all synonyms, not just multi-term
synonyms; user-specified phrase queries (in q param) trigger this situation
when qs is specified, so I generalized it a bit to refer to all phrase+slop
contexts; and I think "combination" is better than "permutation" here.
-
> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch,
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt,
> schema.xml, solrconfig.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
>
>
>
> edismax
> 0.4
> title^100
> title~20^5000
> title~11
> title~22^1000
> text
>
> 3-1 6-3 930%
> *:*
> 25
>
>
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin" against the
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin
> dose" or pf3:"aspirin dose ?"
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7804) TestCloudPivotFacet failures: expected: but was:
[
https://issues.apache.org/jira/browse/SOLR-7804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670242#comment-16670242
]
Steve Rowe commented on SOLR-7804:
--
Another failure from
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/363], reproduces
for me 10/10 iterations:
{noformat}
Checking out Revision e8503da7c43b8ff3eb968c631ca2ac718a2c736b
(refs/remotes/origin/branch_7x)
[...]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCloudPivotFacet
-Dtests.method=test -Dtests.seed=E8745FA6D96CE260 -Dtests.multiplier=2
-Dtests.nightly=true -Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
-Dtests.locale=ko-KR -Dtests.timezone=Pacific/Samoa -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 37.2s J1 | TestCloudPivotFacet.test <<<
[junit4]> Throwable #1: java.lang.AssertionError:
{main(facet=true={!stats%3Dst0}bogus_not_in_any_doc_s={!stats%3Dst2}pivot_z_s1,pivot_x_s,pivot_tdt=15=true=index=0),extra(rows=0=*:*=true={!key%3Dsk1+tag%3Dst1,st2}pivot_d={!key%3Dsk2+tag%3Dst2,st3}pivot_dt1={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_miss=true&_test_sort=index)}
==> Stddev of sk2 => pivot_z_s1,pivot_x_s,pivot_tdt:
{params(rows=0),defaults({main({main(rows=0=*:*=true={!key%3Dsk1+tag%3Dst1,st2}pivot_d={!key%3Dsk2+tag%3Dst2,st3}pivot_dt1={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_miss=true&_test_sort=index),extra(fq={!term+f%3Dpivot_z_s1}a)}),extra(fq={!term+f%3Dpivot_x_s}b)})}
expected:<0.0> but was:<20724.30287367949>
[junit4]>at
__randomizedtesting.SeedInfo.seed([E8745FA6D96CE260:6020607C77908F98]:0)
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:292)
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.test(TestCloudPivotFacet.java:239)
[junit4]>at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
[junit4]>at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
[junit4]>at java.lang.Thread.run(Thread.java:748)
[junit4]> Caused by: java.lang.AssertionError: Stddev of sk2 =>
pivot_z_s1,pivot_x_s,pivot_tdt:
{params(rows=0),defaults({main({main(rows=0=*:*=true={!key%3Dsk1+tag%3Dst1,st2}pivot_d={!key%3Dsk2+tag%3Dst2,st3}pivot_dt1={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_miss=true&_test_sort=index),extra(fq={!term+f%3Dpivot_z_s1}a)}),extra(fq={!term+f%3Dpivot_x_s}b)})}
expected:<0.0> but was:<20724.30287367949>
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.assertNumerics(TestCloudPivotFacet.java:743)
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotStats(TestCloudPivotFacet.java:402)
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotData(TestCloudPivotFacet.java:350)
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:313)
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:324)
[junit4]>at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:282)
[junit4]>... 42 more
[...]
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
{dense_pivot_y_s=FSTOrd50, multiDefault=FSTOrd50,
pivot_x_s=PostingsFormat(name=Memory), pivot_b1=FSTOrd50,
pivot_x_s1=PostingsFormat(name=Direct), pivot_y_s=FSTOrd50,
pivot_z_s=PostingsFormat(name=MockRandom),
pivot_y_s1=PostingsFormat(name=Memory), pivot_z_s1=FSTOrd50,
id=PostingsFormat(name=MockRandom),
dense_pivot_x_s1=PostingsFormat(name=Direct),
pivot_b=PostingsFormat(name=MockRandom),
dense_pivot_y_s1=PostingsFormat(name=Memory),
dense_pivot_x_s=PostingsFormat(name=MockRandom)},
docValues:{pivot_dt1=DocValuesFormat(name=Direct),
dense_pivot_i1=DocValuesFormat(name=Asserting),
pivot_l1=DocValuesFormat(name=Lucene70),
range_facet_l_dv=DocValuesFormat(name=Asserting),
multiDefault=DocValuesFormat(name=Direct),
pivot_f1=DocValuesFormat(name=Direct),
intDefault=DocValuesFormat(name=Lucene70),
pivot_d1=DocValuesFormat(name=Lucene70),
pivot_x_s=DocValuesFormat(name=Memory), pivot_b1=DocValuesFormat(name=Direct),
pivot_dt=DocValuesFormat(name=Asserting),
dense_pivot_ti1=DocValuesFormat(name=Asserting),
range_facet_l=DocValuesFormat(name=Lucene70),
pivot_y_s=DocValuesFormat(name=Direct),
pivot_x_s1=DocValuesFormat(name=Lucene70),
pivot_z_s=DocValuesFormat(name=Asserting),
dense_pivot_ti=DocValuesFormat(name=Lucene70),
pivot_z_s1=DocValuesFormat(name=Direct),
pivot_tl1=DocValuesFormat(name=Direct),
[jira] [Commented] (SOLR-12259) Robustly upgrade indexes
[
https://issues.apache.org/jira/browse/SOLR-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670063#comment-16670063
]
Steve Rowe commented on SOLR-12259:
---
[~erickerickson]: maybe make the Lucene MergePolicy have an abstract method
{{boolean shouldRewrite(SegmentCommitInfo)}}?
> Robustly upgrade indexes
>
>
> Key: SOLR-12259
> URL: https://issues.apache.org/jira/browse/SOLR-12259
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> The general problem statement is that the current upgrade path is trappy and
> cumbersome. It would be a great help "in the field" to make the upgrade
> process less painful.
> Additionally one of the most common things users want to do is enable
> docValues, but currently they often have to re-index.
> Issues:
> 1> if I upgrade from 5x to 6x and then 7x, theres no guarantee that when I go
> to 7x all the segments have been rewritten in 6x format. Say I have a segment
> at max size that has no deletions. It'll never be rewritten until it has
> deleted docs. And perhaps 50% deleted docs currently.
> 2> IndexUpgraderTool explicitly does a forcemerge to 1 segment, which is bad.
> 3> in a large distributed system, running IndexUpgraderTool on all the nodes
> is cumbersome even if <2> is acceptable.
> 4> Users who realize specifying docValues on a field would be A Good Thing
> have to re-index. We have UninvertDocValuesMergePolicyFactory. Wouldn't it be
> nice to be able to have this done all at once without forceMerging to one
> segment.
> Proposal:
> Somehow avoid the above. Currently LUCENE-7976 is a start in that direction.
> It will make TMP respect max segments size so can avoid forceMerges that
> result in one segment. What it does _not_ do is rewrite segments with zero
> (or a small percentage) deleted documents.
> So it doesn't seem like a huge stretch to be able to specify to TMP the
> option to rewrite segments that have no deleted documents. Perhaps a new
> parameter to optimize?
> This would likely require another change to TMP or whatever.
> So upgrading to a new solr would look like
> 1> install the new Solr
> 2> execute
> "http://node:port/solr/collection_or_core/update?optimize=true=true;
> What's not clear to me is whether we'd require
> UninvertDocValuesMergePolicyFactory to be specified and wrap TMP or not.
> Anyway, let's discuss. I'll create yet another LUCENE JIRA for TMP do rewrite
> all segments that I'll link.
> I'll also link several other JIRAs in here, they're coalescing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms
[
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668790#comment-16668790
]
Steve Rowe commented on SOLR-12243:
---
{quote}I think the permutation problem is not new with the recent Lucene fixes.
This problem should also have happened with Span expansions, right? Maybe we
should add an option to limit the number of phrase expansions (as a safety
feature). If those limits are reached, the phrase expansion should be stopped
(maybe then only bigrams and no trigrams).
{quote}
I don't know how Span queries are rewritten, or how the search time complexity
would work out, but AFAIK the Lucene fixes didn't change the permutation
problem, just recast it as explicit clauses.
As far as safety is concerned, doesn't the Boolean clause limit already apply
in this case, since the generated query is a BooleanQuery of PhraseQuery-s?
Elizabeth, I like the idea of adding something along the lines of the text you
suggested to the ref guide. I made a few tweaks (described below) - please let
me know if you think this is okay:
{quote}
h3. Synonyms expansion in phrase queries with slop
When a phrase query with slop (e.g. {{pf}} with {{ps}}) triggers synonym
expansions, a separate clause will be generated for each combination of
synonyms. For example, with configured synonyms {{dog,canine}} and
{{cat,feline}}, the query {{"dog chased cat"}} will generate the following
phrase query clauses:
* {{"dog chased cat"}}
* {{"canine chased cat"}}
* {{"dog chased feline"}}
* {{"canine chased feline"}}{quote}
My changes: this situation happens with all synonyms, not just multi-term
synonyms; user-specified phrase queries (in {{q}} param) trigger this situation
when {{qs}} is specified, so I generalized it a bit to refer to all phrase+slop
contexts; and I think "combination" is better than "permutation" here.
> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch,
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt,
> schema.xml, solrconfig.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
>
>
>
> edismax
> 0.4
> title^100
> title~20^5000
> title~11
> title~22^1000
> text
>
> 3-1 6-3 930%
> *:*
> 25
>
>
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin" against the
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin
> dose" or pf3:"aspirin dose ?"
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12861) Add Solr factory for new ByteBuffersDirectory
[ https://issues.apache.org/jira/browse/SOLR-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667126#comment-16667126 ] Steve Rowe commented on SOLR-12861: --- Thanks [~dweiss], makes sense. I included the following in the Solr upgrade notes: bq. SOLR-12861: Added a Solr factory for ByteBuffersDirectory, which will replace deprecated RAMDirectory in a future version of Solr. Should I instead name 9.0 here as the version in which RAMDirectory will be removed? > Add Solr factory for new ByteBuffersDirectory > - > > Key: SOLR-12861 > URL: https://issues.apache.org/jira/browse/SOLR-12861 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Major > Attachments: SOLR-12861.patch > > > LUCENE-8438 and sub-tasks introduced ByteBuffersDirectory, a RAMDirectory > replacement. Solr needs a factory class in order to expose it to end-users. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms
[
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665837#comment-16665837
]
Steve Rowe commented on SOLR-12243:
---
{quote}
I think something is not right, but am not sure what.
[...]
Put breakpoints at ExtendedDismaxQParser.java in getQuery, and it looks like it
is getting a NullPointerException and falling out at ln.1449
{quote}
In addition to [~ehaubert]'s above-described manual test,
{{TestMultiWordSynonyms.testPf3WithoutReordering()}} in the patch was failing
with the same symptoms.
The problem: The LUCENE-8531 changes cause {{QueryBuilder}} to produce a new
kind of query structure for a phrase with multi-term synonyms and non-zero
slop: a {{BooleanQuery}} of {{PhraseQuery}}-s.
{{ExtendedDismaxQParser.getQuery()}} assumes that {{BooleanQuery}}-s always
consist of {{TermQuery}}-s, and so unconditionally sets the query's
minShouldMatch, but since the parser used to construct the {{pf3}} phrase
shingles had never had its minShouldMatch spec set, it remained null, causing
an NPE when trim was called on it in {{SolrPluginUtils.setMinShouldMatch()}}.
I've attached a modified version of Elizabeth's patch that includes an
{{ExtendedDismaxQParser.getQuery()}} fix: don't set a {{BooleanQuery}}'s
minShouldMatch when {{type==QType.PHRASE}}. The modified patch also uncomments
{{TestMultiWordSynonyms.testPf3WithReordering()}}. All the tests in
{{TestMultiWordSynonyms}} now pass with the patch. I haven't tried to run all
Solr tests yet.
{quote}
Hi, the Lucene issue was committed. I think we can now test this. Nevertheless,
according to my understanding, as for slop!=0 it no longer creates span
queries, the bug is fixed anyways. For slop=0 it creates (faster) span queries,
so the fixes here should apply.
Nevertheless there should be a test for slop=0 and slop!=0 in Edismax tests.
{quote}
Next week I'll look at adding what else ^^ needs testing.
> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch,
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt,
> schema.xml, solrconfig.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
>
>
>
> edismax
> 0.4
> title^100
> title~20^5000
> title~11
> title~22^1000
> text
>
> 3-1 6-3 930%
> *:*
> 25
>
>
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin" against the
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin
> dose" or pf3:"aspirin dose ?"
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms
[
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe updated SOLR-12243:
--
Attachment: SOLR-12243.patch
> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch,
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt,
> schema.xml, solrconfig.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
>
>
>
> edismax
> 0.4
> title^100
> title~20^5000
> title~11
> title~22^1000
> text
>
> 3-1 6-3 930%
> *:*
> 25
>
>
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin" against the
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin
> dose" or pf3:"aspirin dose ?"
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12894) Solr documention for Java Vendors
[
https://issues.apache.org/jira/browse/SOLR-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665414#comment-16665414
]
Steve Rowe edited comment on SOLR-12894 at 10/26/18 4:57 PM:
-
bq. If we are to reccomend users to use OpenJDK maybe we should have a link so
that users can download them on Windows , Linux and Mac?
https://openjdk.java.net/install/ only has packages for linux .
Binaries for more platforms available here:
https://adoptopenjdk.net/releases.html
bq. How does this change look?
+1, except maybe update the {{java \-version}} output in
{{solr-system-requirements.adoc}} to something more modern? E.g. mine from
this issue's description (though it's from the Debian 8 {{openjdk-8-\*}}
packages, so not the most recent).
was (Author: steve_rowe):
bq. If we are to reccomend users to use OpenJDK maybe we should have a link so
that users can download them on Windows , Linux and Mac?
https://openjdk.java.net/install/ only has packages for linux .
Binaries for more platforms available here:
https://adoptopenjdk.net/releases.html
bq. How does this change look?
+1, except maybe update the {{java -version}} output in
{{solr-system-requirements.adoc}} to something more modern? E.g. mine from
this issue's description (though it's from the Debian 8 {{openjdk-8-\*}}
packages, so not the most recent).
> Solr documention for Java Vendors
> -
>
> Key: SOLR-12894
> URL: https://issues.apache.org/jira/browse/SOLR-12894
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: documentation
>Reporter: Varun Thacker
>Priority: Major
> Attachments: SOLR-12894.patch, SOLR-12894.patch
>
>
> I was asked a question recently - "Is using OpenJDK safe with Solr 7.4" . To
> which my answer was yes . This was after I checked with Steve on which
> OpenJDK version runs on his jenkins
> For refrerence it currently uses -
> {code:java}
> openjdk version "1.8.0_171"
> OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-1~bpo8+1-b11)
> OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode){code}
>
> Solr's ref guide (
> [https://lucene.apache.org/solr/guide/installing-solr.html#got-java|https://lucene.apache.org/solr/guide/6_6/installing-solr.html#got-java]
> ) mentions using Oracle 1.8 or higher .
>
> We should mention that both Oracle JDKs and Open JDKs are tested. Perhaps
> even have a compatibility matrix
>
> Also we should note that Java 9 and 10 are short term releases . Hence remove
> wording that using Java8+ with more spefic versions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12894) Solr documention for Java Vendors
[
https://issues.apache.org/jira/browse/SOLR-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665414#comment-16665414
]
Steve Rowe commented on SOLR-12894:
---
bq. If we are to reccomend users to use OpenJDK maybe we should have a link so
that users can download them on Windows , Linux and Mac?
https://openjdk.java.net/install/ only has packages for linux .
Binaries for more platforms available here:
https://adoptopenjdk.net/releases.html
bq. How does this change look?
+1, except maybe update the {{java -version}} output in
{{solr-system-requirements.adoc}} to something more modern? E.g. mine from
this issue's description (though it's from the Debian 8 {{openjdk-8-\*}}
packages, so not the most recent).
> Solr documention for Java Vendors
> -
>
> Key: SOLR-12894
> URL: https://issues.apache.org/jira/browse/SOLR-12894
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: documentation
>Reporter: Varun Thacker
>Priority: Major
> Attachments: SOLR-12894.patch, SOLR-12894.patch
>
>
> I was asked a question recently - "Is using OpenJDK safe with Solr 7.4" . To
> which my answer was yes . This was after I checked with Steve on which
> OpenJDK version runs on his jenkins
> For refrerence it currently uses -
> {code:java}
> openjdk version "1.8.0_171"
> OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-1~bpo8+1-b11)
> OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode){code}
>
> Solr's ref guide (
> [https://lucene.apache.org/solr/guide/installing-solr.html#got-java|https://lucene.apache.org/solr/guide/6_6/installing-solr.html#got-java]
> ) mentions using Oracle 1.8 or higher .
>
> We should mention that both Oracle JDKs and Open JDKs are tested. Perhaps
> even have a compatibility matrix
>
> Also we should note that Java 9 and 10 are short term releases . Hence remove
> wording that using Java8+ with more spefic versions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12868) Request forwarding for v2 API is broken
[
https://issues.apache.org/jira/browse/SOLR-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665342#comment-16665342
]
Steve Rowe commented on SOLR-12868:
---
As of commit {{f33be7a}} on this issue, {{TestV2Request}} is failing nearly
100% of the time, e.g. from
[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7585]:
{noformat}
Checking out Revision f33be7a172d7b4596530d8cb925ba6dd1f1f53f0
(refs/remotes/origin/master)
[...]
[junit4] Suite: org.apache.solr.client.solrj.request.TestV2Request
[...]
[junit4] 1> livenodes: [127.0.0.1:61086_solr, 127.0.0.1:61091_solr,
127.0.0.1:61096_solr, 127.0.0.1:61105_solr]
[junit4] 1> 04:51:45.352 [qtp14589558-428] ERROR
org.apache.solr.api.V2HttpCall - >> path: '/c/_introspect'
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestV2Request
-Dtests.method=testHttpSolrClient -Dtests.seed=F4038E7A593F7B4D
-Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=America/Winnipeg
-Dtests.asserts=true -Dtests.file.encoding=Cp1252
[junit4] FAILURE 6.31s J0 | TestV2Request.testHttpSolrClient <<<
[junit4]> Throwable #1: java.lang.AssertionError: expected:<0> but
was:<1>
[junit4]>at
__randomizedtesting.SeedInfo.seed([F4038E7A593F7B4D:2C1B9650F80B696A]:0)
[junit4]>at
org.apache.solr.client.solrj.request.TestV2Request.doTest(TestV2Request.java:105)
[junit4]>at
org.apache.solr.client.solrj.request.TestV2Request.testHttpSolrClient(TestV2Request.java:70)
[junit4]>at java.lang.Thread.run(Thread.java:748)
[junit4] 1> 04:51:51.726 [qtp4981319-439] ERROR
org.apache.solr.api.V2HttpCall - >> path: '/c/_introspect'
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestV2Request
-Dtests.method=testCloudSolrClient -Dtests.seed=F4038E7A593F7B4D
-Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=America/Winnipeg
-Dtests.asserts=true -Dtests.file.encoding=Cp1252
[junit4] FAILURE 6.02s J0 | TestV2Request.testCloudSolrClient <<<
[junit4]> Throwable #1: java.lang.AssertionError: expected:<0> but
was:<1>
[junit4]>at
__randomizedtesting.SeedInfo.seed([F4038E7A593F7B4D:6DF505CAFC8CF326]:0)
[junit4]>at
org.apache.solr.client.solrj.request.TestV2Request.doTest(TestV2Request.java:105)
[junit4]>at
org.apache.solr.client.solrj.request.TestV2Request.testCloudSolrClient(TestV2Request.java:77)
[junit4]>at java.lang.Thread.run(Thread.java:748)
[junit4] 1> 04:51:53.783
[SUITE-TestV2Request-seed#[F4038E7A593F7B4D]-worker] ERROR
org.apache.zookeeper.server.ZooKeeperServer - ZKShutdownHandler is not
registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN
server state changes
[junit4] 2> NOTE: leaving temporary files on disk at:
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.request.TestV2Request_F4038E7A593F7B4D-001
[junit4] 2> Oct 26, 2018 4:51:53 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 10 leaked
thread(s).
[junit4] 2> Oct 26, 2018 4:53:13 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at
org.apache.solr.client.solrj.request.TestV2Request:
[junit4] 2>1) Thread[id=550, name=Connection evictor,
state=TIMED_WAITING, group=TGRP-TestV2Request]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Oct 26, 2018 4:53:13 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2>1) Thread[id=550, name=Connection evictor,
state=TIMED_WAITING, group=TGRP-TestV2Request]
[junit4] 2> Oct 26, 2018 4:53:13 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=CheapBastard,
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@1e6d741),
locale=lv, timezone=America/Winnipeg
[junit4] 2> NOTE: Windows 10 10.0 x86/Oracle Corporation 1.8.0_172
(32-bit)/cpus=3,threads=1,free=87644232,total=185597952
[junit4] 2> NOTE: All tests run in this JVM: [TestFastWriter,
CollectionAdminRequestRequiredParamsTest, UniformDistributionEvaluatorTest,
ArcCosineEvaluatorTest, LessThanEqualToEvaluatorTest, SolrExampleEmbeddedTest,
TestTimeSource, TestXMLEscaping, GraphExpressionTest, DivideEvaluatorTest,
CloudSolrClientBuilderTest, AscEvaluatorTest, FieldAnalysisResponseTest,
TestV2Request]
[junit4] 2> NOTE: reproduce
[jira] [Commented] (SOLR-12259) Robustly upgrade indexes
[ https://issues.apache.org/jira/browse/SOLR-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662584#comment-16662584 ] Steve Rowe commented on SOLR-12259: --- In addition to the issues mentioned on SOLR-11127 (upgrading .system collection schema from Trie to Points fields), Solr needs somehow to be able to re-index the .system collection on major version upgrade, because of the unsupported index version N-2 problem. > Robustly upgrade indexes > > > Key: SOLR-12259 > URL: https://issues.apache.org/jira/browse/SOLR-12259 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > > The general problem statement is that the current upgrade path is trappy and > cumbersome. It would be a great help "in the field" to make the upgrade > process less painful. > Additionally one of the most common things users want to do is enable > docValues, but currently they often have to re-index. > Issues: > 1> if I upgrade from 5x to 6x and then 7x, theres no guarantee that when I go > to 7x all the segments have been rewritten in 6x format. Say I have a segment > at max size that has no deletions. It'll never be rewritten until it has > deleted docs. And perhaps 50% deleted docs currently. > 2> IndexUpgraderTool explicitly does a forcemerge to 1 segment, which is bad. > 3> in a large distributed system, running IndexUpgraderTool on all the nodes > is cumbersome even if <2> is acceptable. > 4> Users who realize specifying docValues on a field would be A Good Thing > have to re-index. We have UninvertDocValuesMergePolicyFactory. Wouldn't it be > nice to be able to have this done all at once without forceMerging to one > segment. > Proposal: > Somehow avoid the above. Currently LUCENE-7976 is a start in that direction. > It will make TMP respect max segments size so can avoid forceMerges that > result in one segment. What it does _not_ do is rewrite segments with zero > (or a small percentage) deleted documents. > So it doesn't seem like a huge stretch to be able to specify to TMP the > option to rewrite segments that have no deleted documents. Perhaps a new > parameter to optimize? > This would likely require another change to TMP or whatever. > So upgrading to a new solr would look like > 1> install the new Solr > 2> execute > "http://node:port/solr/collection_or_core/update?optimize=true=true; > What's not clear to me is whether we'd require > UninvertDocValuesMergePolicyFactory to be specified and wrap TMP or not. > Anyway, let's discuss. I'll create yet another LUCENE JIRA for TMP do rewrite > all segments that I'll link. > I'll also link several other JIRAs in here, they're coalescing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12879) Query Parser for MinHash/LSH
[
https://issues.apache.org/jira/browse/SOLR-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662504#comment-16662504
]
Steve Rowe commented on SOLR-12879:
---
{{QueryEqualityTest}} is failing 100% of the time w/o a seed, e.g. from
[https://builds.apache.org/job/Lucene-Solr-Tests-master/2895]:
{noformat}
Checking out Revision 3e89b7a771639aacaed6c21406624a2b27231dd7
(refs/remotes/origin/master)
[...]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=QueryEqualityTest
-Dtests.seed=40E6483843AE2CD1 -Dtests.multiplier=2 -Dtests.slow=true
-Dtests.locale=en-SG -Dtests.timezone=America/Los_Angeles -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | QueryEqualityTest (suite) <<<
[junit4]> Throwable #1: java.lang.AssertionError: testParserCoverage was
run w/o any other method explicitly testing qparser: min_hash
[junit4]>at
__randomizedtesting.SeedInfo.seed([40E6483843AE2CD1]:0)
[junit4]>at
org.apache.solr.search.QueryEqualityTest.afterClassParserCoverageTest(QueryEqualityTest.java:59)
[junit4]>at java.lang.Thread.run(Thread.java:748)
{noformat}
> Query Parser for MinHash/LSH
>
>
> Key: SOLR-12879
> URL: https://issues.apache.org/jira/browse/SOLR-12879
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Affects Versions: master (8.0)
>Reporter: Andy Hind
>Assignee: Tommaso Teofili
>Priority: Major
> Fix For: master (8.0)
>
> Attachments: minhash.filter.adoc.fragment, minhash.patch
>
>
> Following on from https://issues.apache.org/jira/browse/LUCENE-6968, provide
> a query parser that builds queries that provide a measure of Jaccard
> similarity. The initial patch includes banded queries that were also proposed
> on the original issue.
>
> I have one outstanding questions:
> * Should the score from the overall query be normalised?
> Note, that the band count is currently approximate and may be one less than
> in practise.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
