[jira] [Commented] (LUCENE-7476) Fix transient failure in JapaneseNumberFilter run from TestFactories
[ https://issues.apache.org/jira/browse/LUCENE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562617#comment-15562617 ] ASF subversion and git services commented on LUCENE-7476: - Commit 257ea3423f5b03495e08a8aa4263f9520deb662b in lucene-solr's branch refs/heads/master from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=257ea34 ] LUCENE-7476: JapaneseNumberFilter should not invoke incrementToken on its input after it's exhausted > Fix transient failure in JapaneseNumberFilter run from TestFactories > > > Key: LUCENE-7476 > URL: https://issues.apache.org/jira/browse/LUCENE-7476 > Project: Lucene - Core > Issue Type: Bug > Components: core/other >Affects Versions: 6.2.1 >Reporter: Andy Hind >Priority: Trivial > Fix For: master (7.0), 6.3 > > Attachments: LUCENE-7476.patch, LUCENE-7476.patch, LUCENE-7476.patch > > > Repeatedly running TestFactories show this test to fail ~10% of the time. > I believe the fix is trivial and related to loosing the state of the > underlying input stream when testing some analyzer life cycle flows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7476) Fix transient failure in JapaneseNumberFilter run from TestFactories
[ https://issues.apache.org/jira/browse/LUCENE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562607#comment-15562607 ] ASF subversion and git services commented on LUCENE-7476: - Commit e585c84f244f4b8c8752152e9f232eba192041dc in lucene-solr's branch refs/heads/branch_6x from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e585c84 ] LUCENE-7476: JapaneseNumberFilter should not invoke incrementToken on its input after it's exhausted > Fix transient failure in JapaneseNumberFilter run from TestFactories > > > Key: LUCENE-7476 > URL: https://issues.apache.org/jira/browse/LUCENE-7476 > Project: Lucene - Core > Issue Type: Bug > Components: core/other >Affects Versions: 6.2.1 >Reporter: Andy Hind >Priority: Trivial > Attachments: LUCENE-7476.patch, LUCENE-7476.patch, LUCENE-7476.patch > > > Repeatedly running TestFactories show this test to fail ~10% of the time. > I believe the fix is trivial and related to loosing the state of the > underlying input stream when testing some analyzer life cycle flows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7476) Fix transient failure in JapaneseNumberFilter run from TestFactories
[
https://issues.apache.org/jira/browse/LUCENE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562489#comment-15562489
]
Michael McCandless commented on LUCENE-7476:
And your original patch indeed fixes the issue.
The problems is the {{incrementToken}} that {{JapaneseNumberFilter}} does lower
down (in the while loop): in this case, today, the preceding tokenizer can
return false while {{JapaneseNumberFilter}} returns true, causing a future call
to {{incrementToken}} to illegally call {{incrementToken}} from the preceding
tokenizer.
> Fix transient failure in JapaneseNumberFilter run from TestFactories
>
>
> Key: LUCENE-7476
> URL: https://issues.apache.org/jira/browse/LUCENE-7476
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/other
>Affects Versions: 6.2.1
>Reporter: Andy Hind
>Priority: Trivial
> Attachments: LUCENE-7476.patch, LUCENE-7476.patch
>
>
> Repeatedly running TestFactories show this test to fail ~10% of the time.
> I believe the fix is trivial and related to loosing the state of the
> underlying input stream when testing some analyzer life cycle flows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7476) Fix transient failure in JapaneseNumberFilter run from TestFactories
[
https://issues.apache.org/jira/browse/LUCENE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561858#comment-15561858
]
Andy Hind commented on LUCENE-7476:
---
Running the tests 100 times via ant produces no issue. This seems to an eclipse
configuration issue.
{code}
ant test -Dtestcase=TestFactories -Dtests.method=test -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
{code}
> Fix transient failure in JapaneseNumberFilter run from TestFactories
>
>
> Key: LUCENE-7476
> URL: https://issues.apache.org/jira/browse/LUCENE-7476
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/other
>Affects Versions: 6.2.1
>Reporter: Andy Hind
>Priority: Trivial
> Attachments: LUCENE-7476.patch
>
>
> Repeatedly running TestFactories show this test to fail ~10% of the time.
> I believe the fix is trivial and related to loosing the state of the
> underlying input stream when testing some analyzer life cycle flows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7476) Fix transient failure in JapaneseNumberFilter run from TestFactories
[
https://issues.apache.org/jira/browse/LUCENE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561799#comment-15561799
]
Andy Hind commented on LUCENE-7476:
---
I spotted this running org.apache.lucene.analysis.core.TestFactories with
@Repeat (iterations = 100) from eclipse
I just got 9 failures running this again. It is odd that I do not see them in
the build failures.
I believe the 9 fails are all the same
{code}
java.lang.IllegalStateException: incrementToken() called while in wrong state:
INCREMENT_FALSE
at
__randomizedtesting.SeedInfo.seed([18C3960FB72D4F07:2AB7AA6A139D55E3]:0)
at org.apache.lucene.analysis.MockTokenizer.fail(MockTokenizer.java:125)
at
org.apache.lucene.analysis.MockTokenizer.incrementToken(MockTokenizer.java:136)
at
org.apache.lucene.analysis.ja.JapaneseNumberFilter.incrementToken(JapaneseNumberFilter.java:152)
at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:716)
at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:627)
at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:525)
at
org.apache.lucene.analysis.core.TestFactories.doTestTokenFilter(TestFactories.java:108)
at
org.apache.lucene.analysis.core.TestFactories.test(TestFactories.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
[jira] [Commented] (LUCENE-7476) Fix transient failure in JapaneseNumberFilter run from TestFactories
[
https://issues.apache.org/jira/browse/LUCENE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1201#comment-1201
]
Michael McCandless commented on LUCENE-7476:
Thanks [~andyhind].
Do you have a {{Reproduce with: ...}} line showing the test failure? What's
spooky about this is that one should never call {{incrementToken}} after it has
already returned false, and token filters should not be required / expected to
guard against such abuse (this patch), so I'd like to understand who is abusing
{{JapaneseNumberFilter}}'s {{incrementToken}} in this test failure.
> Fix transient failure in JapaneseNumberFilter run from TestFactories
>
>
> Key: LUCENE-7476
> URL: https://issues.apache.org/jira/browse/LUCENE-7476
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/other
>Affects Versions: 6.2.1
>Reporter: Andy Hind
>Priority: Trivial
> Attachments: LUCENE-7476.patch
>
>
> Repeatedly running TestFactories show this test to fail ~10% of the time.
> I believe the fix is trivial and related to loosing the state of the
> underlying input stream when testing some analyzer life cycle flows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7476) Fix transient failure in JapaneseNumberFilter run from TestFactories
[ https://issues.apache.org/jira/browse/LUCENE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15551409#comment-15551409 ] Andy Hind commented on LUCENE-7476: --- Patch supplied - TestFactories ran with more then 700 seeds without error. > Fix transient failure in JapaneseNumberFilter run from TestFactories > > > Key: LUCENE-7476 > URL: https://issues.apache.org/jira/browse/LUCENE-7476 > Project: Lucene - Core > Issue Type: Bug > Components: core/other >Affects Versions: 6.2.1 >Reporter: Andy Hind >Priority: Trivial > Attachments: LUCENE-7476.patch > > > Repeatedly running TestFactories show this test to fail ~10% of the time. > I believe the fix is trivial and related to loosing the state of the > underlying input stream when testing some analyzer life cycle flows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
