[JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 443 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/443/ 1 tests failed. REGRESSION: org.apache.solr.update.AutoCommitTest.testCommitWithin Error Message: should not be there query failed XPath: //result[@numFound=0] xml response was: ?xml version=1.0 encoding=UTF-8? response lst name=responseHeaderint name=status0/intint name=QTime0/int/lstresult name=response numFound=1 start=0docint name=id530/intint name=range_facet_si530/intarr name=range_facet_llong530/long/arrarr name=range_facet_sllong530/long/arrarr name=field_tstrwhat's inside?/str/arrarr name=subjectstrinfo/str/arrdate name=timestamp2012-06-29T06:13:08.824Z/datearr name=multiDefaultstrmuLti-Default/str/arrint name=intDefault42/int/doc/result /response request was: start=0q=id:530qt=standardrows=20version=2.2 Stack Trace: java.lang.AssertionError: should not be there query failed XPath: //result[@numFound=0] xml response was: ?xml version=1.0 encoding=UTF-8? response lst name=responseHeaderint name=status0/intint name=QTime0/int/lstresult name=response numFound=1 start=0docint name=id530/intint name=range_facet_si530/intarr name=range_facet_llong530/long/arrarr name=range_facet_sllong530/long/arrarr name=field_tstrwhat's inside?/str/arrarr name=subjectstrinfo/str/arrdate name=timestamp2012-06-29T06:13:08.824Z/datearr name=multiDefaultstrmuLti-Default/str/arrint name=intDefault42/int/doc/result /response request was: start=0q=id:530qt=standardrows=20version=2.2 at __randomizedtesting.SeedInfo.seed([F0BC1EE142D62018:4A6E7199C1F8CE0D]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:269) at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:337) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) at
[jira] [Commented] (LUCENE-4172) clean up redundant throws clauses
[ https://issues.apache.org/jira/browse/LUCENE-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403733#comment-13403733 ] Steven Rowe commented on LUCENE-4172: - bq. What's wrong with?: import package.*; – especially when there are many classes? Some reasons given here: [http://stackoverflow.com/questions/147454/why-is-using-a-wild-card-with-a-java-import-statement-bad]: namespace polution; package additions can trigger compilation failure; compilation is slower. My IDE (IntelliJ) automatically folds imports, so assuming most devs have a similar experience, many classes in a package don't cause actual clutter. bq. And is there a coding standard that Lucene/Solr adheres to? I am unaware. The only standard I've seen referred to is Sun's Java style rules, with the exception of 2 spaces per indent level instead of 4. clean up redundant throws clauses - Key: LUCENE-4172 URL: https://issues.apache.org/jira/browse/LUCENE-4172 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Assignee: Steven Rowe Fix For: 4.0, 5.0 Attachments: LUCENE-4172.patch, LUCENE-4172.patch, LUCENE-4172.patch examples are things like ctors that list throws XYZException but actually dont, and things like 'throws CorruptIndex, LockObtainedFailed, IOException' when all of these are actually IOException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1090 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/ 1 tests failed. REGRESSION: org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463) at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) Build Log: [...truncated 3668 lines...] [junit4] Suite: org.apache.lucene.analysis.ngram.NGramTokenizerTest [junit4] ERROR 3162s J1 | NGramTokenizerTest.testRandomStrings [junit4] Throwable #1: java.lang.RuntimeException: some thread(s) failed [junit4]at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0) [junit4]at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463) [junit4]at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106) [junit4]at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4]at
Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1090 - Failure!
I've seen this one on my build server too. Dawid On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server jenk...@sd-datasolutions.de wrote: Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/ 1 tests failed. REGRESSION: org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463) at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) Build Log: [...truncated 3668 lines...] [junit4] Suite: org.apache.lucene.analysis.ngram.NGramTokenizerTest [junit4] ERROR 3162s J1 | NGramTokenizerTest.testRandomStrings [junit4] Throwable #1: java.lang.RuntimeException: some thread(s) failed [junit4] at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0) [junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463) [junit4] at
Re: Make ivy search maven repo1/repo2?
I don't know what you mean by use it from Solr but I guess regular
targets to assemble the example/ solr distribution will update Lucene
jars. So ant example or dist* should work.
Dawid
On Thu, Jun 28, 2012 at 9:21 PM, Lance Norskog goks...@gmail.com wrote:
The ant build. I have added a new Lucene class and would like to use
it from Solr. What ant build target will propagate the new class into
the Lucene jars which are used by Solr?
On Thu, Jun 28, 2012 at 12:29 AM, Dawid Weiss
dawid.we...@cs.put.poznan.pl wrote:
Are you talking about the maven build or the ant build?
D.
On Thu, Jun 28, 2012 at 9:23 AM, Lance Norskog goks...@gmail.com wrote:
It is working now. I did a complete update- that might be it.
Now, the next question: what build target makes Solr use new Lucene jars?
On Wed, Jun 27, 2012 at 11:37 PM, Dawid Weiss
dawid.we...@cs.put.poznan.pl wrote:
I think it is? By default one of the resolvers pulls from
repo1.maven.org -- look at ivy-settings.xml:
include url=${ivy.default.settings.dir}/ivysettings-public.xml/
include url=${ivy.default.settings.dir}/ivysettings-shared.xml/
include url=${ivy.default.settings.dir}/ivysettings-local.xml/
include url=${ivy.default.settings.dir}/ivysettings-main-chain.xml/
the main chain includes shared repos and shared is:
ibiblio name=public m2compatible=true/
And ibiblio is by default pulling from maven repos:
http://ant.apache.org/ivy/history/latest-milestone/resolver/ibiblio.html
Dawid
On Thu, Jun 28, 2012 at 8:31 AM, Lance Norskog goks...@gmail.com wrote:
My mistake. I meant that Ivy was not using the maven.org repos at all.
On Wed, Jun 27, 2012 at 11:17 PM, Dawid Weiss
dawid.we...@cs.put.poznan.pl wrote:
Interesting. I think they should be identical mirrors of each other.
Benson has more knowledge of Maven -- perheps he'll be able to tell if
this is a bug in the repo sync or something to be expected?
Dawid
On Thu, Jun 28, 2012 at 4:11 AM, Lance Norskog goks...@gmail.com wrote:
How can I get ivy to include the maven.org repo2 in the resolver list?
Is there a reason it is not in the list?
I ask because there is an artifact (extjwnl) which is only on repo2.
--
Lance Norskog
goks...@gmail.com
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
--
Lance Norskog
goks...@gmail.com
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
--
Lance Norskog
goks...@gmail.com
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
--
Lance Norskog
goks...@gmail.com
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: regular but not individually reproducible failure: TestIndexWriterWithThreads.testRollbackAndCommitWithThreads
this seems like a windows / files issue. We can not obtain the write lock quickly enough. Maybe we should do a retry loop in the test if we are on windows? simon On Thu, Jun 28, 2012 at 6:25 PM, Steven A Rowe sar...@syr.edu wrote: I see this roughly 50% of the time I run all Lucene core tests (Win7 64, Sun JDK 1.6.0_21 64-bit), but the seeds don't reproduce the error when it's run individually. The only unusual thing I'm doing on this machine is running some web crawls in a VirtualBox Linux VM, hosted on and writing to a different physical drive than the one I ran Lucene tests on. Here's the most recent failure: Suite: org.apache.lucene.index.TestIndexWriterWithThreads OK 0.04s J0 | TestIndexWriterWithThreads.testIOExceptionDuringAbortOnlyOnce OK 1.01s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads OK 0.97s J0 | TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreads ERROR 7.07s J0 | TestIndexWriterWithThreads.testRollbackAndCommitWithThreads Throwable #1: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([299FEFE00B1E1931:5BC9FB3E94E8E02C]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.index.TestIndexWriterWithThreads.testRollbackAndCommitWithThreads(TestIndexWriterWithThreads.java:588) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) at
[JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 310 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-64/310/ All tests passed Build Log: [...truncated 14343 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/build.xml:62: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build.xml:270: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/common-build.xml:1434: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/common-build.xml:1274: Rat problems were found! Total time: 5 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Thread leaking in junit tests
Hi, In 3.x when a Solr test case leaves threads running there's a warning printed like this: WARNING: test class left thread running: Thread[MultiThreadedHttpConnectionManager cleanup,5,main] I do not see such warnings in 4.x or trunk, how do I enable such warnings there? -- Sami Siren - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux-Java7-64 - Build # 268 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java7-64/268/ All tests passed Build Log: [...truncated 15312 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/build.xml:62: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build.xml:270: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/common-build.xml:1434: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/common-build.xml:1274: Rat problems were found! Total time: 6 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Thread leaking in junit tests
WARNING: test class left thread running: Thread[MultiThreadedHttpConnectionManager cleanup,5,main] I do not see such warnings in 4.x or trunk, how do I enable such warnings there? Open LuceneTestCase and change this attribute to true: @ThreadLeaks(failTestIfLeaking = false) this will effectively kill those leaked threads and fail the test that leaked them. Unfortunately the scenarios here are complex because threads are started within a test scope and belong to suite scope (are reused by all threads in a suite). Previously, the warnings were detected for the suite scope only which would correspond to: @ThreadLeaks(failTestIfLeaking = true, leakedThreadsBelongToSuite = true, linger = 1000) meaning: - fail a suite if threads leaked from it, - if a thread leak occurs within a test, promote it to the suite scope, - linger 1000 milliseconds before deciding a thread has leaked (this is required for thread pools and such because they never wait for their children threads to actually complete). Many Solr and a few Lucene tests currently leak threads so the flag is off. There is an issue to improve reporting of this (and handling of this) here: https://issues.apache.org/jira/browse/LUCENE-3985 I just didn't get to it yet. Or rather: I did, but it requires changes to the runner -- see here: https://github.com/carrotsearch/randomizedtesting/issues/98 Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 444 - Still Failing!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/444/ 1 tests failed. FAILED: org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures Error Message: expected:2 but was:0 Stack Trace: java.lang.AssertionError: expected:2 but was:0 at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:62) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:145) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:238) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12) Build Log: [...truncated 658 lines...] [junit4] Suite: org.apache.lucene.util.TestMaxFailuresRule [junit4] FAILURE 0.06s J0 | TestMaxFailuresRule.testMaxFailures [junit4] Throwable #1: java.lang.AssertionError: expected:2 but was:0 [junit4]at org.junit.Assert.fail(Assert.java:93) [junit4]at org.junit.Assert.failNotEquals(Assert.java:647) [junit4]at org.junit.Assert.assertEquals(Assert.java:128) [junit4]at org.junit.Assert.assertEquals(Assert.java:472) [junit4]at org.junit.Assert.assertEquals(Assert.java:456) [junit4]at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:62) [junit4]at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4]at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit4]at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4]at java.lang.reflect.Method.invoke(Method.java:601) [junit4]at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) [junit4]at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) [junit4]at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) [junit4]at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) [junit4]at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit4]at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) [junit4]at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) [junit4]at org.junit.rules.RunRules.evaluate(RunRules.java:18) [junit4]at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) [junit4]at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) [junit4]at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) [junit4]
[jira] [Updated] (LUCENE-4171) Performance improvements to Packed64
[ https://issues.apache.org/jira/browse/LUCENE-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-4171: --- Attachment: LUCENE-4171.patch Attached patch with constructor #1 size calculation change. Unit-test still passes. Performance improvements to Packed64 Key: LUCENE-4171 URL: https://issues.apache.org/jira/browse/LUCENE-4171 Project: Lucene - Java Issue Type: Sub-task Components: core/other Affects Versions: 4.0, 5.0 Environment: Tested with 4 different Intel machines Reporter: Toke Eskildsen Priority: Minor Labels: performance Attachments: LUCENE-4171.patch, LUCENE-4171.patch, LUCENE-4171.patch, LUCENE-4171.patch Original Estimate: 4h Remaining Estimate: 4h Based on the performance measurements of PackedInts.Mutable's in LUCENE-4062, a new version of Packed64 has been created that is consistently faster than the old Packed64 for both get and set. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-4.x-java7 - Build # 70 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x-java7/70/ All tests passed Build Log: [...truncated 20607 lines...] [...truncated 20497 lines...] rat-sources: [echo] [echo] * [echo] Summary [echo] --- [echo] Generated at: 2012-06-29T08:30:26+00:00 [echo] Notes: 0 [echo] Binaries: 0 [echo] Archives: 0 [echo] Standards: 47 [echo] [echo] Apache Licensed: 46 [echo] Generated Documents: 0 [echo] [echo] JavaDocs are generated and so license header is optional [echo] Generated files do not required license headers [echo] [echo] 1 Unknown Licenses [echo] [echo] *** [echo] [echo] Unapproved licenses: [echo] [echo] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/query/package-info.java [echo] [echo] *** [echo] [echo] Archives: [echo] [echo] * [echo] Files with Apache License headers will be marked AL [echo] Binary files (which do not require AL headers) will be marked B [echo] Compressed archives will be marked A [echo] Notices, licenses etc will be marked N [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SimpleSpatialFieldInfo.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SpatialFieldInfo.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SpatialStrategy.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/package-info.java [...truncated 72 lines...] [...truncated 20607 lines...] [...truncated 20607 lines...] [...truncated 20607 lines...] [...truncated 20607 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Use of @Slow annotation on tests
Hi. I would like to change the use of @Slow. Currently only this test: Test2BTerms#test2BTerms is annotated with this annotation. I don't know how slow it really is, but I would like to: * change this particular test to @Ignore or give it an assumption inside, * change the annotation group @Slow to be ON by default (so that all tests are run by default). * annotate the top of the list (tests longer than 10 seconds?) here: https://issues.apache.org/jira/browse/LUCENE-4174 with @Slow. My motivation is that I would like to be able to run a quick test run from time to time and for this I'd like to skip the slowest, most time-consuming tests. I could do it by running: ant -Dtests.slow=off currently this isn't possible. Looking forward to hearing from you. Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1091 - Still Failing!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1091/ All tests passed Build Log: [...truncated 14206 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/build.xml:62: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/build.xml:270: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/common-build.xml:1445: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/common-build.xml:1285: Rat problems were found! Total time: 5 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-4181) Change the semantics of @Slow on tests
Dawid Weiss created LUCENE-4181: --- Summary: Change the semantics of @Slow on tests Key: LUCENE-4181 URL: https://issues.apache.org/jira/browse/LUCENE-4181 Project: Lucene - Java Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial Fix For: 4.0, 5.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Thread leaking in junit tests
Thanks for the detailed explanation Dawid! -- Sami Siren On Fri, Jun 29, 2012 at 11:14 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: WARNING: test class left thread running: Thread[MultiThreadedHttpConnectionManager cleanup,5,main] I do not see such warnings in 4.x or trunk, how do I enable such warnings there? Open LuceneTestCase and change this attribute to true: @ThreadLeaks(failTestIfLeaking = false) this will effectively kill those leaked threads and fail the test that leaked them. Unfortunately the scenarios here are complex because threads are started within a test scope and belong to suite scope (are reused by all threads in a suite). Previously, the warnings were detected for the suite scope only which would correspond to: @ThreadLeaks(failTestIfLeaking = true, leakedThreadsBelongToSuite = true, linger = 1000) meaning: - fail a suite if threads leaked from it, - if a thread leak occurs within a test, promote it to the suite scope, - linger 1000 milliseconds before deciding a thread has leaked (this is required for thread pools and such because they never wait for their children threads to actually complete). Many Solr and a few Lucene tests currently leak threads so the flag is off. There is an issue to improve reporting of this (and handling of this) here: https://issues.apache.org/jira/browse/LUCENE-3985 I just didn't get to it yet. Or rather: I did, but it requires changes to the runner -- see here: https://github.com/carrotsearch/randomizedtesting/issues/98 Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Thread leaking in junit tests
My pleasure. Dealing with leaked threads is a tough area and it needs to be addressed at some point sooner or later. The warnings are not enough because nobody looks at them (for example in jenkins runs). Tests (suites) simply shouldn't leak threads, period (yes, it is my idee fixe to have this done ;). Dawid On Fri, Jun 29, 2012 at 10:56 AM, Sami Siren ssi...@gmail.com wrote: Thanks for the detailed explanation Dawid! -- Sami Siren On Fri, Jun 29, 2012 at 11:14 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: WARNING: test class left thread running: Thread[MultiThreadedHttpConnectionManager cleanup,5,main] I do not see such warnings in 4.x or trunk, how do I enable such warnings there? Open LuceneTestCase and change this attribute to true: @ThreadLeaks(failTestIfLeaking = false) this will effectively kill those leaked threads and fail the test that leaked them. Unfortunately the scenarios here are complex because threads are started within a test scope and belong to suite scope (are reused by all threads in a suite). Previously, the warnings were detected for the suite scope only which would correspond to: @ThreadLeaks(failTestIfLeaking = true, leakedThreadsBelongToSuite = true, linger = 1000) meaning: - fail a suite if threads leaked from it, - if a thread leak occurs within a test, promote it to the suite scope, - linger 1000 milliseconds before deciding a thread has leaked (this is required for thread pools and such because they never wait for their children threads to actually complete). Many Solr and a few Lucene tests currently leak threads so the flag is off. There is an issue to improve reporting of this (and handling of this) here: https://issues.apache.org/jira/browse/LUCENE-3985 I just didn't get to it yet. Or rather: I did, but it requires changes to the runner -- see here: https://github.com/carrotsearch/randomizedtesting/issues/98 Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 14796 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/14796/ All tests passed Build Log: [...truncated 19528 lines...] [...truncated 19409 lines...] rat-sources: [echo] [echo] * [echo] Summary [echo] --- [echo] Generated at: 2012-06-29T09:01:43+00:00 [echo] Notes: 0 [echo] Binaries: 0 [echo] Archives: 0 [echo] Standards: 54 [echo] [echo] Apache Licensed: 53 [echo] Generated Documents: 0 [echo] [echo] JavaDocs are generated and so license header is optional [echo] Generated files do not required license headers [echo] [echo] 1 Unknown Licenses [echo] [echo] *** [echo] [echo] Unapproved licenses: [echo] [echo] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/query/package-info.java [echo] [echo] *** [echo] [echo] Archives: [echo] [echo] * [echo] Files with Apache License headers will be marked AL [echo] Binary files (which do not require AL headers) will be marked B [echo] Compressed archives will be marked A [echo] Notices, licenses etc will be marked N [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SimpleSpatialFieldInfo.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SpatialFieldInfo.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SpatialStrategy.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/bbox/AreaSimilarity.java [...truncated 81 lines...] [...truncated 19528 lines...] [...truncated 19528 lines...] [...truncated 19528 lines...] [...truncated 19528 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-4160) Bring back the functional equivalent of tests.iters.min
[ https://issues.apache.org/jira/browse/LUCENE-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss resolved LUCENE-4160. - Resolution: Fixed Fix Version/s: 4.0 Bring back the functional equivalent of tests.iters.min --- Key: LUCENE-4160 URL: https://issues.apache.org/jira/browse/LUCENE-4160 Project: Lucene - Java Issue Type: Bug Components: general/test Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial Fix For: 4.0, 5.0 Attachments: LUCENE-4160.patch What is needed is effectively saying: repeat this test N times, but stop once you hit a failure. Previously it was tests.iters.min=X which is (still) kind of confusing to me because I don't understand how X is related to the original question. I propose to implement a boolean tests.fastfail which would ignore any tests running on the same JVM after the first failure has been hit. Those with fond memories of tests.iters.min speak up, please. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 311 - Still Failing!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-64/311/ All tests passed Build Log: [...truncated 14319 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/build.xml:62: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build.xml:270: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/common-build.xml:1434: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/common-build.xml:1274: Rat problems were found! Total time: 5 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Thread leaking in junit tests
On Fri, Jun 29, 2012 at 12:01 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: My pleasure. Dealing with leaked threads is a tough area and it needs to be addressed at some point sooner or later. The warnings are not enough because nobody looks at them (for example in jenkins runs). Tests (suites) simply shouldn't leak threads, period (yes, it is my idee fixe to have this done ;). I absolutely agree, even though IMO warning is better than silence. Anyway I got what I wanted: a failure in my test ;) -- Sami Siren - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Thread leaking in junit tests
I absolutely agree, even though IMO warning is better than silence. I plan to work on it soon, we've released Carrot2 and Lingo3G and there's a chunk of free time for me to work on randomized testing. Anyway I got what I wanted: a failure in my test ;) So maybe I'll hint you a bit more -- look at the log messages before the test fails with a leaked thread. The runner will try to probe the thread's stack at random intervals to give you an idea where it's been before it was killed. The probing cuts off identical stack portion so you can pretty much tell what a thread was doing. /** * The number of probes of the offending thread's stack trace before an attempt * is made to kill it. Snapshots are taken at random intervals between 10 and 100 * milliseconds each and dumped to system logger to facilitate debugging of the offending thread. * * pSetting this value to 0 means no samples will be taken. */ int stackSamples() default 10; Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3832) Port BasicAutomata.stringUnion from Brics to Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403805#comment-13403805 ] Dawid Weiss commented on LUCENE-3832: - It'd be possible to avoid the sort and still make makeUnionOfStrings(CollectionString) possible if we also exposed Utf16AsUtf8 comparator... Then the automaton builder could accept either byte[] or char[] and just do its job without any additional overhead of copying. Ok, I'll remove that method if you don't like it. If there's a need we can revert it using the trick above. Port BasicAutomata.stringUnion from Brics to Lucene --- Key: LUCENE-3832 URL: https://issues.apache.org/jira/browse/LUCENE-3832 Project: Lucene - Java Issue Type: Improvement Components: core/FSTs Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial Fix For: 4.0, 5.0 Attachments: LUCENE-3832.patch, LUCENE-3832.patch Brics has my code to build Automaton from a set of sorted strings in one step (Daciuk/Mihov's algorithm again). This should be easily portable to Lucene and is quite useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4172) clean up redundant throws clauses
[ https://issues.apache.org/jira/browse/LUCENE-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403806#comment-13403806 ] Michael McCandless commented on LUCENE-4172: I prefer the individual imports, not wildcards. clean up redundant throws clauses - Key: LUCENE-4172 URL: https://issues.apache.org/jira/browse/LUCENE-4172 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Assignee: Steven Rowe Fix For: 4.0, 5.0 Attachments: LUCENE-4172.patch, LUCENE-4172.patch, LUCENE-4172.patch examples are things like ctors that list throws XYZException but actually dont, and things like 'throws CorruptIndex, LockObtainedFailed, IOException' when all of these are actually IOException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1090 - Failure!
This is the root cause: [junit4] Caused by: java.lang.AssertionError: ram was 41605728 expected: 40986328 flush mem: 27371536 activeMem: 14234192 pendingMem: 0 flushingMem: 2 blockedMem: 0 peakDeltaMem: 3715948 [junit4]at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A]:0) [junit4]at org.apache.lucene.index.DocumentsWriterFlushControl.assertMemory(DocumentsWriterFlushControl.java:114) [junit4]at org.apache.lucene.index.DocumentsWriterFlushControl.doAfterDocument(DocumentsWriterFlushControl.java:181) [junit4]at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:348) [junit4]at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1174) [junit4]at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1134) [junit4]at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:157) [junit4]at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145) [junit4]at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:555) [junit4]at org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:57) [junit4]at org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:414) Is there an issue open for this? I couldn't find it on quick search... Mike McCandless http://blog.mikemccandless.com On Fri, Jun 29, 2012 at 3:22 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: I've seen this one on my build server too. Dawid On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server jenk...@sd-datasolutions.de wrote: Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/ 1 tests failed. REGRESSION: org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463) at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at
Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1090 - Failure!
I don't think there's a jira entry for this, but I also saw it on my build server (ram was X expected Y). Don't know what to call it though (what this assertion actually does). Dawid On Fri, Jun 29, 2012 at 11:59 AM, Michael McCandless luc...@mikemccandless.com wrote: This is the root cause: [junit4] Caused by: java.lang.AssertionError: ram was 41605728 expected: 40986328 flush mem: 27371536 activeMem: 14234192 pendingMem: 0 flushingMem: 2 blockedMem: 0 peakDeltaMem: 3715948 [junit4] at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A]:0) [junit4] at org.apache.lucene.index.DocumentsWriterFlushControl.assertMemory(DocumentsWriterFlushControl.java:114) [junit4] at org.apache.lucene.index.DocumentsWriterFlushControl.doAfterDocument(DocumentsWriterFlushControl.java:181) [junit4] at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:348) [junit4] at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1174) [junit4] at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1134) [junit4] at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:157) [junit4] at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145) [junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:555) [junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:57) [junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:414) Is there an issue open for this? I couldn't find it on quick search... Mike McCandless http://blog.mikemccandless.com On Fri, Jun 29, 2012 at 3:22 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: I've seen this one on my build server too. Dawid On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server jenk...@sd-datasolutions.de wrote: Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/ 1 tests failed. REGRESSION: org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463) at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at
[jira] [Resolved] (LUCENE-3832) Port BasicAutomata.stringUnion from Brics to Lucene
[ https://issues.apache.org/jira/browse/LUCENE-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss resolved LUCENE-3832. - Resolution: Fixed Port BasicAutomata.stringUnion from Brics to Lucene --- Key: LUCENE-3832 URL: https://issues.apache.org/jira/browse/LUCENE-3832 Project: Lucene - Java Issue Type: Improvement Components: core/FSTs Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial Fix For: 4.0, 5.0 Attachments: LUCENE-3832.patch, LUCENE-3832.patch Brics has my code to build Automaton from a set of sorted strings in one step (Daciuk/Mihov's algorithm again). This should be easily portable to Lucene and is quite useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Use of @Slow annotation on tests
On Fri, Jun 29, 2012 at 4:42 AM, Dawid Weiss dawid.we...@gmail.com wrote: Hi. I would like to change the use of @Slow. Currently only this test: Test2BTerms#test2BTerms is annotated with this annotation. I don't know how slow it really is, couple of hours :) My motivation is that I would like to be able to run a quick test run from time to time and for this I'd like to skip the slowest, most time-consuming tests. I could do it by running: ant -Dtests.slow=off currently this isn't possible. Looking forward to hearing from you. +1! Of course from time to time we should try to fix these slow tests to not be slow. I know last I looked at this was in january and now tests are taking about twice as long... but its an uphill battle and I agree we should have a way to skip these bad apples. -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-4.x - Build # 23 - Still Failing
Build: https://builds.apache.org/job/Lucene-4.x/23/ All tests passed Build Log: [...truncated 28881 lines...] [...truncated 28765 lines...] rat-sources: [echo] [echo] * [echo] Summary [echo] --- [echo] Generated at: 2012-06-29T10:17:24+00:00 [echo] Notes: 0 [echo] Binaries: 0 [echo] Archives: 0 [echo] Standards: 47 [echo] [echo] Apache Licensed: 46 [echo] Generated Documents: 0 [echo] [echo] JavaDocs are generated and so license header is optional [echo] Generated files do not required license headers [echo] [echo] 1 Unknown Licenses [echo] [echo] *** [echo] [echo] Unapproved licenses: [echo] [echo] /usr/home/hudson/hudson-slave/workspace/Lucene-4.x/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/query/package-info.java [echo] [echo] *** [echo] [echo] Archives: [echo] [echo] * [echo] Files with Apache License headers will be marked AL [echo] Binary files (which do not require AL headers) will be marked B [echo] Compressed archives will be marked A [echo] Notices, licenses etc will be marked N [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-4.x/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SimpleSpatialFieldInfo.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-4.x/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SpatialFieldInfo.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-4.x/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SpatialStrategy.java [echo] AL /usr/home/hudson/hudson-slave/workspace/Lucene-4.x/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/package-info.java [...truncated 78 lines...] [...truncated 28881 lines...] [...truncated 28881 lines...] [...truncated 28881 lines...] [...truncated 28881 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-4181) Change the semantics of @Slow on tests
[ https://issues.apache.org/jira/browse/LUCENE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-4181: Attachment: LUCENE-4181.patch Applied @Slow on the slowest tests. Change the semantics of @Slow on tests -- Key: LUCENE-4181 URL: https://issues.apache.org/jira/browse/LUCENE-4181 Project: Lucene - Java Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial Fix For: 4.0, 5.0 Attachments: LUCENE-4181.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Use of @Slow annotation on tests
is annotated with this annotation. I don't know how slow it really is, couple of hours :) We need @SuperSlow. Or @YouWillTurnToDustBeforeItCompletes. +1! Of course from time to time we should try to fix these slow tests Yep, it is unavoidable that the time will be increasing but we have the tools to collect stats periodically and we definitely should both try to decrease the time for the slowest tests and try to make life easier for developers by marking the slowest ones @Nightly and @Slow. I consider @Nightly stronger than @Slow because they're turned off by default. @Slow would be optional -- those who want a quick run can override it, those who don't care can run the full suite. I filed this issue: https://issues.apache.org/jira/browse/LUCENE-4181 I'm measuring the speed difference for the same seed out of curiosity, will apply shortly. Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Use of @Slow annotation on tests
On Fri, Jun 29, 2012 at 6:31 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: is annotated with this annotation. I don't know how slow it really is, couple of hours :) We need @SuperSlow. Or @YouWillTurnToDustBeforeItCompletes. +1! Of course from time to time we should try to fix these slow tests Yep, it is unavoidable that the time will be increasing but we have the tools to collect stats periodically and we definitely should both try to decrease the time for the slowest tests and try to make life easier for developers by marking the slowest ones @Nightly and @Slow. I consider @Nightly stronger than @Slow because they're turned off by default. @Slow would be optional -- those who want a quick run can override it, those who don't care can run the full suite. I filed this issue: https://issues.apache.org/jira/browse/LUCENE-4181 I'm measuring the speed difference for the same seed out of curiosity, will apply shortly. I like the idea: maybe Test2BTerms should be @Weekly? I know we don't use this (yet) in jenkins but the mechanism is there if someone wants it right? -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4181) Change the semantics of @Slow on tests
[
https://issues.apache.org/jira/browse/LUCENE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403826#comment-13403826
]
Dawid Weiss commented on LUCENE-4181:
-
Before (full tests, solr and lucene):
{noformat}
BUILD SUCCESSFUL
Total time: 11 minutes 29 seconds
{noformat}
After:
{noformat}
BUILD SUCCESSFUL
Total time: 7 minutes 45 seconds
{noformat}
Change the semantics of @Slow on tests
--
Key: LUCENE-4181
URL: https://issues.apache.org/jira/browse/LUCENE-4181
Project: Lucene - Java
Issue Type: Task
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Trivial
Fix For: 4.0, 5.0
Attachments: LUCENE-4181.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-4181) Change the semantics of @Slow on tests
[ https://issues.apache.org/jira/browse/LUCENE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss resolved LUCENE-4181. - Resolution: Fixed I'll commit it in since there was just one test marked as @Slow and it doesn't affect the default behavior. Change the semantics of @Slow on tests -- Key: LUCENE-4181 URL: https://issues.apache.org/jira/browse/LUCENE-4181 Project: Lucene - Java Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial Fix For: 4.0, 5.0 Attachments: LUCENE-4181.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Use of @Slow annotation on tests
Yep, sure. All these test groups can be triggered via system properties so sure -- a weekly build could enable the @Weekly group and run a full test. I don't know how heavy it is on the filesystem though (and if Jenkins can handle this particular test). Dawid On Fri, Jun 29, 2012 at 12:46 PM, Robert Muir rcm...@gmail.com wrote: On Fri, Jun 29, 2012 at 6:31 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: is annotated with this annotation. I don't know how slow it really is, couple of hours :) We need @SuperSlow. Or @YouWillTurnToDustBeforeItCompletes. +1! Of course from time to time we should try to fix these slow tests Yep, it is unavoidable that the time will be increasing but we have the tools to collect stats periodically and we definitely should both try to decrease the time for the slowest tests and try to make life easier for developers by marking the slowest ones @Nightly and @Slow. I consider @Nightly stronger than @Slow because they're turned off by default. @Slow would be optional -- those who want a quick run can override it, those who don't care can run the full suite. I filed this issue: https://issues.apache.org/jira/browse/LUCENE-4181 I'm measuring the speed difference for the same seed out of curiosity, will apply shortly. I like the idea: maybe Test2BTerms should be @Weekly? I know we don't use this (yet) in jenkins but the mechanism is there if someone wants it right? -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
How to start apache solr automatically in linux server for drupal 7
Hello all, I am a newbie to drupal. I am working on drupal commerce for an ecommerce site.Here i need to implement apache solr search for autocomplete of product search . For this function , am running java -jar start.jar comand in apache-solr directory. I need this to be automatic in server.We using linux/SUSE server.How can i start apache-solr automatically in linux/SUSE server. Please provide guidelines to me. THanks Arun
[jira] [Assigned] (LUCENE-4094) Randomize file.encoding
[
https://issues.apache.org/jira/browse/LUCENE-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Weiss reassigned LUCENE-4094:
---
Assignee: Dawid Weiss
Randomize file.encoding
---
Key: LUCENE-4094
URL: https://issues.apache.org/jira/browse/LUCENE-4094
Project: Lucene - Java
Issue Type: Sub-task
Components: general/test
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Trivial
Stated in the code:
{code}
// TODO we can't randomize this yet (it drives ant crazy) but this makes
tests reproduce
// in case machines have different default charsets...
sb.append( -Dargs=\-Dfile.encoding= +
System.getProperty(file.encoding) + \);
{code}
But this should work without any problems with junit4 because communication
streams are separate and we're decoding output properly (or so I hope).
Try and see what happens :)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Use of @Slow annotation on tests
On Fri, Jun 29, 2012 at 6:50 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Yep, sure. All these test groups can be triggered via system properties so sure -- a weekly build could enable the @Weekly group and run a full test. I don't know how heavy it is on the filesystem though (and if Jenkins can handle this particular test). Its very heavy on disk too. But i dont see an alternative for this test. if only it were like Test2BDocs -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: How to start apache solr automatically in linux server for drupal 7
Hi, Install the daemon Linux package. This one allows to correctly fork processes into background. It also creates PID files and set the user account on starting. After that you only need a upstart (e.g. used on ubuntu) or init.d script to start it on boot. Alternatively add it as @reboot cron job for any user it should run under. Uwe -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de Arun Kumar a...@novalnet-solutions.com schrieb: Hello all, I am a newbie to drupal. I am working on drupal commerce for an ecommerce site.Here i need to implement apache solr search for autocomplete of product search . For this function , am running java -jar start.jar comand in apache-solr directory. I need this to be automatic in server.We using linux/SUSE server.How can i start apache-solr automatically in linux/SUSE server. Please provide guidelines to me. THanks Arun
[jira] [Resolved] (LUCENE-4178) FieldType.setTokenized shoudl be true by default
[
https://issues.apache.org/jira/browse/LUCENE-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-4178.
-
Resolution: Fixed
Fix Version/s: 5.0
4.0
FieldType.setTokenized shoudl be true by default
Key: LUCENE-4178
URL: https://issues.apache.org/jira/browse/LUCENE-4178
Project: Lucene - Java
Issue Type: Bug
Affects Versions: 4.0
Reporter: Robert Muir
Fix For: 4.0, 5.0
Attachments: LUCENE-4178.patch
This is really confusing, see LUCENE-4176 where a user hit this.
The example code from the user there reads:
{code}
FieldType fieldType = new FieldType();
fieldType.setIndexed(true);
fieldType.setStored(true);
{code}
Its really trappy that this does not invoke the analyzer: because
historically we did this unless you specified NOT_ANALYZED.
So i think fieldType.setTokenized(true) should be the default: things like
StringField can turn it off.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1725) Script based UpdateRequestProcessorFactory
[
https://issues.apache.org/jira/browse/SOLR-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403838#comment-13403838
]
Robert Muir commented on SOLR-1725:
---
{quote}
fixed InputStreamReader to use UTF-8
{quote}
I couldn't resist: I think instead of
{code}
InputStream input = resourceLoader.openResource(fileName);
return new InputStreamReader(input, UTF-8);
{code}
I would try something like
{code}
InputStream input = resourceLoader.openResource(fileName);
CharsetDecoder decoder = IOUtils.CHARSET_UTF_8.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT);
return new InputStreamReader(input, decoder);
{code}
we could maybe add some sugar like this somewhere: I think
we should be throwing exceptions on invalid encoding when possible.
I can't stand how java silently defaults to replacing errors with \uFFFD
Script based UpdateRequestProcessorFactory
--
Key: SOLR-1725
URL: https://issues.apache.org/jira/browse/SOLR-1725
Project: Solr
Issue Type: New Feature
Components: update
Affects Versions: 1.4
Reporter: Uri Boness
Assignee: Erik Hatcher
Labels: UpdateProcessor
Fix For: 4.1
Attachments: SOLR-1725-rev1.patch, SOLR-1725.patch, SOLR-1725.patch,
SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch,
SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch
A script based UpdateRequestProcessorFactory (Uses JDK6 script engine
support). The main goal of this plugin is to be able to configure/write
update processors without the need to write and package Java code.
The update request processor factory enables writing update processors in
scripts located in {{solr.solr.home}} directory. The functory accepts one
(mandatory) configuration parameter named {{scripts}} which accepts a
comma-separated list of file names. It will look for these files under the
{{conf}} directory in solr home. When multiple scripts are defined, their
execution order is defined by the lexicographical order of the script file
name (so {{scriptA.js}} will be executed before {{scriptB.js}}).
The script language is resolved based on the script file extension (that is,
a *.js files will be treated as a JavaScript script), therefore an extension
is mandatory.
Each script file is expected to have one or more methods with the same
signature as the methods in the {{UpdateRequestProcessor}} interface. It is
*not* required to define all methods, only those hat are required by the
processing logic.
The following variables are define as global variables for each script:
* {{req}} - The SolrQueryRequest
* {{rsp}}- The SolrQueryResponse
* {{logger}} - A logger that can be used for logging purposes in the script
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1725) Script based UpdateRequestProcessorFactory
[
https://issues.apache.org/jira/browse/SOLR-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403841#comment-13403841
]
Robert Muir commented on SOLR-1725:
---
We do have sugar for this already actually:
IOUtils.getDecodingReader(InputStream, Charset)
Script based UpdateRequestProcessorFactory
--
Key: SOLR-1725
URL: https://issues.apache.org/jira/browse/SOLR-1725
Project: Solr
Issue Type: New Feature
Components: update
Affects Versions: 1.4
Reporter: Uri Boness
Assignee: Erik Hatcher
Labels: UpdateProcessor
Fix For: 4.1
Attachments: SOLR-1725-rev1.patch, SOLR-1725.patch, SOLR-1725.patch,
SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch,
SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch
A script based UpdateRequestProcessorFactory (Uses JDK6 script engine
support). The main goal of this plugin is to be able to configure/write
update processors without the need to write and package Java code.
The update request processor factory enables writing update processors in
scripts located in {{solr.solr.home}} directory. The functory accepts one
(mandatory) configuration parameter named {{scripts}} which accepts a
comma-separated list of file names. It will look for these files under the
{{conf}} directory in solr home. When multiple scripts are defined, their
execution order is defined by the lexicographical order of the script file
name (so {{scriptA.js}} will be executed before {{scriptB.js}}).
The script language is resolved based on the script file extension (that is,
a *.js files will be treated as a JavaScript script), therefore an extension
is mandatory.
Each script file is expected to have one or more methods with the same
signature as the methods in the {{UpdateRequestProcessor}} interface. It is
*not* required to define all methods, only those hat are required by the
processing logic.
The following variables are define as global variables for each script:
* {{req}} - The SolrQueryRequest
* {{rsp}}- The SolrQueryResponse
* {{logger}} - A logger that can be used for logging purposes in the script
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4161) Make PackedInts usable by codecs
[
https://issues.apache.org/jira/browse/LUCENE-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403843#comment-13403843
]
Michael McCandless commented on LUCENE-4161:
Wow, this patch is impressive! Lots of amazing changes... very cool
how you factored out a simple common API for bulk write/read of all
the formats.
Should computeN be non-static method on BulkOperation base class?
(Just seems odd to make a static method whose first arg is an instance
of that class anyway...).
Can we find a better name for computeN? I think n is the number of
blocks we can buffer up given the RAM budget? computeNumBlocks?
computeNumBufferedBlocks? computeBufferedBlocksCount? Something
else...?
I suspect, to use these for codecs, we will want to have versions that
work on int[] values instead (everything we encode are ints:
docIDs/deltas, term freqs, offsets, positions).
Code styling: can we use three lines, ie:
{noformat}
-if (valueCount MAX_SIZE) {
- throw new ArrayIndexOutOfBoundsException(MAX_SIZE exceeded);
-}
{noformat}
instead of one line:
{noformat}
+if (valueCount MAX_SIZE) { throw new
ArrayIndexOutOfBoundsException(MAX_SIZE exceeded); }
{noformat}
in general?
Does this change the on-disk format? I think no? (if those
Format.getId()s match?) If it does change we need back compat (4.0.0 alpha
has left the station...).
Make PackedInts usable by codecs
Key: LUCENE-4161
URL: https://issues.apache.org/jira/browse/LUCENE-4161
Project: Lucene - Java
Issue Type: Improvement
Components: core/store
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-4161.patch
Some codecs might be interested in using
PackedInts.{Writer,Reader,ReaderIterator} to read and write fixed-size values
efficiently.
The problem is that the serialization format is self contained, and always
writes the name of the codec, its version, its number of bits per value and
its format. For example, if you want to use packed ints to store your
postings list, this is a lot of overhead (at least ~60 bytes per term, in
case you only use one Writer per term, more otherwise).
Users should be able to externalize the storage of metadata to save space.
For example, to use PackedInts to store a postings list, one should be able
to store the codec name, its version and the number of bits per doc in the
header of the terms+postings list instead of having to write it once (or
more!) per term.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-4182) DocumentsWriterFlushControl.assertMemory tripped by NGramTokenizerTest.testRandomStrings
Michael McCandless created LUCENE-4182:
--
Summary: DocumentsWriterFlushControl.assertMemory tripped by
NGramTokenizerTest.testRandomStrings
Key: LUCENE-4182
URL: https://issues.apache.org/jira/browse/LUCENE-4182
Project: Lucene - Java
Issue Type: Bug
Reporter: Michael McCandless
{noformat}
Build:
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/
1 tests failed.
REGRESSION:
org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings
Error Message:
some thread(s) failed
Stack Trace:
java.lang.RuntimeException: some thread(s) failed
at
__randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0)
at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463)
at
org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at
org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
at
org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
Build Log:
[...truncated 3668 lines...]
[junit4] Suite: org.apache.lucene.analysis.ngram.NGramTokenizerTest
[junit4] ERROR 3162s J1 | NGramTokenizerTest.testRandomStrings
[junit4] Throwable #1: java.lang.RuntimeException: some thread(s) failed
[junit4]at
__randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0)
[junit4]at
Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1090 - Failure!
OK I opened LUCENE-4182. Mike McCandless http://blog.mikemccandless.com On Fri, Jun 29, 2012 at 6:03 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: I don't think there's a jira entry for this, but I also saw it on my build server (ram was X expected Y). Don't know what to call it though (what this assertion actually does). Dawid On Fri, Jun 29, 2012 at 11:59 AM, Michael McCandless luc...@mikemccandless.com wrote: This is the root cause: [junit4] Caused by: java.lang.AssertionError: ram was 41605728 expected: 40986328 flush mem: 27371536 activeMem: 14234192 pendingMem: 0 flushingMem: 2 blockedMem: 0 peakDeltaMem: 3715948 [junit4] at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A]:0) [junit4] at org.apache.lucene.index.DocumentsWriterFlushControl.assertMemory(DocumentsWriterFlushControl.java:114) [junit4] at org.apache.lucene.index.DocumentsWriterFlushControl.doAfterDocument(DocumentsWriterFlushControl.java:181) [junit4] at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:348) [junit4] at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1174) [junit4] at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1134) [junit4] at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:157) [junit4] at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145) [junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:555) [junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:57) [junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:414) Is there an issue open for this? I couldn't find it on quick search... Mike McCandless http://blog.mikemccandless.com On Fri, Jun 29, 2012 at 3:22 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: I've seen this one on my build server too. Dawid On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server jenk...@sd-datasolutions.de wrote: Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/ 1 tests failed. REGRESSION: org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463) at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at
maven artifacts review
Hello. I think I created a staging instance of 4.0.0-ALPHA (lucene and solr). Could someone eyeball it before I push it out there?: https://repository.apache.org/content/repositories/orgapachelucene-286/ Thanks -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-4080) SegmentReader.numDeletedDocs() sometimes gives an incorrect numDeletedDocs
[
https://issues.apache.org/jira/browse/LUCENE-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-4080:
-
Attachment: LUCENE-4080.patch
New patch (simpler than the previous ones). This time, I only removed
{{MergeState.IndexReaderAndLiveDocs}} and modified {{IndexWriter.mergeMiddle}}
so that it fixes the merge reader in case its del count doesn't match the
{{ReadersAndLiveDocs}} del count. There shouldn't be more contention as with
the current trunk version.
SegmentReader.numDeletedDocs() sometimes gives an incorrect numDeletedDocs
--
Key: LUCENE-4080
URL: https://issues.apache.org/jira/browse/LUCENE-4080
Project: Lucene - Java
Issue Type: Bug
Components: core/index
Affects Versions: 4.0, 4.1
Reporter: Adrien Grand
Priority: Trivial
Fix For: 4.1
Attachments: LUCENE-4080.patch, LUCENE-4080.patch, LUCENE-4080.patch
At merge time, SegmentReader sometimes gives an incorrect value for
numDeletedDocs.
From LUCENE-2357:
bq. As far as I know, [SegmenterReader.numDeletedDocs() is] only unreliable
in this context (SegmentReader passed to SegmentMerger for merging); this is
because we allow newly marked deleted docs to happen concurrently up until
the moment we need to pass the SR instance to the merger (search for // Must
sync to ensure BufferedDeletesStream in IndexWriter.java) ... but it would
be nice to fix that, so I think open a new issue (it won't block this one)?
We should be able to make a new SR instance, sharing the same core as the
current one but using the correct delCount...
bq. It would be cleaner (but I think hairier) to create a new SR for merging
that holds the correct delCount, but let's do that under the separate issue.
bq. it would be best if the SegmentReader's numDeletedDocs were always
correct, but, fixing that in IndexWriter is somewhat tricky. Ie, the fix
could be hairy but the end result (SegmentReader.numDeletedDocs can always
be trusted) would be cleaner...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Windows-Java6-64 - Build # 190 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java6-64/190/ 1 tests failed. REGRESSION: org.apache.solr.spelling.suggest.SuggesterWFSTTest.testSuggestions Error Message: Exception during query Stack Trace: java.lang.RuntimeException: Exception during query at __randomizedtesting.SeedInfo.seed([F42478FFA4C95D09:52F8410A6AFD0AF7]:0) at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:461) at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:428) at org.apache.solr.spelling.suggest.SuggesterTest.testSuggestions(SuggesterTest.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) Caused by: java.lang.NullPointerException at org.apache.lucene.search.suggest.fst.WFSTCompletionLookup.lookupPrefix(WFSTCompletionLookup.java:193) at org.apache.lucene.search.suggest.fst.WFSTCompletionLookup.lookup(WFSTCompletionLookup.java:155) at
Re: maven artifacts review
I smoke tested the solrj artifact with a dead simple app, seems to work. -- Sami Siren On Fri, Jun 29, 2012 at 2:55 PM, Robert Muir rcm...@gmail.com wrote: Hello. I think I created a staging instance of 4.0.0-ALPHA (lucene and solr). Could someone eyeball it before I push it out there?: https://repository.apache.org/content/repositories/orgapachelucene-286/ Thanks -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Use of @Slow annotation on tests
+1 Perhaps an ant target quicktest or something instead of the -D? I don't have any strong feelings which would be better so whatever the people doing the work think. But running ant test on my 2009-vintage laptop has been painful lately On Fri, Jun 29, 2012 at 6:54 AM, Robert Muir rcm...@gmail.com wrote: On Fri, Jun 29, 2012 at 6:50 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Yep, sure. All these test groups can be triggered via system properties so sure -- a weekly build could enable the @Weekly group and run a full test. I don't know how heavy it is on the filesystem though (and if Jenkins can handle this particular test). Its very heavy on disk too. But i dont see an alternative for this test. if only it were like Test2BDocs -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: svn commit: r1355346 - in /lucene/dev/trunk/lucene: CHANGES.txt core/src/java/org/apache/lucene/util/packed/Packed64.java
Can we move this CHANGES entry under the beta section?
On Fri, Jun 29, 2012 at 8:52 AM, jpou...@apache.org wrote:
Author: jpountz
Date: Fri Jun 29 12:52:54 2012
New Revision: 1355346
URL: http://svn.apache.org/viewvc?rev=1355346view=rev
Log:
LUCENE-4171: Performance improvements to Packed64 (Toke Eskildsen).
Modified:
lucene/dev/trunk/lucene/CHANGES.txt
lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/util/packed/Packed64.java
Modified: lucene/dev/trunk/lucene/CHANGES.txt
URL:
http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/CHANGES.txt?rev=1355346r1=1355345r2=1355346view=diff
==
--- lucene/dev/trunk/lucene/CHANGES.txt (original)
+++ lucene/dev/trunk/lucene/CHANGES.txt Fri Jun 29 12:52:54 2012
@@ -1036,6 +1036,9 @@ Optimizations
the cloned instances. WeakIdentityMap was extended to support
iterating over its keys. (Uwe Schindler)
+* LUCENE-4171: Performance improvements to Packed64.
+ (Toke Eskildsen via Adrien Grand)
+
Bug fixes
* LUCENE-2803: The FieldCache can miss values if an entry for a reader
Modified:
lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/util/packed/Packed64.java
URL:
http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/util/packed/Packed64.java?rev=1355346r1=1355345r2=1355346view=diff
==
---
lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/util/packed/Packed64.java
(original)
+++
lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/util/packed/Packed64.java
Fri Jun 29 12:52:54 2012
@@ -25,99 +25,40 @@ import java.util.Arrays;
/**
* Space optimized random access capable array of values with a fixed number
of
- * bits. For 32 bits/value and less, performance on 32 bit machines is not
- * optimal. Consider using {@link Packed32} for such a setup.
+ * bits/value. Values are packed contiguously.
* /pp
- * The implementation strives to avoid conditionals and expensive operations,
- * sacrificing code clarity to achieve better performance.
+ * The implementation strives to perform af fast as possible under the
+ * constraint of contiguous bits, by avoiding expensive operations. This
comes
+ * at the cost of code clarity.
+ * /pp
+ * Technical details: This implementation is a refinement of a non-branching
+ * version. The non-branching get and set methods meant that 2 or 4 atomics
in
+ * the underlying array were always accessed, even for the cases where only
+ * 1 or 2 were needed. Even with caching, this had a detrimental effect on
+ * performance.
+ * Related to this issue, the old implementation used lookup tables for
shifts
+ * and masks, which also proved to be a bit slower than calculating the
shifts
+ * and masks on the fly.
+ * See https://issues.apache.org/jira/browse/LUCENE-4062 for details.
+ *
*/
-
class Packed64 extends PackedInts.MutableImpl {
static final int BLOCK_SIZE = 64; // 32 = int, 64 = long
static final int BLOCK_BITS = 6; // The #bits representing BLOCK_SIZE
static final int MOD_MASK = BLOCK_SIZE - 1; // x % BLOCK_SIZE
- private static final int ENTRY_SIZE = BLOCK_SIZE + 1;
- static final int FAC_BITPOS = 3;
-
- /*
- * In order to make an efficient value-getter, conditionals should be
- * avoided. A value can be positioned inside of a block, requiring shifting
- * left or right or it can span two blocks, requiring a left-shift on the
- * first block and a right-shift on the right block.
- * /pp
- * By always shifting the first block both left and right, we get exactly
- * the right bits. By always shifting the second block right and applying
- * a mask, we get the right bits there. After that, we | the two bitsets.
- */
- static final int[][] SHIFTS =
- new int[ENTRY_SIZE][ENTRY_SIZE * FAC_BITPOS];
- static final long[][] MASKS = new long[ENTRY_SIZE][ENTRY_SIZE];
-
- static { // Generate shifts
- for (int elementBits = 1 ; elementBits = BLOCK_SIZE ; elementBits++) {
- for (int bitPos = 0 ; bitPos BLOCK_SIZE ; bitPos++) {
- int[] currentShifts = SHIFTS[elementBits];
- int base = bitPos * FAC_BITPOS;
- currentShifts[base ] = bitPos;
- currentShifts[base + 1] = BLOCK_SIZE - elementBits;
- if (bitPos = BLOCK_SIZE - elementBits) { // Single block
- currentShifts[base + 2] = 0;
- MASKS[elementBits][bitPos] = 0;
- } else { // Two blocks
- int rBits = elementBits - (BLOCK_SIZE - bitPos);
- currentShifts[base + 2] = BLOCK_SIZE - rBits;
- MASKS[elementBits][bitPos] = ~(~0L rBits);
- }
- }
- }
- }
-
- /*
- * The setter requires more masking than the getter.
- */
- private static
Re: maven artifacts review
Thanks for reviewing Sami. I'll wait a bit more before deploying these just to be sure. Realistically I put the non-maven artifacts out very late last night, and there is the website stuff too, so I'm targeting monday for flipping the switch on the website/sending emails out, so there is no rush. On Fri, Jun 29, 2012 at 8:33 AM, Sami Siren ssi...@gmail.com wrote: I smoke tested the solrj artifact with a dead simple app, seems to work. -- Sami Siren On Fri, Jun 29, 2012 at 2:55 PM, Robert Muir rcm...@gmail.com wrote: Hello. I think I created a staging instance of 4.0.0-ALPHA (lucene and solr). Could someone eyeball it before I push it out there?: https://repository.apache.org/content/repositories/orgapachelucene-286/ Thanks -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
question about maven artifacts publishing instructions
Currently we have: http://wiki.apache.org/lucene-java/PublishMavenArtifacts http://wiki.apache.org/solr/PublishMavenArtifacts But it seems to me if you follow the lucene instructions, that it does lucene+solr in one shot. So I think we can nuke one of these pages and just add a note that it does both lucene and solr artifacts at once (not obvious since you run it from the lucene/ directory) ? -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: svn commit: r1355346 - in /lucene/dev/trunk/lucene: CHANGES.txt core/src/java/org/apache/lucene/util/packed/Packed64.java
On Fri, Jun 29, 2012 at 2:55 PM, Robert Muir rcm...@gmail.com wrote: Can we move this CHANGES entry under the beta section? Fixed. Thanks, Robert. -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (LUCENE-4171) Performance improvements to Packed64
[ https://issues.apache.org/jira/browse/LUCENE-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand closed LUCENE-4171. Resolution: Fixed Fix Version/s: 4.0 Assignee: Adrien Grand Lucene Fields: Patch Available (was: New,Patch Available) Committed (r1355346 and r1355352 on trunk, r1355359 on branch_4x). Thanks a lot, Toke! Performance improvements to Packed64 Key: LUCENE-4171 URL: https://issues.apache.org/jira/browse/LUCENE-4171 Project: Lucene - Java Issue Type: Sub-task Components: core/other Affects Versions: 4.0, 5.0 Environment: Tested with 4 different Intel machines Reporter: Toke Eskildsen Assignee: Adrien Grand Priority: Minor Labels: performance Fix For: 4.0 Attachments: LUCENE-4171.patch, LUCENE-4171.patch, LUCENE-4171.patch, LUCENE-4171.patch Original Estimate: 4h Remaining Estimate: 4h Based on the performance measurements of PackedInts.Mutable's in LUCENE-4062, a new version of Packed64 has been created that is consistently faster than the old Packed64 for both get and set. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Solr-trunk - Build # 1899 - Still Failing
Build: https://builds.apache.org/job/Solr-trunk/1899/
1 tests failed.
FAILED: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
Thread threw an uncaught exception, thread: Thread[Lucene Merge Thread #2,6,]
Stack Trace:
java.lang.RuntimeException: Thread threw an uncaught exception, thread:
Thread[Lucene Merge Thread #2,6,]
at
com.carrotsearch.randomizedtesting.RunnerThreadGroup.processUncaught(RunnerThreadGroup.java:96)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:857)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at
org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
at
org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
Caused by: org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.store.AlreadyClosedException: this Directory is closed
at __randomizedtesting.SeedInfo.seed([BD9802CED4A8287]:0)
at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:507)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:480)
Caused by: org.apache.lucene.store.AlreadyClosedException: this Directory is
closed
at org.apache.lucene.store.Directory.ensureOpen(Directory.java:244)
at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:241)
at
org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:320)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3135)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
Build Log:
[...truncated 43682 lines...]
[junit4] 2 28995 T54 oascc.ZkStateReader$3.process Updating live nodes
[junit4] 2 28996 T41 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4] 2 28996 T22 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /140.211.11.196:38088 which had sessionid
0x1383865c5b90002
[junit4] 2 28997 T20 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4] 2 28998 T54 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2 29006 T54 oasc.Overseer.init Overseer
(id=87882202493812739-127.0.0.1:63883_solr-n_01) starting
[junit4] 2 29006 T25 oazs.PrepRequestProcessor.pRequest Got user-level
KeeperException when processing sessionid:0x1383865c5b90003 type:create
cxid:0x60 zxid:0xfffe txntype:unknown reqpath:n/a Error
Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4] 2 29008 T25 oazs.PrepRequestProcessor.pRequest Got user-level
KeeperException when processing sessionid:0x1383865c5b90003 type:create
cxid:0x61 zxid:0xfffe txntype:unknown reqpath:n/a Error
Path:/solr/overseer Error:KeeperErrorCode = NodeExists for
[jira] [Commented] (LUCENE-4161) Make PackedInts usable by codecs
[
https://issues.apache.org/jira/browse/LUCENE-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403910#comment-13403910
]
Adrien Grand commented on LUCENE-4161:
--
bq. Can we find a better name for computeN?
The meaning of {{n}} is actually a bit complicated. For every number of bits
per value, there is a minimum number of blocks (b) / values (v) you need to
write in order to reach the next block boundary:
* 16 bits per value - b=1, v=4
* 24 bits per value - b=3, v=8
* 50 bits per value - b=25, v=32
* 63 bits per value - b=63, v = 64
* ...
A bulk read consists in copying {{n*v}} values that are contained in {{n*b}}
blocks into a long[] (higher values of {{n}} are likely to yield a better
throughput) = this requires {{n * (b + v)}} longs in memory, this is why I
compute {{n}} as {{ramBudget / (8 * (b + v))}} (since a long is 8 bytes). I
called it {{n}} in the method name because I have no idea how to name it...
iterations, maybe?
bq. I suspect, to use these for codecs, we will want to have versions that work
on int[] values instead (everything we encode are ints: docIDs/deltas, term
freqs, offsets, positions).
I hesitated to do this since it would involve some code duplication, but I
guess it can't be avoided if we want this API to be actually used... What
additional methods do you think we need?
* {{PackedReaderIterator.nextInts(int count)}}
* others?
bq. [static computeN], [code style]
You are right, I will fix it!
bq. Does this change the on-disk format?
No, it doesn't. I will add unit tests for that...
Make PackedInts usable by codecs
Key: LUCENE-4161
URL: https://issues.apache.org/jira/browse/LUCENE-4161
Project: Lucene - Java
Issue Type: Improvement
Components: core/store
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-4161.patch
Some codecs might be interested in using
PackedInts.{Writer,Reader,ReaderIterator} to read and write fixed-size values
efficiently.
The problem is that the serialization format is self contained, and always
writes the name of the codec, its version, its number of bits per value and
its format. For example, if you want to use packed ints to store your
postings list, this is a lot of overhead (at least ~60 bytes per term, in
case you only use one Writer per term, more otherwise).
Users should be able to externalize the storage of metadata to save space.
For example, to use PackedInts to store a postings list, one should be able
to store the codec name, its version and the number of bits per doc in the
header of the terms+postings list instead of having to write it once (or
more!) per term.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-4062) More fine-grained control over the packed integer implementation that is chosen
[
https://issues.apache.org/jira/browse/LUCENE-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toke Eskildsen updated LUCENE-4062:
---
Attachment: PackedIntsBenchmark.java
Packed64Strategy.java
Nice trick with the inheritance in Packed64SingleBlock.java. I really helped a
lot on the machines I tested with. I tried to use the Strategy Pattern myself,
using switch on the final bitsPerValue in set get, but it seems that Hotspot
is not keen on optimizing such a conditional away. I have now used the
inheritance method to provide faster implementations in Packed64 for 1, 2, 4,
8, 16, 32 and 64 bpv. Due to the existence of the Direct classes, this is only
relevant for 1, 2 4 bpv though.
Some new measurements at http://ekot.dk/misc/packedints/ with the following
implementations:
- contiguous: The Packed64 that is being reviewed at LYCENE-4171
- padding: The new Packed64SingleBlock.java with sub-classes
- 3 blocks: As before
- direct: As before
- old padding: Packed64SingleBlock.java without sub-classes
- contiguous strategy: Packed64Strategy.java, which uses sub-classes to
optimize for bpv 2^n
I changed PackedIntsBenchmark to report the highest measured performance from
the iterations instead of the mean. The rationale is that the mean is
vulnerable to machine load fluxations and GC throughout the whole measurement
process.
My observations are that the measurements from atria pc254, which are
relatively old machines, are very similar and fits well with the theory:
Padding does give higher or equal performance than Packed64 at all other bpv's
than 2^n.
The story is not so clear for mars, which is a very fast machine: For the
current test case of 10M values, the padding only provides better performance
after 18-20 bpv and for some bpv, Packed64 is a bit faster. I suspect that
point would be moved if other processes were competing for memory cache. Still
pending are measurements from my i7-machine. I hope to add them this evening or
tomorrow.
Also pending is why the performance of Direct set is so low for mars. It makes
no sense: Even if there is some freaky hit for requesting bytes, shorts or ints
on that machine, performance should be = everything else at 64bpv.
When should we stop optimizing? For 1 bpv, a list of booleans would probably be
faster, so should we make a PackedBoolean? Similarly, Packed8, Packed16 and
Packed32 would also make sense if we were really determined. None of the
implementations are hard to make, but it becomes a hassle to update when a new
feature is needed.
More fine-grained control over the packed integer implementation that is
chosen
---
Key: LUCENE-4062
URL: https://issues.apache.org/jira/browse/LUCENE-4062
Project: Lucene - Java
Issue Type: Improvement
Components: core/other
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Labels: performance
Fix For: 4.0, 5.0
Attachments: LUCENE-4062-2.patch, LUCENE-4062.patch,
LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch,
LUCENE-4062.patch, LUCENE-4062.patch, Packed64SingleBlock.java,
Packed64Strategy.java, Packed64calc.java, PackedIntsBenchmark.java,
PackedIntsBenchmark.java, PackedIntsBenchmark.java,
measurements_te_graphs.pdf, measurements_te_i7.txt, measurements_te_p4.txt,
measurements_te_xeon.txt
In order to save space, Lucene has two main PackedInts.Mutable implentations,
one that is very fast and is based on a byte/short/integer/long array
(Direct*) and another one which packs bits in a memory-efficient manner
(Packed*).
The packed implementation tends to be much slower than the direct one, which
discourages some Lucene components to use it. On the other hand, if you store
21 bits integers in a Direct32, this is a space loss of (32-21)/32=35%.
If you accept to trade some space for speed, you could store 3 of these 21
bits integers in a long, resulting in an overhead of 1/3 bit per value. One
advantage of this approach is that you never need to read more than one block
to read or write a value, so this can be significantly faster than Packed32
and Packed64 which always need to read/write two blocks in order to avoid
costly branches.
I ran some tests, and for 1000 21 bits values, this implementation takes
less than 2% more space and has 44% faster writes and 30% faster reads. The
12 bits version (5 values per block) has the same performance improvement and
a 6% memory overhead compared to the packed implementation.
In order to select the best implementation for a given integer size, I wrote
the {{PackedInts.getMutable(valueCount, bitsPerValue,
acceptableOverheadPerValue)}} method. This method
[JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/14/ 1 tests failed. FAILED: org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures Error Message: expected:100 but was:1 Stack Trace: java.lang.AssertionError: expected:100 but was:1 at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Build Log: [...truncated 8705 lines...] Running org.apache.lucene.util.TestMaxFailuresRule Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.013 sec FAILURE! [...truncated 474 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4062) More fine-grained control over the packed integer implementation that is chosen
[
https://issues.apache.org/jira/browse/LUCENE-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403971#comment-13403971
]
Adrien Grand commented on LUCENE-4062:
--
This is very interesting!
Results computed on mars are very strange so I think we should not take them
into account to make decisions...
bq. When should we stop optimizing?
This is a relevant question. I think there is no problem going further provided
that we have evidence that the changes improve performance in most cases and
that the code remains maintainable. For example, it is probably not necessary
to work on specialized Packed64 for the 2^n bits cases since there are direct
impls for the 8, 16, 32 and 64 bits cases and since the new Packed64SingleBlock
impl seems to be as fast for 1, 2 and 4 bits per value.
However, I think it makes sense to replace Packed64SingleBlock with the
inheritance-based impl since it seems faster than the current impl and than
Packed64. I will open an issue if the graphs computed on your core i7 computer
confirm this trend.
More fine-grained control over the packed integer implementation that is
chosen
---
Key: LUCENE-4062
URL: https://issues.apache.org/jira/browse/LUCENE-4062
Project: Lucene - Java
Issue Type: Improvement
Components: core/other
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Labels: performance
Fix For: 4.0, 5.0
Attachments: LUCENE-4062-2.patch, LUCENE-4062.patch,
LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch,
LUCENE-4062.patch, LUCENE-4062.patch, Packed64SingleBlock.java,
Packed64Strategy.java, Packed64calc.java, PackedIntsBenchmark.java,
PackedIntsBenchmark.java, PackedIntsBenchmark.java,
measurements_te_graphs.pdf, measurements_te_i7.txt, measurements_te_p4.txt,
measurements_te_xeon.txt
In order to save space, Lucene has two main PackedInts.Mutable implentations,
one that is very fast and is based on a byte/short/integer/long array
(Direct*) and another one which packs bits in a memory-efficient manner
(Packed*).
The packed implementation tends to be much slower than the direct one, which
discourages some Lucene components to use it. On the other hand, if you store
21 bits integers in a Direct32, this is a space loss of (32-21)/32=35%.
If you accept to trade some space for speed, you could store 3 of these 21
bits integers in a long, resulting in an overhead of 1/3 bit per value. One
advantage of this approach is that you never need to read more than one block
to read or write a value, so this can be significantly faster than Packed32
and Packed64 which always need to read/write two blocks in order to avoid
costly branches.
I ran some tests, and for 1000 21 bits values, this implementation takes
less than 2% more space and has 44% faster writes and 30% faster reads. The
12 bits version (5 values per block) has the same performance improvement and
a 6% memory overhead compared to the packed implementation.
In order to select the best implementation for a given integer size, I wrote
the {{PackedInts.getMutable(valueCount, bitsPerValue,
acceptableOverheadPerValue)}} method. This method select the fastest
implementation that has less than {{acceptableOverheadPerValue}} wasted bits
per value. For example, if you accept an overhead of 20%
({{acceptableOverheadPerValue = 0.2f * bitsPerValue}}), which is pretty
reasonable, here is what implementations would be selected:
* 1: Packed64SingleBlock1
* 2: Packed64SingleBlock2
* 3: Packed64SingleBlock3
* 4: Packed64SingleBlock4
* 5: Packed64SingleBlock5
* 6: Packed64SingleBlock6
* 7: Direct8
* 8: Direct8
* 9: Packed64SingleBlock9
* 10: Packed64SingleBlock10
* 11: Packed64SingleBlock12
* 12: Packed64SingleBlock12
* 13: Packed64
* 14: Direct16
* 15: Direct16
* 16: Direct16
* 17: Packed64
* 18: Packed64SingleBlock21
* 19: Packed64SingleBlock21
* 20: Packed64SingleBlock21
* 21: Packed64SingleBlock21
* 22: Packed64
* 23: Packed64
* 24: Packed64
* 25: Packed64
* 26: Packed64
* 27: Direct32
* 28: Direct32
* 29: Direct32
* 30: Direct32
* 31: Direct32
* 32: Direct32
* 33: Packed64
* 34: Packed64
* 35: Packed64
* 36: Packed64
* 37: Packed64
* 38: Packed64
* 39: Packed64
* 40: Packed64
* 41: Packed64
* 42: Packed64
* 43: Packed64
* 44: Packed64
* 45: Packed64
* 46: Packed64
* 47: Packed64
* 48: Packed64
* 49: Packed64
* 50: Packed64
* 51: Packed64
* 52: Packed64
* 53: Packed64
* 54: Direct64
* 55: Direct64
* 56: Direct64
* 57: Direct64
* 58: Direct64
* 59: Direct64
* 60: Direct64
* 61: Direct64
* 62:
RE: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync
Dawid, You checked in changes to this test (TestMaxFailuresRule.java) at revision 1355279. This Jenkins Maven build's checked out svn revision is 1355359. So it looks like there might be a problem with your commit? Steve -Original Message- From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] Sent: Friday, June 29, 2012 11:26 AM To: dev@lucene.apache.org Subject: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/14/ 1 tests failed. FAILED: org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures Error Message: expected:100 but was:1 Stack Trace: java.lang.AssertionError: expected:100 but was:1 at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Build Log: [...truncated 8705 lines...] Running org.apache.lucene.util.TestMaxFailuresRule Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.013 sec FAILURE! [...truncated 474 lines...]
RE: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync
Hmm, this is likely a Maven Surefire-only problem. I'm clueless about the interaction between Surefire your randomized runner, Dawid - should this test be ignored under Surefire? Under other test runners too, maybe? (IntelliJ, Eclipse) Steve -Original Message- From: Steven A Rowe [mailto:sar...@syr.edu] Sent: Friday, June 29, 2012 11:40 AM To: dev@lucene.apache.org Subject: RE: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync Dawid, You checked in changes to this test (TestMaxFailuresRule.java) at revision 1355279. This Jenkins Maven build's checked out svn revision is 1355359. So it looks like there might be a problem with your commit? Steve -Original Message- From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] Sent: Friday, June 29, 2012 11:26 AM To: dev@lucene.apache.org Subject: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/14/ 1 tests failed. FAILED: org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures Error Message: expected:100 but was:1 Stack Trace: java.lang.AssertionError: expected:100 but was:1 at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Build Log: [...truncated 8705 lines...] Running org.apache.lucene.util.TestMaxFailuresRule Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.013 sec FAILURE! [...truncated 474 lines...]
[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #533: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/533/ 1 tests failed. FAILED: org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures Error Message: expected:100 but was:1 Stack Trace: java.lang.AssertionError: expected:100 but was:1 at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Build Log: [...truncated 8385 lines...] Running org.apache.lucene.util.TestMaxFailuresRule Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.013 sec FAILURE! [...truncated 690 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: question about maven artifacts publishing instructions
Hmm, interesting. When I wrote those instructions in the process of performing them for the last release, the Lucene invocation only staged the Lucene artifacts, likewise for the Solr invocation and artifacts. Where did the artifacts come from? Did you use the download script referred to in the instructions? When you invoked 'ant stage-maven-artifacts', what value did you give the maven.dist.dir sysprop? Was it a parent directory of both the Lucene and the Solr artifacts? Steve -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Friday, June 29, 2012 9:19 AM To: dev@lucene.apache.org Subject: question about maven artifacts publishing instructions Currently we have: http://wiki.apache.org/lucene-java/PublishMavenArtifacts http://wiki.apache.org/solr/PublishMavenArtifacts But it seems to me if you follow the lucene instructions, that it does lucene+solr in one shot. So I think we can nuke one of these pages and just add a note that it does both lucene and solr artifacts at once (not obvious since you run it from the lucene/ directory) ? -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: question about maven artifacts publishing instructions
On Fri, Jun 29, 2012 at 12:57 PM, Steven A Rowe sar...@syr.edu wrote: Was it a parent directory of both the Lucene and the Solr artifacts? yes. is this a problem? -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: question about maven artifacts publishing instructions
I think it's fine, but later today I'll take a look at the stage-maven-artifacts ant target implementation to make sure. - Steve -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Friday, June 29, 2012 1:01 PM To: dev@lucene.apache.org Subject: Re: question about maven artifacts publishing instructions On Fri, Jun 29, 2012 at 12:57 PM, Steven A Rowe sar...@syr.edu wrote: Was it a parent directory of both the Lucene and the Solr artifacts? yes. is this a problem? -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 438 - Failure!
I added the missing ASL header. On Thu, Jun 28, 2012 at 4:54 PM, Policeman Jenkins Server jenk...@sd-datasolutions.de wrote: Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/438/ All tests passed Build Log: [...truncated 15182 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/build.xml:62: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/lucene/build.xml:270: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/lucene/common-build.xml:1435: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/lucene/common-build.xml:1275: Rat problems were found! Total time: 6 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure
Query, Searcher, Weight, Similarity = ?
Hi, I'm new and that is my disclaimer to the stupid question I am about to ask. Am trying to form a conceptual picture of the relation between Query -- Weight -- IndexReader, Scorer, Searcher -- Similarity *From what I gather : (and someone please validate or correct me) * 1. We want *Queries* to be RE-USABLE instances hence *Weight* is a specific Queries state !? 2. *Searcher* is STATEFUL, and though it processes a *Query*, the state for that *Searcher* is delegated to the WEIGHT !? 3. *IndexReader* Reads an Index, and the *Searcher* uses the Reader to SEARCH, using a QUERY 4. From the JavaDocs of Weight class IndexReader dependent state should reside in the Scorer. -- Means, when *weights* are calculated, the final result of the Calculation goes into a STATEFUL object represented by the *Scorer* which is also Iterable !? 5. *Searcher* can be assigned a *Similarity* algorithm. ... hence using that algorithm, it calculates *Weight*, which eventually leads to the construction of an Iterable *Scorer* !? 6. While Indexing, its simple there is a direct relation between IndexWriterConfig -- Similarity +Q) Apart from the validation of my understanding, is there a Sequence Diagram explaining the process of calculation, during a Query? +Q) There are different implementations of Queries. Do they differ in how they mash up all the other stuff? Looks like if i mess each of the other entities, I can pretty much produce whatever Query?! thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Query-Searcher-Weight-Similarity-tp3992080.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Query, Searcher, Weight, Similarity = ?
On Fri, Jun 29, 2012 at 9:02 AM, Arjun Dhar dhar...@yahoo.com wrote: Hi, I'm new and that is my disclaimer to the stupid question I am about to ask. Am trying to form a conceptual picture of the relation between Query -- Weight -- IndexReader, Scorer, Searcher -- Similarity *From what I gather : (and someone please validate or correct me) * 1. We want *Queries* to be RE-USABLE instances hence *Weight* is a specific Queries state !? Queries are independent of a Searcher. When executing a Query, it creates a Weight specifically for that searcher. This contains things things like IDF computations: collection-wide state. 2. *Searcher* is STATEFUL, and though it processes a *Query*, the state for that *Searcher* is delegated to the WEIGHT !? Searcher wraps an indexreader (usually a composite indexreader containing multiple segments like a DirectoryReader) to provide search capabilities. It also has extension points that are search specific: one of these is Similarity, but there are others. For example, in 4.0 you can override methods to provide collection-wide stats where the collection is distributed: consisting of indexes across multiple machines 3. *IndexReader* Reads an Index, and the *Searcher* uses the Reader to SEARCH, using a QUERY yes. 4. From the JavaDocs of Weight class IndexReader dependent state should reside in the Scorer. -- Means, when *weights* are calculated, the final result of the Calculation goes into a STATEFUL object represented by the *Scorer* which is also Iterable !? This could maybe be clarified to say per-segment state. So if you have an IndexSearcher wrapping a DirectoryReader with 4 index segments, in the typical case the Weight holds the state of the entire collection: e.g. IDF across all 4 segments. The Weight creates 4 Scorers: a Scorer for each segment in that DirectoryReader. Any per-segment information such as the document length normalization (norms) array resides in each of those Scorers. 5. *Searcher* can be assigned a *Similarity* algorithm. ... hence using that algorithm, it calculates *Weight*, which eventually leads to the construction of an Iterable *Scorer* !? A Similarity is a hook for term weighting. But term weighting is not the entire scoring algorithm in many cases: Scorers don't have to use Similarity to compute things: they can use whatever logic they want. 6. While Indexing, its simple there is a direct relation between IndexWriterConfig -- Similarity this is for computing document length normalization information (norms) at indexing time. Currently thats the only way that IndexWriter interacts with Similarity. +Q) Apart from the validation of my understanding, is there a Sequence Diagram explaining the process of calculation, during a Query? have a look at https://builds.apache.org/job/Lucene-trunk/javadoc/ , click Searching and Scoring in Lucene. I don't think there are any diagrams there, but there is more information available. +Q) There are different implementations of Queries. Do they differ in how they mash up all the other stuff? Looks like if i mess each of the other entities, I can pretty much produce whatever Query?! See the link above for more information, especially the section on writing custom queries. -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: version checkout
: So I guess there is no way of me doing a patch for 3.6.0 since the issue in : JIRA https://issues.apache.org/jira/browse/SOLR-3574 I reported is not a : bug (but a new feature). Ok. Than, I'll do a patch for the 4.0 As a general rule, features should always be developed against trunk, and then once all the kinks are worked out and test are written, they can be backported -- usually backporting is as easy as an svn merge command, so 90% of the time there is no reason to worry about attaching specific patches for each backported branch to jira (unless you really want to, or if you are aware of some specific deviation in the branches that will make the backporting hard) The reason to always start with trunk and work backwards is because that way you never risk getting into a situation where a feature is implimented in version X, but then version X+1 comes out and the feature is now missing. Bug fixes should usualy be developed in the same way, the only differnece is sometimes the bug only affects specific versions, so you start with the newest version the bug is known to affect (ie: maybe because of some refactoring or other improvements a bug only exists in 3.6 and 4.0, but not on trunk) -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Use of @Slow annotation on tests
+1 Love it On Jun 29, 2012, at 4:42 AM, Dawid Weiss wrote: Hi. I would like to change the use of @Slow. Currently only this test: Test2BTerms#test2BTerms is annotated with this annotation. I don't know how slow it really is, but I would like to: * change this particular test to @Ignore or give it an assumption inside, * change the annotation group @Slow to be ON by default (so that all tests are run by default). * annotate the top of the list (tests longer than 10 seconds?) here: https://issues.apache.org/jira/browse/LUCENE-4174 with @Slow. My motivation is that I would like to be able to run a quick test run from time to time and for this I'd like to skip the slowest, most time-consuming tests. I could do it by running: ant -Dtests.slow=off currently this isn't possible. Looking forward to hearing from you. Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: IRC chat tonight on lucene spatial?
That was a bust :-( Instead, I propose 3:30am UTC (same time) Sunday night. I'll use UTC times because it is more universal for us as we span the globe. ~ David On Jun 28, 2012, at 4:24 PM, Smiley, David W. wrote: Chris, Ryan, I propose, if possible, that we chat on irc at #lucene-dev at ~11:30pm ish ET (7 hours from now). I'd like to discuss the core elements of the Spatial Strategy API, namely makeQuery, makeFilter, makeValueSource, and SpatialOperation. I plan to stay logged in tonight to #lucene-dev till ~2am when I hit the sack. ~ David - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Query, Searcher, Weight, Similarity = ?
This has been elucidating, thanks! On a related topic: I need the ability to pull results lazily, so that I can decide whether to terminate the search iteration early, and ultimately I need to delegate that decision to callers of *my* API. My first question is: did I overlook support for this in Lucene's user-facing API? What I see in Lucene (Collector-based) seems to be all push-style; the Searcher retrieves all, or the top N results, stores them and returns them all at once. Basically what I wanted was a method nextDoc() that I could call repeatedly to retrieve all of the docIDs returned by the search, or at any rate, as many as needed. Not finding this, I wrote a class that would do. What I ended up doing was subclassing DocIdSetIterator and copying some of the logic I saw in IndexSearcher (IIRC) (create a Weight, iterate over subReaders creating Scorers, retrieve docID from scorers, correcting for docBase offset). So my second question is, assuming this isn't already provided somewhere, does it belong in IndexSearcher? Is it worth posting a patch? I'm a little concerned that because I ended up having to access some internal members marked as experimental (subReaders, I think?), this might end up not supported or having to track changes in Lucene's internal API. Last question: what order are documents returned in if you create Scorers with ordered=true - is that always ascending docID order? -Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: IRC chat tonight on lucene spatial?
I meant 3:30am UTC Monday, not Sunday. The UTC/ET thing confused me. On Jun 29, 2012, at 2:03 PM, Smiley, David W. wrote: That was a bust :-( Instead, I propose 3:30am UTC (same time) Sunday night. I'll use UTC times because it is more universal for us as we span the globe. ~ David On Jun 28, 2012, at 4:24 PM, Smiley, David W. wrote: Chris, Ryan, I propose, if possible, that we chat on irc at #lucene-dev at ~11:30pm ish ET (7 hours from now). I'd like to discuss the core elements of the Spatial Strategy API, namely makeQuery, makeFilter, makeValueSource, and SpatialOperation. I plan to stay logged in tonight to #lucene-dev till ~2am when I hit the sack. ~ David - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Query, Searcher, Weight, Similarity = ?
On Fri, Jun 29, 2012 at 2:12 PM, Mike Sokolov soko...@ifactory.com wrote: This has been elucidating, thanks! On a related topic: I need the ability to pull results lazily, so that I can decide whether to terminate the search iteration early, and ultimately I need to delegate that decision to callers of *my* API. The typical solution to this is to just throw an exception from your collector when you are satisfied: see TimeLimitingCollector. Basically what I wanted was a method nextDoc() that I could call repeatedly to retrieve all of the docIDs returned by the search, or at any rate, as many as needed. the push and pull are no different here I think, though I dont know your use case. When your collector is satisfied, just throw an exception :) Last question: what order are documents returned in if you create Scorers with ordered=true - is that always ascending docID order? Yes, if you allow for out-of-order scoring, currently only BooleanScorer will do anything with that (i think only when its a top-level scorer), returning out-of-order hits within each window of 2k docs -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
unit test for search components that works only for distributed search
I have written a custom search component, but the code only supports distributed search. Since we dont use non-distributed search and solr's search flow works differently for non-distributed case, we decided to support only on distributed search. I am trying to write unit test for my custom component. I could see that BaseDistributedSearchTestCase's query method compares the result of a single sharded control and multisharded test. I cannot use this method as my component would work only for multisharded search. Ideally I would like to use something like SolrTestCaseJ4's assertQ method, where i can use an xpath expression to validate the results. Does SolrTestCaseJ4 already support distributed search or do I need to customize it ? Or is there any other way to write unit test case that would work for distributed-only cases ? Thanks for your help! Srini -- View this message in context: http://lucene.472066.n3.nabble.com/unit-test-for-search-components-that-works-only-for-distributed-search-tp3992132.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-3584) NPE when trying to spellcheck.build
Andy Lester created SOLR-3584: - Summary: NPE when trying to spellcheck.build Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3584) NPE when trying to spellcheck.build
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404117#comment-13404117 ] Andy Lester commented on SOLR-3584: --- Full NPE: HTTP Status 500 - null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:679) NPE when trying to spellcheck.build --- Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Query, Searcher, Weight, Similarity = ?
On 06/29/2012 02:17 PM, Robert Muir wrote: On Fri, Jun 29, 2012 at 2:12 PM, Mike Sokolovsoko...@ifactory.com wrote: This has been elucidating, thanks! On a related topic: I need the ability to pull results lazily, so that I can decide whether to terminate the search iteration early, and ultimately I need to delegate that decision to callers of *my* API. The typical solution to this is to just throw an exception from your collector when you are satisfied: see TimeLimitingCollector. I can't do that because am writing a middle layer that is called by a consumer that I can't change and that doesn't know anything about Lucene. I need to implement an Iterator-style interface. My consumer will simply call next() repeatedly, and then stop at some point, and close() me so I can clean up. So to satisfy that I think I would have to collect all results. -Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3584) NPE when trying to spellcheck.build
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404119#comment-13404119 ] Andy Lester commented on SOLR-3584: --- Relevant config from my solrconfig.xml !-- Suggester from page 223 of Apache Solr 3 Enterprise Search Server -- searchComponent name=suggest class=solr.SpellCheckComponent lst name=spellChecker str name=classnameorg.apache.solr.spelling.suggest.Suggester/str str name=lookupImplorg.apache.solr.spelling.suggest.fst.FSTLookup/str str name=fieldtitlea/str float name=threshold0.005/float str name=buildOnCommittrue/str int name=weightBuckets100/int /lst /searchComponent requestHandler name=/suggest class=solr.SearchHandler lst name=defaults str name=spellchecktrue/str str name=spellcheck.dictionarya_suggest/str str name=spellcheck.onlyMorePopulartrue/str str name=spellcheck.count5/str str name=spellcheck.collatetrue/str /lst arr name=components strsuggest/str /arr /requestHandler NPE when trying to spellcheck.build --- Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-3584) NPE when trying to spellcheck.build
[
https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404119#comment-13404119
]
Andy Lester edited comment on SOLR-3584 at 6/29/12 6:57 PM:
Relevant config from my solrconfig.xml
{{
!--
Suggester from page 223 of Apache Solr 3 Enterprise Search Server
--
searchComponent name=suggest class=solr.SpellCheckComponent
lst name=spellChecker
str
name=classnameorg.apache.solr.spelling.suggest.Suggester/str
str
name=lookupImplorg.apache.solr.spelling.suggest.fst.FSTLookup/str
str name=fieldtitlea/str
float name=threshold0.005/float
str name=buildOnCommittrue/str
int name=weightBuckets100/int
/lst
/searchComponent
requestHandler name=/suggest class=solr.SearchHandler
lst name=defaults
str name=spellchecktrue/str
str name=spellcheck.dictionarya_suggest/str
str name=spellcheck.onlyMorePopulartrue/str
str name=spellcheck.count5/str
str name=spellcheck.collatetrue/str
/lst
arr name=components
strsuggest/str
/arr
/requestHandler
}}
was (Author: petdance):
Relevant config from my solrconfig.xml
!--
Suggester from page 223 of Apache Solr 3 Enterprise Search Server
--
searchComponent name=suggest class=solr.SpellCheckComponent
lst name=spellChecker
str
name=classnameorg.apache.solr.spelling.suggest.Suggester/str
str
name=lookupImplorg.apache.solr.spelling.suggest.fst.FSTLookup/str
str name=fieldtitlea/str
float name=threshold0.005/float
str name=buildOnCommittrue/str
int name=weightBuckets100/int
/lst
/searchComponent
requestHandler name=/suggest class=solr.SearchHandler
lst name=defaults
str name=spellchecktrue/str
str name=spellcheck.dictionarya_suggest/str
str name=spellcheck.onlyMorePopulartrue/str
str name=spellcheck.count5/str
str name=spellcheck.collatetrue/str
/lst
arr name=components
strsuggest/str
/arr
/requestHandler
NPE when trying to spellcheck.build
---
Key: SOLR-3584
URL: https://issues.apache.org/jira/browse/SOLR-3584
Project: Solr
Issue Type: Bug
Components: spellchecker
Affects Versions: 3.6
Environment: uname -a
Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011
x86_64 x86_64 x86_64 GNU/Linux
Reporter: Andy Lester
I'm trying to force indexing of my spellchecker component on my titles core.
/solr/titles/suggest?q=atspellcheck=true gives me back no results.
/solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an
NPE starting as
null java.lang.NullPointerException at
org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109)
at
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-3584) NPE when trying to spellcheck.build
[
https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404119#comment-13404119
]
Andy Lester edited comment on SOLR-3584 at 6/29/12 6:58 PM:
Relevant config from my solrconfig.xml
{code:xml}
!--
Suggester from page 223 of Apache Solr 3 Enterprise Search Server
--
searchComponent name=suggest class=solr.SpellCheckComponent
lst name=spellChecker
str
name=classnameorg.apache.solr.spelling.suggest.Suggester/str
str
name=lookupImplorg.apache.solr.spelling.suggest.fst.FSTLookup/str
str name=fieldtitlea/str
float name=threshold0.005/float
str name=buildOnCommittrue/str
int name=weightBuckets100/int
/lst
/searchComponent
requestHandler name=/suggest class=solr.SearchHandler
lst name=defaults
str name=spellchecktrue/str
str name=spellcheck.dictionarya_suggest/str
str name=spellcheck.onlyMorePopulartrue/str
str name=spellcheck.count5/str
str name=spellcheck.collatetrue/str
/lst
arr name=components
strsuggest/str
/arr
/requestHandler
{code}
was (Author: petdance):
Relevant config from my solrconfig.xml
{{
!--
Suggester from page 223 of Apache Solr 3 Enterprise Search Server
--
searchComponent name=suggest class=solr.SpellCheckComponent
lst name=spellChecker
str
name=classnameorg.apache.solr.spelling.suggest.Suggester/str
str
name=lookupImplorg.apache.solr.spelling.suggest.fst.FSTLookup/str
str name=fieldtitlea/str
float name=threshold0.005/float
str name=buildOnCommittrue/str
int name=weightBuckets100/int
/lst
/searchComponent
requestHandler name=/suggest class=solr.SearchHandler
lst name=defaults
str name=spellchecktrue/str
str name=spellcheck.dictionarya_suggest/str
str name=spellcheck.onlyMorePopulartrue/str
str name=spellcheck.count5/str
str name=spellcheck.collatetrue/str
/lst
arr name=components
strsuggest/str
/arr
/requestHandler
}}
NPE when trying to spellcheck.build
---
Key: SOLR-3584
URL: https://issues.apache.org/jira/browse/SOLR-3584
Project: Solr
Issue Type: Bug
Components: spellchecker
Affects Versions: 3.6
Environment: uname -a
Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011
x86_64 x86_64 x86_64 GNU/Linux
Reporter: Andy Lester
I'm trying to force indexing of my spellchecker component on my titles core.
/solr/titles/suggest?q=atspellcheck=true gives me back no results.
/solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an
NPE starting as
null java.lang.NullPointerException at
org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109)
at
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3584) NPE when trying to spellcheck.build
[
https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404125#comment-13404125
]
Andy Lester commented on SOLR-3584:
---
My Tomcat catalina.out log file doesn't have anything different:
INFO: [titles] webapp=/solr path=/suggest
params={spellcheck=trueq=atspellcheck.build=true} status=500 QTime=0
Jun 29, 2012 1:53:37 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
at
org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:679)
NPE when trying to spellcheck.build
---
Key: SOLR-3584
URL: https://issues.apache.org/jira/browse/SOLR-3584
Project: Solr
Issue Type: Bug
Components: spellchecker
Affects Versions: 3.6
Environment: uname -a
Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011
x86_64 x86_64 x86_64 GNU/Linux
Reporter: Andy Lester
I'm trying to force indexing of my spellchecker component on my titles core.
/solr/titles/suggest?q=atspellcheck=true gives me back no results.
/solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an
NPE starting as
null java.lang.NullPointerException at
org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109)
at
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-3584) NPE when trying to spellcheck.build
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer resolved SOLR-3584. -- Resolution: Invalid Andy, Please use the user's mailing list for questions like this. I think you just have a little config problem. In solrconfig.xml, under lst name=spellChecker, add str name=namea_suggest/str . NPE when trying to spellcheck.build --- Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3584) NPE when trying to spellcheck.build
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404142#comment-13404142 ] Andy Lester commented on SOLR-3584: --- I was doing as instructed by Chris Hostetter (hoss) in #solr. He said: please file a bug, because no matter how badly you might have munged your configs, or sent a bad request, Solr shouldn't just throw an NPE, someone should be catching it and giving you back a useful message about the context NPE when trying to spellcheck.build --- Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3584) Solr shouldn't throw NPE when spellcheck dictionary isn'
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer updated SOLR-3584: - Priority: Minor (was: Major) Summary: Solr shouldn't throw NPE when spellcheck dictionary isn' (was: NPE when trying to spellcheck.build) Solr shouldn't throw NPE when spellcheck dictionary isn' Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester Priority: Minor I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Query, Searcher, Weight, Similarity = ?
On 06/29/2012 02:55 PM, Mike Sokolov wrote: On 06/29/2012 02:17 PM, Robert Muir wrote: On Fri, Jun 29, 2012 at 2:12 PM, Mike Sokolovsoko...@ifactory.com wrote: This has been elucidating, thanks! On a related topic: I need the ability to pull results lazily, so that I can decide whether to terminate the search iteration early, and ultimately I need to delegate that decision to callers of *my* API. The typical solution to this is to just throw an exception from your collector when you are satisfied: see TimeLimitingCollector. I can't do that because am writing a middle layer that is called by a consumer that I can't change and that doesn't know anything about Lucene. I need to implement an Iterator-style interface. My consumer will simply call next() repeatedly, and then stop at some point, and close() me so I can clean up. So to satisfy that I think I would have to collect all results. Or the other solution I considered was a multi-threaded collector, but all-in-all it seemed simplest to write a pull-style iterator :) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3584) Solr shouldn't throw NPE when spellcheck dictionary isn'
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404144#comment-13404144 ] Andy Lester commented on SOLR-3584: --- It seems that the cause is in the searchComponent section. I had lst name=spellChecker where it should have been lst name=spellchecker. I can reliably get the NPE by switching to the miscapitalized version. Solr shouldn't throw NPE when spellcheck dictionary isn' Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester Priority: Minor I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3584) Solr shouldn't throw NPE on config errors
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer updated SOLR-3584: - Summary: Solr shouldn't throw NPE on config errors (was: Solr shouldn't throw NPE when spellcheck dictionary isn') Solr shouldn't throw NPE on config errors - Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester Priority: Minor I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-3584) Solr shouldn't throw NPE on config errors
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer reopened SOLR-3584: -- Re-open with a clarified title Solr shouldn't throw NPE on config errors - Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester Priority: Minor I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3584) Spellcheck shouldn't throw NPE on config errors
[ https://issues.apache.org/jira/browse/SOLR-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer updated SOLR-3584: - Summary: Spellcheck shouldn't throw NPE on config errors (was: Solr shouldn't throw NPE on config errors) Spellcheck shouldn't throw NPE on config errors --- Key: SOLR-3584 URL: https://issues.apache.org/jira/browse/SOLR-3584 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 3.6 Environment: uname -a Linux hostname 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Sep 26 10:18:37 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux Reporter: Andy Lester Priority: Minor I'm trying to force indexing of my spellchecker component on my titles core. /solr/titles/suggest?q=atspellcheck=true gives me back no results. /solr/titles/suggest?q=atspellcheck=truespellcheck.build=true gives me an NPE starting as null java.lang.NullPointerException at org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:109) at -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync
This is odd, really. I'll take a look but not today. Dawid On Fri, Jun 29, 2012 at 5:44 PM, Steven A Rowe sar...@syr.edu wrote: Hmm, this is likely a Maven Surefire-only problem. I'm clueless about the interaction between Surefire your randomized runner, Dawid - should this test be ignored under Surefire? Under other test runners too, maybe? (IntelliJ, Eclipse) Steve -Original Message- From: Steven A Rowe [mailto:sar...@syr.edu] Sent: Friday, June 29, 2012 11:40 AM To: dev@lucene.apache.org Subject: RE: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync Dawid, You checked in changes to this test (TestMaxFailuresRule.java) at revision 1355279. This Jenkins Maven build's checked out svn revision is 1355359. So it looks like there might be a problem with your commit? Steve -Original Message- From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] Sent: Friday, June 29, 2012 11:26 AM To: dev@lucene.apache.org Subject: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/14/ 1 tests failed. FAILED: org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures Error Message: expected:100 but was:1 Stack Trace: java.lang.AssertionError: expected:100 but was:1 at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Build Log: [...truncated 8705 lines...] Running org.apache.lucene.util.TestMaxFailuresRule Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.013 sec FAILURE! [...truncated 474 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail:
[JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 318 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-64/318/ 1 tests failed. REGRESSION: org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.testCompositePk_DeltaImport_add_delete Error Message: Exception during query Stack Trace: java.lang.RuntimeException: Exception during query at __randomizedtesting.SeedInfo.seed([5335FF5573083C1:D792779ADE378833]:0) at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:461) at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:428) at org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.testCompositePk_DeltaImport_add_delete(TestSqlEntityProcessorDelta2.java:282) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//*[@numFound='1'] xml response was: ?xml version=1.0 encoding=UTF-8? response lst name=responseHeaderint
[jira] [Commented] (LUCENE-4062) More fine-grained control over the packed integer implementation that is chosen
[
https://issues.apache.org/jira/browse/LUCENE-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404234#comment-13404234
]
Toke Eskildsen commented on LUCENE-4062:
+1 for replacing the Packed64SingleBlock with the optimized version. It is
consistently better.
I have updated the charts at http://ekot.dk/misc/packedints/ with i7
measurements. For the three non-i7-based Intel processors, Packed64SingleBlock
seems clear-cut for the Best possible performance without going all-out with
Direct.
I tried running two tests in parallel on te-prime (see
http://ekot.dk/misc/packedints/te-prime_parallel.html) and got very consistent
results:
* For set, contiguous strategy is faster than or equal to padded for bpvs
3-12 and 16-21 (with bpv 1 and 2 being hard to measure). Since padded only
supports bpv 1-10, 12, 16, 21 32 with non-conforming bpvs rounded up, this
effectively means that there is zero gain in using padded over contiguous with
regard to set on that machine.
* For get, the picture is the same, except for process 2, where 17-21 was
slower for contiguous than for padded. Sigh. I should of course have started 3
processes so voting would have been easier. The same pattern, although noisier,
can be seen on mars.
My preliminary conclusion for i7-processors is thus that Packed64Strategy is
the right choice for Best possible performance without going all-out with
Direct. I am getting very curious about the untested AMD architecture now.
A by-product of the te-prime parallel test is that the amount of cache seems to
matter little when it comes to selecting the most fitting implementation. Thank
$diety for small favors.
More fine-grained control over the packed integer implementation that is
chosen
---
Key: LUCENE-4062
URL: https://issues.apache.org/jira/browse/LUCENE-4062
Project: Lucene - Java
Issue Type: Improvement
Components: core/other
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Labels: performance
Fix For: 4.0, 5.0
Attachments: LUCENE-4062-2.patch, LUCENE-4062.patch,
LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch,
LUCENE-4062.patch, LUCENE-4062.patch, Packed64SingleBlock.java,
Packed64Strategy.java, Packed64calc.java, PackedIntsBenchmark.java,
PackedIntsBenchmark.java, PackedIntsBenchmark.java,
measurements_te_graphs.pdf, measurements_te_i7.txt, measurements_te_p4.txt,
measurements_te_xeon.txt
In order to save space, Lucene has two main PackedInts.Mutable implentations,
one that is very fast and is based on a byte/short/integer/long array
(Direct*) and another one which packs bits in a memory-efficient manner
(Packed*).
The packed implementation tends to be much slower than the direct one, which
discourages some Lucene components to use it. On the other hand, if you store
21 bits integers in a Direct32, this is a space loss of (32-21)/32=35%.
If you accept to trade some space for speed, you could store 3 of these 21
bits integers in a long, resulting in an overhead of 1/3 bit per value. One
advantage of this approach is that you never need to read more than one block
to read or write a value, so this can be significantly faster than Packed32
and Packed64 which always need to read/write two blocks in order to avoid
costly branches.
I ran some tests, and for 1000 21 bits values, this implementation takes
less than 2% more space and has 44% faster writes and 30% faster reads. The
12 bits version (5 values per block) has the same performance improvement and
a 6% memory overhead compared to the packed implementation.
In order to select the best implementation for a given integer size, I wrote
the {{PackedInts.getMutable(valueCount, bitsPerValue,
acceptableOverheadPerValue)}} method. This method select the fastest
implementation that has less than {{acceptableOverheadPerValue}} wasted bits
per value. For example, if you accept an overhead of 20%
({{acceptableOverheadPerValue = 0.2f * bitsPerValue}}), which is pretty
reasonable, here is what implementations would be selected:
* 1: Packed64SingleBlock1
* 2: Packed64SingleBlock2
* 3: Packed64SingleBlock3
* 4: Packed64SingleBlock4
* 5: Packed64SingleBlock5
* 6: Packed64SingleBlock6
* 7: Direct8
* 8: Direct8
* 9: Packed64SingleBlock9
* 10: Packed64SingleBlock10
* 11: Packed64SingleBlock12
* 12: Packed64SingleBlock12
* 13: Packed64
* 14: Direct16
* 15: Direct16
* 16: Direct16
* 17: Packed64
* 18: Packed64SingleBlock21
* 19: Packed64SingleBlock21
* 20: Packed64SingleBlock21
* 21: Packed64SingleBlock21
* 22: Packed64
* 23: Packed64
* 24: Packed64
* 25: Packed64
* 26:
RE: question about maven artifacts publishing instructions
Calling 'ant stage-maven-artifacts' with sysprop maven.dist.dir set to a parent directory containing both Lucene and Solr artifacts should be no problem. 'ant stage-maven-artifacts' invokes dev-tools/scripts/write.stage.maven.build.xml.pl, which works by: a) recursing over the directory pointed to by the maven.dist.dir sysprop looking for POM files; b) generating an Ant build file containing one m2-deploy invocation for each discovered POM file along with its co-located artifact(s) and signature(s). 'stage-maven-artifacts then calls the generated Ant build file to stage everything to the staging repository pointed to by sysprops m2.repository.id and m2.repository.url. The script assumes that the directory pointed to by sysprop maven.dist.dir (or directories nested therein, as you've discovered, Robert) was constructed using 'ant generate-maven-artifacts' and that the appropriate files were signed, ordinarily using 'ant sign-artifacts'. Steve -Original Message- From: Steven A Rowe [mailto:sar...@syr.edu] Sent: Friday, June 29, 2012 1:17 PM To: dev@lucene.apache.org Subject: RE: question about maven artifacts publishing instructions I think it's fine, but later today I'll take a look at the stage-maven-artifacts ant target implementation to make sure. - Steve -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Friday, June 29, 2012 1:01 PM To: dev@lucene.apache.org Subject: Re: question about maven artifacts publishing instructions On Fri, Jun 29, 2012 at 12:57 PM, Steven A Rowe sar...@syr.edu wrote: Was it a parent directory of both the Lucene and the Solr artifacts? yes. is this a problem? -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync
I'll @Ignore it until I figure out what's happening. On Fri, Jun 29, 2012 at 11:13 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: This is odd, really. I'll take a look but not today. Dawid On Fri, Jun 29, 2012 at 5:44 PM, Steven A Rowe sar...@syr.edu wrote: Hmm, this is likely a Maven Surefire-only problem. I'm clueless about the interaction between Surefire your randomized runner, Dawid - should this test be ignored under Surefire? Under other test runners too, maybe? (IntelliJ, Eclipse) Steve -Original Message- From: Steven A Rowe [mailto:sar...@syr.edu] Sent: Friday, June 29, 2012 11:40 AM To: dev@lucene.apache.org Subject: RE: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync Dawid, You checked in changes to this test (TestMaxFailuresRule.java) at revision 1355279. This Jenkins Maven build's checked out svn revision is 1355359. So it looks like there might be a problem with your commit? Steve -Original Message- From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] Sent: Friday, June 29, 2012 11:26 AM To: dev@lucene.apache.org Subject: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #14: POMs out of sync Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/14/ 1 tests failed. FAILED: org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures Error Message: expected:100 but was:1 Stack Trace: java.lang.AssertionError: expected:100 but was:1 at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.lucene.util.TestMaxFailuresRule.testMaxFailures(TestMaxFailuresRule.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Build Log: [...truncated 8705 lines...] Running org.apache.lucene.util.TestMaxFailuresRule Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.013 sec FAILURE! [...truncated 474 lines...]
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 14800 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/14800/ 1 tests failed. REGRESSION: org.apache.lucene.analysis.core.TestRandomChains.testRandomChains Error Message: startOffset must be non-negative, and endOffset must be = startOffset, startOffset=3,endOffset=2 Stack Trace: java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be = startOffset, startOffset=3,endOffset=2 at __randomizedtesting.SeedInfo.seed([B2497B1B5B64D0D2:8FA8527A1C76CD12]:0) at org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:51) at org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilter.java:323) at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:78) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:637) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:547) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:443) at org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:860) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) at
Re: Use of @Slow annotation on tests
bq. Perhaps an ant target quicktest or something instead of the -D?
You can set this permanently in your defaults (or set a shell alias with -D? :)
!-- Give user a chance to override without editing this file
(and without typing -D each time it compiles it --
property file=${user.home}/lucene.build.properties/
property file=${user.home}/build.properties/
property file=${basedir}/build.properties/
property file=${common.dir}/build.properties/
I'd rather not have yet another ant target if not really needed...
Dawid
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-4062) More fine-grained control over the packed integer implementation that is chosen
[ https://issues.apache.org/jira/browse/LUCENE-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404234#comment-13404234 ] Toke Eskildsen edited comment on LUCENE-4062 at 6/29/12 10:03 PM: -- +1 for replacing the Packed64SingleBlock with the optimized version. It is consistently better. I have updated the charts at http://ekot.dk/misc/packedints/ with i7 measurements. For the three non-i7-based Intel processors, Packed64SingleBlock seems clear-cut for the Best possible performance without going all-out with Direct. I tried running two tests in parallel on te-prime (see http://ekot.dk/misc/packedints/te-prime_parallel.html) and got very consistent results: * For set, contiguous strategy is faster than or equal to padded for bpvs 3-12 and 16-21 (with bpv 1 and 2 being hard to measure). Since padded only supports bpv 1-10, 12, 16, 21 32 with non-conforming bpvs rounded up, this effectively means that there is zero gain in using padded over contiguous with regard to set on that machine. * For get, the picture is nearly the same, except for bpv 17-21 where contiguous is slower than padded (observed in process 2 as well as the single-thread run). The difference is less than 10% though. The same pattern, although noisier, can be seen on mars. My preliminary conclusion for i7-processors is thus that Packed64Strategy is the right choice for Best possible performance without going all-out with Direct. I am getting very curious about the untested AMD architecture now. A by-product of the te-prime parallel test is that the amount of cache seems to matter little when it comes to selecting the most fitting implementation. Thank $diety for small favors. was (Author: toke): +1 for replacing the Packed64SingleBlock with the optimized version. It is consistently better. I have updated the charts at http://ekot.dk/misc/packedints/ with i7 measurements. For the three non-i7-based Intel processors, Packed64SingleBlock seems clear-cut for the Best possible performance without going all-out with Direct. I tried running two tests in parallel on te-prime (see http://ekot.dk/misc/packedints/te-prime_parallel.html) and got very consistent results: * For set, contiguous strategy is faster than or equal to padded for bpvs 3-12 and 16-21 (with bpv 1 and 2 being hard to measure). Since padded only supports bpv 1-10, 12, 16, 21 32 with non-conforming bpvs rounded up, this effectively means that there is zero gain in using padded over contiguous with regard to set on that machine. * For get, the picture is the same, except for process 2, where 17-21 was slower for contiguous than for padded. Sigh. I should of course have started 3 processes so voting would have been easier. The same pattern, although noisier, can be seen on mars. My preliminary conclusion for i7-processors is thus that Packed64Strategy is the right choice for Best possible performance without going all-out with Direct. I am getting very curious about the untested AMD architecture now. A by-product of the te-prime parallel test is that the amount of cache seems to matter little when it comes to selecting the most fitting implementation. Thank $diety for small favors. More fine-grained control over the packed integer implementation that is chosen --- Key: LUCENE-4062 URL: https://issues.apache.org/jira/browse/LUCENE-4062 Project: Lucene - Java Issue Type: Improvement Components: core/other Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Labels: performance Fix For: 4.0, 5.0 Attachments: LUCENE-4062-2.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, LUCENE-4062.patch, Packed64SingleBlock.java, Packed64Strategy.java, Packed64calc.java, PackedIntsBenchmark.java, PackedIntsBenchmark.java, PackedIntsBenchmark.java, measurements_te_graphs.pdf, measurements_te_i7.txt, measurements_te_p4.txt, measurements_te_xeon.txt In order to save space, Lucene has two main PackedInts.Mutable implentations, one that is very fast and is based on a byte/short/integer/long array (Direct*) and another one which packs bits in a memory-efficient manner (Packed*). The packed implementation tends to be much slower than the direct one, which discourages some Lucene components to use it. On the other hand, if you store 21 bits integers in a Direct32, this is a space loss of (32-21)/32=35%. If you accept to trade some space for speed, you could store 3 of these 21 bits integers in a long, resulting in an overhead of 1/3 bit per value. One advantage of this
