[
https://issues.apache.org/jira/browse/LUCENE-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475671#comment-13475671
]
Uwe Schindler commented on LUCENE-4482:
---------------------------------------
bq. We don't test on VirtualBOX, so I don't know for sure. In general, Zing
works fine when run on top of hypervisors that fully support things like 2MB
page mappings (the same sort of support needed for hugetlb feature to work).
Unfortunately, there are some hypervisors out there (e.g. some versions of Xen
for paravirt guests) that don't support that, and will crash a vanilla linux
kernel trying to use hugetlb. Zing won't work in such cases either, and for the
same reasons...
In that case, the VM is not running inside a guest OS, but in parallel to a
hypervisor using the same linux kernel. The question was if the 2 modules may
conflict to each other. But I could also imagine to use Zing inside a virtual
machine on one of my servers using Lucene (once the bugs are fixed).
bq. A 4 to 6 order of magnitude difference in pause time and in sustainable
allocation rate is so big that a C4 that uses the vanilla memory management api
would be unusable at this point. Think of the difference between a 20usec phase
shift and a 20 second pause...
Have you thought about making this kernel module available to the kernel
developers for other potential use cases (like virtual machines) also needing
to re-allocate lots of RAM and influence paging/unmapping/mapping?
I did not find the GPL source code of your kernel module only the binary
downloads. Where can I get it. It's easy to hook it's build into DKMS (if its a
standard Makefile, see https://help.ubuntu.com/community/DKMS) without any
custom debian package.
> Likely Zing JVM bug causes failures in TestPayloadNearQuery
> -----------------------------------------------------------
>
> Key: LUCENE-4482
> URL: https://issues.apache.org/jira/browse/LUCENE-4482
> Project: Lucene - Core
> Issue Type: Bug
> Environment: Lucene trunk, rev 1397735
> Zing:
> {noformat}
> java version "1.6.0_31"
> Java(TM) SE Runtime Environment (build 1.6.0_31-6)
> Java HotSpot(TM) 64-Bit Tiered VM (build
> 1.6.0_31-ZVM_5.2.3.0-b6-product-azlinuxM-X86_64, mixed mode)
> {noformat}
> Ubuntu 12.04 LTS 3.2.0-23-generic kernel
> Reporter: Michael McCandless
> Attachments: LUCENE-4482.patch
>
>
> I dug into one of the Lucene test failures when running with Zing JVM
> (available free for open source devs...). At least one other test
> sometimes fails but I haven't dug into that yet.
> I managed to get the failure easily reproduced: with the attached
> patch, on rev 1397735 checkout, if you cd to lucene/core and run:
> {noformat}
> ant test -Dtests.jvms=1 -Dtests.seed=C3802435F5FB39D0
> -Dtests.showSuccess=true
> {noformat}
> Then you'll hit several failures in TestPayloadNearQuery, eg:
> {noformat}
> Suite: org.apache.lucene.search.payloads.TestPayloadNearQuery
> 1> FAILED
> 2> NOTE: reproduce with: ant test -Dtestcase=TestPayloadNearQuery
> -Dtests.method=test -Dtests.seed=C3802435F5FB39D0 -Dtests.slow=true
> -Dtests.locale=ga -Dtests.timezone=America/Adak -Dtests.file.encoding=US-ASCII
> ERROR 0.01s | TestPayloadNearQuery.test <<<
> > Throwable #1: java.lang.RuntimeException: overridden idfExplain method
> in TestPayloadNearQuery.BoostingSimilarity was not called
> > at
> __randomizedtesting.SeedInfo.seed([C3802435F5FB39D0:4BD41BEF5B075428]:0)
> > at
> org.apache.lucene.search.similarities.TFIDFSimilarity.computeWeight(TFIDFSimilarity.java:740)
> > at org.apache.lucene.search.spans.SpanWeight.<init>(SpanWeight.java:62)
> > at
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanWeight.<init>(PayloadNearQuery.java:147)
> > at
> org.apache.lucene.search.payloads.PayloadNearQuery.createWeight(PayloadNearQuery.java:75)
> > at
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:648)
> > at
> org.apache.lucene.search.AssertingIndexSearcher.createNormalizedWeight(AssertingIndexSearcher.java:60)
> > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:265)
> > at
> org.apache.lucene.search.payloads.TestPayloadNearQuery.test(TestPayloadNearQuery.java:146)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> > at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> > at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> > at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> > at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> > at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> > at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> > at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> > at java.lang.Thread.run(Thread.java:661)
> {noformat}
> The patch at least isolates the JVM bug even if it's not exactly a
> minimal test :) Somehow the idfExplain method, which
> is overridden in this test's BoostingSimilarity, fails to be called
> (the super.idfExplain is called instead), which leads to the test
> failures.
> The failure does not happen if you run this test in isolation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
