[jira] [Created] (SOLR-6131) Remove deprecated Token class from solr.spelling package
Spyros Kapnissis created SOLR-6131: -- Summary: Remove deprecated Token class from solr.spelling package Key: SOLR-6131 URL: https://issues.apache.org/jira/browse/SOLR-6131 Project: Solr Issue Type: Improvement Components: spellchecker Affects Versions: 4.8.1 Reporter: Spyros Kapnissis Priority: Minor Attachments: SOLR-6131.patch The deprecated Token class is used everywhere in the spelling package. I am attaching a patch that refactors/replaces all occurrences with the AttributeSource class. The tests are passing. Note: the AttributeSource class also replaces Token as a hash key in many places. Having stricter equals/hashCode requirements than Token, I am a bit concerned that it could produce some duplicate suggestions, especially in the case of ConjunctionSolrSpellChecker where merging of the different spell checking suggestions takes place. If this initial approach is fine, I can create some extra checks/unit tests for this. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6119) TestReplicationHandler attempts to remove open folders
[ https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016279#comment-14016279 ] Varun Thacker commented on SOLR-6119: - Dawid, Thanks for committing the fixes. Regarding the SnapPuller issue, I put a breakpoint on on L149 and ran the test in debug mode from my ide. When I take a thread dump after the execution pauses I don't see any SnapPuller threads. Could you tell me how to reproduce it? > TestReplicationHandler attempts to remove open folders > -- > > Key: SOLR-6119 > URL: https://issues.apache.org/jira/browse/SOLR-6119 > Project: Solr > Issue Type: Bug >Reporter: Dawid Weiss >Priority: Minor > Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch, > SOLR-6119.patch, SOLR-6119.patch > > > TestReplicationHandler has a weird logic around the 'snapDir' variable. It > attempts to remove snapshot folders, even though they're not closed yet. My > recent patch uncovered the bug but I don't know how to fix it cleanly -- the > test itself seems to be very fragile (for example I don't understand the > 'namedBackup' variable which is always set to true, yet there are > conditionals around it). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6128) SolrResourceLoader Error messages
[ https://issues.apache.org/jira/browse/SOLR-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016272#comment-14016272 ] Omer Arslan commented on SOLR-6128: --- Hi Ahmet, Thanks for your reply, is it normal to face these Warning Messages? How do I clear them? > SolrResourceLoader Error messages > - > > Key: SOLR-6128 > URL: https://issues.apache.org/jira/browse/SOLR-6128 > Project: Solr > Issue Type: Bug > Components: contrib - Solr Cell (Tika extraction) >Affects Versions: 4.8.1 > Environment: Windows Server 2012 R2 >Reporter: Omer Arslan >Priority: Minor > > Solr loaded a deprecated plugin/analysis class [solr.IntField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.LongField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.FloatField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.DoubleField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.DateField]. Please > consult documentation how to replace it accordingly. > No stored data found for /rest/managed > No registered observers for /rest/managed -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6119) TestReplicationHandler attempts to remove open folders
[ https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016264#comment-14016264 ] ASF subversion and git services commented on SOLR-6119: --- Commit 1599423 from [~dawidweiss] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599423 ] SOLR-6119: TestReplicationHandler attempts to remove open folders (and other fixes). > TestReplicationHandler attempts to remove open folders > -- > > Key: SOLR-6119 > URL: https://issues.apache.org/jira/browse/SOLR-6119 > Project: Solr > Issue Type: Bug >Reporter: Dawid Weiss >Priority: Minor > Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch, > SOLR-6119.patch, SOLR-6119.patch > > > TestReplicationHandler has a weird logic around the 'snapDir' variable. It > attempts to remove snapshot folders, even though they're not closed yet. My > recent patch uncovered the bug but I don't know how to fix it cleanly -- the > test itself seems to be very fragile (for example I don't understand the > 'namedBackup' variable which is always set to true, yet there are > conditionals around it). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6119) TestReplicationHandler attempts to remove open folders
[ https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016261#comment-14016261 ] ASF subversion and git services commented on SOLR-6119: --- Commit 1599422 from [~dawidweiss] in branch 'dev/trunk' [ https://svn.apache.org/r1599422 ] SOLR-6119: TestReplicationHandler attempts to remove open folders (and other fixes). > TestReplicationHandler attempts to remove open folders > -- > > Key: SOLR-6119 > URL: https://issues.apache.org/jira/browse/SOLR-6119 > Project: Solr > Issue Type: Bug >Reporter: Dawid Weiss >Priority: Minor > Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch, > SOLR-6119.patch, SOLR-6119.patch > > > TestReplicationHandler has a weird logic around the 'snapDir' variable. It > attempts to remove snapshot folders, even though they're not closed yet. My > recent patch uncovered the bug but I don't know how to fix it cleanly -- the > test itself seems to be very fragile (for example I don't understand the > 'namedBackup' variable which is always set to true, yet there are > conditionals around it). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6119) TestReplicationHandler attempts to remove open folders
[
https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016263#comment-14016263
]
Dawid Weiss commented on SOLR-6119:
---
I've committed your latest patch, Varun, thanks. I still don't think SnapPuller
should be working after
{code}
masterJetty.stop();
masterClient.shutdown();
{code}
but it may be my misunderstanding of Solr's infrastructure (I don't work with
Solr much).
Thanks for help!
> TestReplicationHandler attempts to remove open folders
> --
>
> Key: SOLR-6119
> URL: https://issues.apache.org/jira/browse/SOLR-6119
> Project: Solr
> Issue Type: Bug
>Reporter: Dawid Weiss
>Priority: Minor
> Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch,
> SOLR-6119.patch, SOLR-6119.patch
>
>
> TestReplicationHandler has a weird logic around the 'snapDir' variable. It
> attempts to remove snapshot folders, even though they're not closed yet. My
> recent patch uncovered the bug but I don't know how to fix it cleanly -- the
> test itself seems to be very fragile (for example I don't understand the
> 'namedBackup' variable which is always set to true, yet there are
> conditionals around it).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - Failure
Hi Rory, I will file a JIRA issue next time we see this error, with more detailed environment, GC, etc info. Dawid On Mon, Jun 2, 2014 at 8:49 PM, Rory O'Donnell wrote: > Hi Uwe, > > Can you log a bug and send me the Java Incident number, I will follow up. > > Rgds,Rory > > On 02/06/2014 11:28, Uwe Schindler wrote: >> >> Hi, >> >> I think we should send some message to Rory/Oracle about this >> ChuckNorrisException - I have not yet done anything. I am a little bit >> afraid, because RAMUsageEstimator closely works with sun.misc.Unsafe >> (although not at this part of the code that fails), but they could maybe >> don't like this dependency of the code. >> >> Uwe >> >> - >> Uwe Schindler >> H.-H.-Meier-Allee 63, D-28213 Bremen >> http://www.thetaphi.de >> eMail: u...@thetaphi.de >> >> >>> -Original Message- >>> From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf >>> Of Dawid Weiss >>> Sent: Monday, June 02, 2014 11:03 AM >>> To: dev@lucene.apache.org >>> Subject: Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - >>> Failure >>> >>> Has anybody contacted Oracle about this? Is this something known? >>> >>> Dawid >>> >>> On Mon, Jun 2, 2014 at 10:58 AM, Apache Jenkins Server >>> wrote: Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2005/ 1 tests failed. REGRESSION: org.apache.lucene.search.join.TestJoinUtil.testMultiValueRandomJoin Error Message: Stack Trace: java.lang.ArrayStoreException: at >>> >>> __randomizedtesting.SeedInfo.seed([42A81A27864842D0:33CDC2C277FFF4D >>> C]:0) at >>> >>> org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsage >>> Estimator.java:674) at >>> >>> org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEs >>> timator.java:420) at >>> >>> org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java: >>> 333) at >>> >>> org.apache.lucene.search.FieldCache$CacheEntry.estimateSize(FieldCache.j >>> ava:739) at >>> >>> org.apache.lucene.util.FieldCacheSanityChecker.check(FieldCacheSanityChec >>> ker.java:104) at >>> >>> >>> org.apache.lucene.util.FieldCacheSanityChecker.checkSanity(FieldCacheSanit >>> yChecker.java:87) at >>> >>> org.apache.lucene.util.LuceneTestCase.assertSaneFieldCaches(LuceneTestC >>> ase.java:781) at >>> >>> >>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCa >>> cheSanity.java:55) at >>> >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA >>> fterRule.java:46) at >>> >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 >>> .evaluate(SystemPropertiesInvariantRule.java:55) at >>> >>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleTh >>> readAndTestName.java:49) at >>> >>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule >>> IgnoreAfterMaxFailures.java:65) at >>> >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure >>> .java:48) at >>> >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat >>> ementAdapter.java:36) at >>> >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. >>> run(ThreadLeakControl.java:360) at >>> >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask >>> (ThreadLeakControl.java:793) at >>> >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadL >>> eakControl.java:453) at >>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran >>> domizedRunner.java:836) at >>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Rando >>> mizedRunner.java:738) at >>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando >>> mizedRunner.java:772) at >>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando >>> mizedRunner.java:783) at >>> >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA >>> fterRule.java:46) at >>> >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl >>> assName.java:42) at >>> >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 >>> .evaluate(SystemPropertiesInvariantRule.java:55) at >>> >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet >>> hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at >>> >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet >>> hodsRule$1
[jira] [Updated] (LUCENE-5730) FSDirectory.open should return mmap for 64-bit OS X
[
https://issues.apache.org/jira/browse/LUCENE-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-5730:
Attachment: LUCENE-5730.patch
> FSDirectory.open should return mmap for 64-bit OS X
> ---
>
> Key: LUCENE-5730
> URL: https://issues.apache.org/jira/browse/LUCENE-5730
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
> Attachments: LUCENE-5730.patch
>
>
> {noformat}
> Report after iter 19:
> Task QPS trunk StdDev QPS patch StdDev
> Pct diff
>OrHighNotHigh 18.70 (10.3%) 19.79 (8.4%)
> 5.8% ( -11% - 27%)
>OrNotHighHigh 25.15 (11.5%) 26.65 (8.0%)
> 6.0% ( -12% - 28%)
> OrHighHigh 20.92 (9.9%) 22.22 (7.2%)
> 6.2% ( -9% - 25%)
> HighTerm 79.38 (10.8%) 84.42 (13.0%)
> 6.4% ( -15% - 33%)
> Prefix3 125.67 (8.6%) 136.21 (10.5%)
> 8.4% ( -9% - 30%)
> LowTerm 445.68 (13.7%) 484.84 (13.8%)
> 8.8% ( -16% - 42%)
> OrNotHighLow 45.83 (12.9%) 50.22 (8.8%)
> 9.6% ( -10% - 35%)
> OrNotHighMed 41.65 (12.0%) 46.05 (12.1%)
> 10.6% ( -12% - 39%)
> HighSloppyPhrase5.82 (8.9%)6.46 (10.4%)
> 10.9% ( -7% - 33%)
>MedPhrase 127.03 (16.3%) 141.19 (12.8%)
> 11.2% ( -15% - 48%)
> IntNRQ4.85 (3.9%)5.39 (7.4%)
> 11.2% ( 0% - 23%)
> MedTerm 101.62 (13.0%) 113.27 (12.2%)
> 11.5% ( -12% - 42%)
> OrHighNotLow 69.01 (12.9%) 77.77 (15.3%)
> 12.7% ( -13% - 47%)
> AndHighHigh 43.57 (8.1%) 49.24 (8.4%)
> 13.0% ( -3% - 32%)
> HighPhrase9.30 (6.4%) 10.55 (8.4%)
> 13.4% ( -1% - 30%)
>OrHighLow 41.49 (10.1%) 47.45 (13.9%)
> 14.3% ( -8% - 42%)
> HighSpanNear 62.59 (7.5%) 71.79 (10.1%)
> 14.7% ( -2% - 34%)
>OrHighMed 41.74 (12.9%) 48.66 (12.2%)
> 16.6% ( -7% - 47%)
> AndHighMed 59.09 (8.2%) 69.27 (8.1%)
> 17.2% ( 0% - 36%)
> MedSloppyPhrase7.14 (6.2%)8.42 (7.2%)
> 17.9% ( 4% - 33%)
>LowPhrase 16.34 (6.3%) 19.34 (5.9%)
> 18.3% ( 5% - 32%)
> OrHighNotMed 50.43 (13.0%) 59.81 (13.5%)
> 18.6% ( -6% - 51%)
> MedSpanNear 56.50 (12.4%) 69.91 (14.5%)
> 23.7% ( -2% - 57%)
> LowSloppyPhrase 43.81 (10.0%) 57.88 (14.5%)
> 32.1% ( 6% - 62%)
> Wildcard 20.94 (7.4%) 28.53 (9.6%)
> 36.3% ( 18% - 57%)
> LowSpanNear 10.16 (5.4%) 14.05 (10.2%)
> 38.4% ( 21% - 57%)
> Fuzzy1 33.04 (8.3%) 58.74 (13.4%)
> 77.8% ( 51% - 108%)
> Fuzzy2 21.37 (4.4%) 38.41 (6.7%)
> 79.7% ( 65% - 95%)
> Respell 26.68 (7.1%) 50.00 (12.5%)
> 87.4% ( 63% - 115%)
> PKLookup 52.16 (12.4%) 101.00 (24.2%)
> 93.7% ( 50% - 148%)
> AndHighLow 269.54 (9.2%) 537.98 (16.1%)
> 99.6% ( 67% - 137%)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5730) FSDirectory.open should return mmap for 64-bit OS X
Robert Muir created LUCENE-5730:
---
Summary: FSDirectory.open should return mmap for 64-bit OS X
Key: LUCENE-5730
URL: https://issues.apache.org/jira/browse/LUCENE-5730
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Attachments: LUCENE-5730.patch
{noformat}
Report after iter 19:
Task QPS trunk StdDev QPS patch StdDev
Pct diff
OrHighNotHigh 18.70 (10.3%) 19.79 (8.4%)
5.8% ( -11% - 27%)
OrNotHighHigh 25.15 (11.5%) 26.65 (8.0%)
6.0% ( -12% - 28%)
OrHighHigh 20.92 (9.9%) 22.22 (7.2%)
6.2% ( -9% - 25%)
HighTerm 79.38 (10.8%) 84.42 (13.0%)
6.4% ( -15% - 33%)
Prefix3 125.67 (8.6%) 136.21 (10.5%)
8.4% ( -9% - 30%)
LowTerm 445.68 (13.7%) 484.84 (13.8%)
8.8% ( -16% - 42%)
OrNotHighLow 45.83 (12.9%) 50.22 (8.8%)
9.6% ( -10% - 35%)
OrNotHighMed 41.65 (12.0%) 46.05 (12.1%)
10.6% ( -12% - 39%)
HighSloppyPhrase5.82 (8.9%)6.46 (10.4%)
10.9% ( -7% - 33%)
MedPhrase 127.03 (16.3%) 141.19 (12.8%)
11.2% ( -15% - 48%)
IntNRQ4.85 (3.9%)5.39 (7.4%)
11.2% ( 0% - 23%)
MedTerm 101.62 (13.0%) 113.27 (12.2%)
11.5% ( -12% - 42%)
OrHighNotLow 69.01 (12.9%) 77.77 (15.3%)
12.7% ( -13% - 47%)
AndHighHigh 43.57 (8.1%) 49.24 (8.4%)
13.0% ( -3% - 32%)
HighPhrase9.30 (6.4%) 10.55 (8.4%)
13.4% ( -1% - 30%)
OrHighLow 41.49 (10.1%) 47.45 (13.9%)
14.3% ( -8% - 42%)
HighSpanNear 62.59 (7.5%) 71.79 (10.1%)
14.7% ( -2% - 34%)
OrHighMed 41.74 (12.9%) 48.66 (12.2%)
16.6% ( -7% - 47%)
AndHighMed 59.09 (8.2%) 69.27 (8.1%)
17.2% ( 0% - 36%)
MedSloppyPhrase7.14 (6.2%)8.42 (7.2%)
17.9% ( 4% - 33%)
LowPhrase 16.34 (6.3%) 19.34 (5.9%)
18.3% ( 5% - 32%)
OrHighNotMed 50.43 (13.0%) 59.81 (13.5%)
18.6% ( -6% - 51%)
MedSpanNear 56.50 (12.4%) 69.91 (14.5%)
23.7% ( -2% - 57%)
LowSloppyPhrase 43.81 (10.0%) 57.88 (14.5%)
32.1% ( 6% - 62%)
Wildcard 20.94 (7.4%) 28.53 (9.6%)
36.3% ( 18% - 57%)
LowSpanNear 10.16 (5.4%) 14.05 (10.2%)
38.4% ( 21% - 57%)
Fuzzy1 33.04 (8.3%) 58.74 (13.4%)
77.8% ( 51% - 108%)
Fuzzy2 21.37 (4.4%) 38.41 (6.7%)
79.7% ( 65% - 95%)
Respell 26.68 (7.1%) 50.00 (12.5%)
87.4% ( 63% - 115%)
PKLookup 52.16 (12.4%) 101.00 (24.2%)
93.7% ( 50% - 148%)
AndHighLow 269.54 (9.2%) 537.98 (16.1%)
99.6% ( 67% - 137%)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6130) solr-cell dependencies weren't fully upgraded with the Tika 1.4->1.5 upgrade
Steve Rowe created SOLR-6130: Summary: solr-cell dependencies weren't fully upgraded with the Tika 1.4->1.5 upgrade Key: SOLR-6130 URL: https://issues.apache.org/jira/browse/SOLR-6130 Project: Solr Issue Type: Bug Affects Versions: 4.8 Reporter: Steve Rowe Assignee: Steve Rowe Fix For: 4.9, 5.0, 4.8.2 There are problems with the solr-cell dependency configuration: # Despite the fact that the asm:asm dependency was removed in LUCENE-4263, and re-addition effectively vetoed by Uwe/Robert in SOLR-4209, asm:asm:3.1 was re-added with no apparent discussion by SOLR-1301 in Solr 4.7. # The Tika 1.5 upgrade (SOLR-5763) failed to properly upgrade the asm:asm:3.1 dependency to org.ow2.asm:asm-debug-all:4.1 (see TIKA-1053). # New Tika dependency com.uwyn:jhighlight:1.0 was not added. [~thetaphi], do you have any opinions on the asm issues? In particular, would it make sense to have an additional asm dependency (asm-debug-all in addition to asm)? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5729) explore random-access methods to IndexInput
Robert Muir created LUCENE-5729:
---
Summary: explore random-access methods to IndexInput
Key: LUCENE-5729
URL: https://issues.apache.org/jira/browse/LUCENE-5729
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Traditionally lucene access is mostly reading lists of postings and geared at
that, but for random-access stuff like docvalues, it just creates overhead.
So today we are hacking around it, by doing this random access with
seek+readXXX, but this is inefficient (additional checks by the jdk that we
dont need).
As a hack, I added the following to IndexInput, changed direct packed ints
decode to use them, and implemented in MMapDir:
{code}
byte readByte(long pos) --> ByteBuffer.get(pos)
short readShort(long pos) --> ByteBuffer.getShort(pos)
int readInt(long pos) --> ByteBuffer.getInt(pos)
long readLong(long pos) --> ByteBuffer.getLong(pos)
{code}
This gives ~30% performance improvement for docvalues (numerics, sorting
strings, etc)
We should do a few things first before working this (LUCENE-5728: use slice api
in decode, pad packed ints so we only have one i/o call ever, etc etc) but I
think we need to figure out such an API.
It could either be on indexinput like my hack (this is similar to ByteBuffer
API with both relative and absolute methods), or we could have a separate API.
But i guess arguably IOContext exists to supply hints too, so I dont know which
is the way to go.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5886) Store the response in zk for async calls so that it can be returned by REQUESTSTATUS API
[ https://issues.apache.org/jira/browse/SOLR-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta updated SOLR-5886: --- Summary: Store the response in zk for async calls so that it can be returned by REQUESTSTATUS API (was: Propagate more information in case of failed async tasks) > Store the response in zk for async calls so that it can be returned by > REQUESTSTATUS API > > > Key: SOLR-5886 > URL: https://issues.apache.org/jira/browse/SOLR-5886 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Anshum Gupta >Assignee: Anshum Gupta > > As of now, only the state of a pre-submitted tasks is returned in the > response to the REQUESTSTATUS Collections API call. > Pass more information, specially in case of a call erroring out. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-6026) Also check work-queue while processing a REQUESTSTATUS Collection API Call
[ https://issues.apache.org/jira/browse/SOLR-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta reassigned SOLR-6026: -- Assignee: Anshum Gupta > Also check work-queue while processing a REQUESTSTATUS Collection API Call > -- > > Key: SOLR-6026 > URL: https://issues.apache.org/jira/browse/SOLR-6026 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.8 >Reporter: Anshum Gupta >Assignee: Anshum Gupta > Fix For: 4.9 > > Attachments: SOLR-6026.patch, SOLR-6026.patch > > > REQUESTSTATUS API call should check for the following: > * work-queue (submitted task) > * running-map (running task/in progress) > * completed-map > * failure-map > Right now it checks everything but the work-queue. Add that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6026) Also check work-queue while processing a REQUESTSTATUS Collection API Call
[ https://issues.apache.org/jira/browse/SOLR-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta resolved SOLR-6026. Resolution: Fixed > Also check work-queue while processing a REQUESTSTATUS Collection API Call > -- > > Key: SOLR-6026 > URL: https://issues.apache.org/jira/browse/SOLR-6026 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.8 >Reporter: Anshum Gupta >Assignee: Anshum Gupta > Fix For: 4.8.1 > > Attachments: SOLR-6026.patch, SOLR-6026.patch > > > REQUESTSTATUS API call should check for the following: > * work-queue (submitted task) > * running-map (running task/in progress) > * completed-map > * failure-map > Right now it checks everything but the work-queue. Add that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6026) Also check work-queue while processing a REQUESTSTATUS Collection API Call
[ https://issues.apache.org/jira/browse/SOLR-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta updated SOLR-6026: --- Fix Version/s: (was: 4.8.1) 4.9 > Also check work-queue while processing a REQUESTSTATUS Collection API Call > -- > > Key: SOLR-6026 > URL: https://issues.apache.org/jira/browse/SOLR-6026 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.8 >Reporter: Anshum Gupta >Assignee: Anshum Gupta > Fix For: 4.9 > > Attachments: SOLR-6026.patch, SOLR-6026.patch > > > REQUESTSTATUS API call should check for the following: > * work-queue (submitted task) > * running-map (running task/in progress) > * completed-map > * failure-map > Right now it checks everything but the work-queue. Add that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6127) Improve Solr's exampledocs data
[ https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015851#comment-14015851 ] Uwe Schindler commented on SOLR-6127: - bq. Sorry I am python ignorant, may be it is a good idea to use same/compatible version that can run smokeTestRelease.py ? All our tools require Python 3.3. Python 2.7 is no longer used by Lucene (in most cases, I think regenerating MOMAN automaton may need 2.7?). > Improve Solr's exampledocs data > --- > > Key: SOLR-6127 > URL: https://issues.apache.org/jira/browse/SOLR-6127 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Varun Thacker >Priority: Minor > Fix For: 5.0 > > Attachments: film.csv, film.json, film.xml, freebase_film_dump.py, > freebase_film_dump.py > > > Currently > - The CSV example has 10 documents. > - The JSON example has 4 documents. > - The XML example has 32 documents. > 1. We should have equal number of documents and the same documents in all the > example formats > 2. A data set which is slightly more comprehensive. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6127) Improve Solr's exampledocs data
[ https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015846#comment-14015846 ] Uwe Schindler commented on SOLR-6127: - Hi, I think the license of the data is fine (CC-BY, previously known as CC-A), so we can include the files with the distribution. In any case we have to add the attribution in our NOTICE.txt file (Solr part). We should also add a license header to the files (CC header). I am not sure if JSON and CSV supports this, but XML for sure does. > Improve Solr's exampledocs data > --- > > Key: SOLR-6127 > URL: https://issues.apache.org/jira/browse/SOLR-6127 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Varun Thacker >Priority: Minor > Fix For: 5.0 > > Attachments: film.csv, film.json, film.xml, freebase_film_dump.py, > freebase_film_dump.py > > > Currently > - The CSV example has 10 documents. > - The JSON example has 4 documents. > - The XML example has 32 documents. > 1. We should have equal number of documents and the same documents in all the > example formats > 2. A data set which is slightly more comprehensive. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6127) Improve Solr's exampledocs data
[ https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Thacker updated SOLR-6127: Attachment: freebase_film_dump.py film.xml film.json film.csv Updated patch with the Apache License. Also I attached the outputs in all 3 formats. Once you put your developer key on L24 you should be able to run it without any exceptions. If you run into any exceptions post the stack trace and I will fix it. You can get your key from https://code.google.com/apis/console/ I will soon start working on updating the solrconfig and schema files. > Improve Solr's exampledocs data > --- > > Key: SOLR-6127 > URL: https://issues.apache.org/jira/browse/SOLR-6127 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Varun Thacker >Priority: Minor > Fix For: 5.0 > > Attachments: film.csv, film.json, film.xml, freebase_film_dump.py, > freebase_film_dump.py > > > Currently > - The CSV example has 10 documents. > - The JSON example has 4 documents. > - The XML example has 32 documents. > 1. We should have equal number of documents and the same documents in all the > example formats > 2. A data set which is slightly more comprehensive. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6127) Improve Solr's exampledocs data
[
https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015794#comment-14015794
]
Steve Rowe commented on SOLR-6127:
--
{quote}
Uwe Schindler - Would using the data from freebase (
https://developers.google.com/freebase/faq#rules_for_using_data ) be a
licensing issue?
{quote}
Apache releases may contain material licensed under CC-A, which AFAICT is the
same thing as CC-BY, under which Freebase licenses everything except for full
images - see http://www.apache.org/legal/resolved.html#category-a - Category A
includes CC-A 2.5 and 3.0.
{quote}
If thats not a concern here is a script which fetches 200 rows of film data (
http://www.freebase.com/film ) and dumps it into JSON, XML and CSV.
The number of documents can be adjusted. You would need to put in the API KEY
for it to run.
Any opinions if this is a good idea?
{quote}
+1
{quote}
I get exceptions with {noformat}Python 2.7.1 (r271:86832, Jun 16 2011,
16:59:05) {noformat} too. Sorry I am python ignorant, may be it is a good idea
to use same/compatible version that can run {{smokeTestRelease.py}} ?
{quote}
+1
> Improve Solr's exampledocs data
> ---
>
> Key: SOLR-6127
> URL: https://issues.apache.org/jira/browse/SOLR-6127
> Project: Solr
> Issue Type: Improvement
> Components: documentation
>Reporter: Varun Thacker
>Priority: Minor
> Fix For: 5.0
>
> Attachments: freebase_film_dump.py
>
>
> Currently
> - The CSV example has 10 documents.
> - The JSON example has 4 documents.
> - The XML example has 32 documents.
> 1. We should have equal number of documents and the same documents in all the
> example formats
> 2. A data set which is slightly more comprehensive.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5728) use slice() api in packedints decode
[
https://issues.apache.org/jira/browse/LUCENE-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-5728:
Attachment: LUCENE-5728.patch
here is a quick prototype. The nocommit is the thing to figure out.
> use slice() api in packedints decode
>
>
> Key: LUCENE-5728
> URL: https://issues.apache.org/jira/browse/LUCENE-5728
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
> Attachments: LUCENE-5728.patch
>
>
> Today, for example 8-bpv decoder looks like this:
> {code}
> in.seek(startPointer + index);
> return in.readByte() & 0xFF;
> {code}
> If instead we take a slice of 'in', we can remove an addition. Its not much,
> but helps a little. additionally we already (in PackedInts.java) compute the
> number of bytes, so we could make this an actual slice of the range, which
> would return an error on abuse instead of garbage data.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5728) use slice() api in packedints decode
Robert Muir created LUCENE-5728:
---
Summary: use slice() api in packedints decode
Key: LUCENE-5728
URL: https://issues.apache.org/jira/browse/LUCENE-5728
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Today, for example 8-bpv decoder looks like this:
{code}
in.seek(startPointer + index);
return in.readByte() & 0xFF;
{code}
If instead we take a slice of 'in', we can remove an addition. Its not much,
but helps a little. additionally we already (in PackedInts.java) compute the
number of bytes, so we could make this an actual slice of the range, which
would return an error on abuse instead of garbage data.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5678) Investigate to use FileoutputStream instead of RandomAccessFile in FSIndexOutput
[ https://issues.apache.org/jira/browse/LUCENE-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5678: -- Fix Version/s: 4.9 > Investigate to use FileoutputStream instead of RandomAccessFile in > FSIndexOutput > > > Key: LUCENE-5678 > URL: https://issues.apache.org/jira/browse/LUCENE-5678 > Project: Lucene - Core > Issue Type: Task > Components: core/store >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5678.patch, LUCENE-5678.patch, LUCENE-5678.patch, > LUCENE-5678.patch, LUCENE-5678.patch, LUCENE-5678.patch > > > We no longer allow seeking in IndexOutput, so there is no need to use > RandomAccessFile. We can change this with a < 1 KiB patch. > Further improvements would be to merge this with OutputStreamIndexOutput, so > we get many simplifications. > There is also no reason anymore to separate DataOutput from IndexOutput. The > only additional thing is IndexOutput#getFilePointer(), which is handled by > an internal counter (does not use getFilePointer of the underlying RAF) and > checksums. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (LUCENE-5678) Investigate to use FileoutputStream instead of RandomAccessFile in FSIndexOutput
[ https://issues.apache.org/jira/browse/LUCENE-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler reopened LUCENE-5678: --- Reopen for backport after LUCENE-5727. > Investigate to use FileoutputStream instead of RandomAccessFile in > FSIndexOutput > > > Key: LUCENE-5678 > URL: https://issues.apache.org/jira/browse/LUCENE-5678 > Project: Lucene - Core > Issue Type: Task > Components: core/store >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5678.patch, LUCENE-5678.patch, LUCENE-5678.patch, > LUCENE-5678.patch, LUCENE-5678.patch, LUCENE-5678.patch > > > We no longer allow seeking in IndexOutput, so there is no need to use > RandomAccessFile. We can change this with a < 1 KiB patch. > Further improvements would be to merge this with OutputStreamIndexOutput, so > we get many simplifications. > There is also no reason anymore to separate DataOutput from IndexOutput. The > only additional thing is IndexOutput#getFilePointer(), which is handled by > an internal counter (does not use getFilePointer of the underlying RAF) and > checksums. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - Failure
Hi Uwe, Can you log a bug and send me the Java Incident number, I will follow up. Rgds,Rory On 02/06/2014 11:28, Uwe Schindler wrote: Hi, I think we should send some message to Rory/Oracle about this ChuckNorrisException - I have not yet done anything. I am a little bit afraid, because RAMUsageEstimator closely works with sun.misc.Unsafe (although not at this part of the code that fails), but they could maybe don't like this dependency of the code. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf Of Dawid Weiss Sent: Monday, June 02, 2014 11:03 AM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - Failure Has anybody contacted Oracle about this? Is this something known? Dawid On Mon, Jun 2, 2014 at 10:58 AM, Apache Jenkins Server wrote: Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2005/ 1 tests failed. REGRESSION: org.apache.lucene.search.join.TestJoinUtil.testMultiValueRandomJoin Error Message: Stack Trace: java.lang.ArrayStoreException: at __randomizedtesting.SeedInfo.seed([42A81A27864842D0:33CDC2C277FFF4D C]:0) at org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsage Estimator.java:674) at org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEs timator.java:420) at org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java: 333) at org.apache.lucene.search.FieldCache$CacheEntry.estimateSize(FieldCache.j ava:739) at org.apache.lucene.util.FieldCacheSanityChecker.check(FieldCacheSanityChec ker.java:104) at org.apache.lucene.util.FieldCacheSanityChecker.checkSanity(FieldCacheSanit yChecker.java:87) at org.apache.lucene.util.LuceneTestCase.assertSaneFieldCaches(LuceneTestC ase.java:781) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCa cheSanity.java:55) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA fterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 .evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleTh readAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule IgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure .java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat ementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask (ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadL eakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran domizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Rando mizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando mizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando mizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA fterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl assName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 .evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat ementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat ementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat ementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss ertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure .java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule IgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore TestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.Statement
[jira] [Commented] (LUCENE-5727) Remove IndexOutput.seek
[ https://issues.apache.org/jira/browse/LUCENE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015720#comment-14015720 ] Uwe Schindler commented on LUCENE-5727: --- +1 once this is in 4.x, I can backport the new IndexOutput stuff based on OutputStream. > Remove IndexOutput.seek > --- > > Key: LUCENE-5727 > URL: https://issues.apache.org/jira/browse/LUCENE-5727 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.9 > > Attachments: LUCENE-5727.patch > > > This has been deprecated / unused for a few years, and several things assume > append-only functionality: checksumming in the index format, HDFS integration > in solr, etc. > I think its time to remove it. We can continue to test 3.x codec with a > simple hack in PreFlexRW. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5727) Remove IndexOutput.seek
[ https://issues.apache.org/jira/browse/LUCENE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015711#comment-14015711 ] Michael McCandless commented on LUCENE-5727: +1 > Remove IndexOutput.seek > --- > > Key: LUCENE-5727 > URL: https://issues.apache.org/jira/browse/LUCENE-5727 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.9 > > Attachments: LUCENE-5727.patch > > > This has been deprecated / unused for a few years, and several things assume > append-only functionality: checksumming in the index format, HDFS integration > in solr, etc. > I think its time to remove it. We can continue to test 3.x codec with a > simple hack in PreFlexRW. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5727) Remove IndexOutput.seek
[ https://issues.apache.org/jira/browse/LUCENE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-5727: Attachment: LUCENE-5727.patch patch. > Remove IndexOutput.seek > --- > > Key: LUCENE-5727 > URL: https://issues.apache.org/jira/browse/LUCENE-5727 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.9 > > Attachments: LUCENE-5727.patch > > > This has been deprecated / unused for a few years, and several things assume > append-only functionality: checksumming in the index format, HDFS integration > in solr, etc. > I think its time to remove it. We can continue to test 3.x codec with a > simple hack in PreFlexRW. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5727) Remove IndexOutput.seek
Robert Muir created LUCENE-5727: --- Summary: Remove IndexOutput.seek Key: LUCENE-5727 URL: https://issues.apache.org/jira/browse/LUCENE-5727 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Fix For: 4.9 This has been deprecated / unused for a few years, and several things assume append-only functionality: checksumming in the index format, HDFS integration in solr, etc. I think its time to remove it. We can continue to test 3.x codec with a simple hack in PreFlexRW. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5724) CompoundFileWriter loses the IOContext sometimes
[
https://issues.apache.org/jira/browse/LUCENE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless resolved LUCENE-5724.
Resolution: Fixed
> CompoundFileWriter loses the IOContext sometimes
>
>
> Key: LUCENE-5724
> URL: https://issues.apache.org/jira/browse/LUCENE-5724
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5724.patch
>
>
> Nightly build hit OOME with this
> {noformat}
> ant test -Dtestcase=Test2BPostings -Dtests.method=test
> -Dtests.seed=33378E77AE43B10B -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=sl -Dtests.timezone=America/Argentina/ComodRivadavia
> -Dtests.file.encoding=UTF-8
> {noformat}
> The test was using NRTCachingDirectory, but the OOME happens because
> CompoundFileWriter's getOutput fails to pass down the incoming IOContext.
> IndexWriter has properly set up the IOContext for flush, put a huge file size
> in there, but by the time NRTCachingDirectory saw it, it was 0 bytes, and
> then many 100s of MB proceeded to be written into the RAMFile.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5724) CompoundFileWriter loses the IOContext sometimes
[
https://issues.apache.org/jira/browse/LUCENE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015687#comment-14015687
]
ASF subversion and git services commented on LUCENE-5724:
-
Commit 1599293 from [~mikemccand] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1599293 ]
LUCENE-5724: add CHANGES
> CompoundFileWriter loses the IOContext sometimes
>
>
> Key: LUCENE-5724
> URL: https://issues.apache.org/jira/browse/LUCENE-5724
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5724.patch
>
>
> Nightly build hit OOME with this
> {noformat}
> ant test -Dtestcase=Test2BPostings -Dtests.method=test
> -Dtests.seed=33378E77AE43B10B -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=sl -Dtests.timezone=America/Argentina/ComodRivadavia
> -Dtests.file.encoding=UTF-8
> {noformat}
> The test was using NRTCachingDirectory, but the OOME happens because
> CompoundFileWriter's getOutput fails to pass down the incoming IOContext.
> IndexWriter has properly set up the IOContext for flush, put a huge file size
> in there, but by the time NRTCachingDirectory saw it, it was 0 bytes, and
> then many 100s of MB proceeded to be written into the RAMFile.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5724) CompoundFileWriter loses the IOContext sometimes
[
https://issues.apache.org/jira/browse/LUCENE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015686#comment-14015686
]
ASF subversion and git services commented on LUCENE-5724:
-
Commit 1599292 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1599292 ]
LUCENE-5724: add CHANGES
> CompoundFileWriter loses the IOContext sometimes
>
>
> Key: LUCENE-5724
> URL: https://issues.apache.org/jira/browse/LUCENE-5724
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5724.patch
>
>
> Nightly build hit OOME with this
> {noformat}
> ant test -Dtestcase=Test2BPostings -Dtests.method=test
> -Dtests.seed=33378E77AE43B10B -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=sl -Dtests.timezone=America/Argentina/ComodRivadavia
> -Dtests.file.encoding=UTF-8
> {noformat}
> The test was using NRTCachingDirectory, but the OOME happens because
> CompoundFileWriter's getOutput fails to pass down the incoming IOContext.
> IndexWriter has properly set up the IOContext for flush, put a huge file size
> in there, but by the time NRTCachingDirectory saw it, it was 0 bytes, and
> then many 100s of MB proceeded to be written into the RAMFile.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5724) CompoundFileWriter loses the IOContext sometimes
[
https://issues.apache.org/jira/browse/LUCENE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015684#comment-14015684
]
ASF subversion and git services commented on LUCENE-5724:
-
Commit 1599291 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1599291 ]
LUCENE-5724: fix CompoundFileWriter to not suppress the incoming IOContext
> CompoundFileWriter loses the IOContext sometimes
>
>
> Key: LUCENE-5724
> URL: https://issues.apache.org/jira/browse/LUCENE-5724
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5724.patch
>
>
> Nightly build hit OOME with this
> {noformat}
> ant test -Dtestcase=Test2BPostings -Dtests.method=test
> -Dtests.seed=33378E77AE43B10B -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=sl -Dtests.timezone=America/Argentina/ComodRivadavia
> -Dtests.file.encoding=UTF-8
> {noformat}
> The test was using NRTCachingDirectory, but the OOME happens because
> CompoundFileWriter's getOutput fails to pass down the incoming IOContext.
> IndexWriter has properly set up the IOContext for flush, put a huge file size
> in there, but by the time NRTCachingDirectory saw it, it was 0 bytes, and
> then many 100s of MB proceeded to be written into the RAMFile.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5724) CompoundFileWriter loses the IOContext sometimes
[
https://issues.apache.org/jira/browse/LUCENE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015676#comment-14015676
]
ASF subversion and git services commented on LUCENE-5724:
-
Commit 1599288 from [~mikemccand] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1599288 ]
LUCENE-5724: fix CompoundFileWriter to not suppress the incoming IOContext
> CompoundFileWriter loses the IOContext sometimes
>
>
> Key: LUCENE-5724
> URL: https://issues.apache.org/jira/browse/LUCENE-5724
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5724.patch
>
>
> Nightly build hit OOME with this
> {noformat}
> ant test -Dtestcase=Test2BPostings -Dtests.method=test
> -Dtests.seed=33378E77AE43B10B -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=sl -Dtests.timezone=America/Argentina/ComodRivadavia
> -Dtests.file.encoding=UTF-8
> {noformat}
> The test was using NRTCachingDirectory, but the OOME happens because
> CompoundFileWriter's getOutput fails to pass down the incoming IOContext.
> IndexWriter has properly set up the IOContext for flush, put a huge file size
> in there, but by the time NRTCachingDirectory saw it, it was 0 bytes, and
> then many 100s of MB proceeded to be written into the RAMFile.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5726) Make all resourceDescriptions of indexinputs consistent
[ https://issues.apache.org/jira/browse/LUCENE-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015668#comment-14015668 ] Robert Muir commented on LUCENE-5726: - I think for this issue, the trick is to make CompoundFileDirectory not pass just the filename (_1.fdt) as the resource description, but instead something more complex. it should be a nicely named string that makes it clear its a CFS slice (ideally including the original handle description completely). I think its ok to build a good string here, because this is not called per query or anything. We can enhance the javadocs of slice() to say that the caller is responsible for passing the description in or something like that (caller has the original indexinput to toString() if it wants to use that). > Make all resourceDescriptions of indexinputs consistent > --- > > Key: LUCENE-5726 > URL: https://issues.apache.org/jira/browse/LUCENE-5726 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > This was weakly tested in a roundabout way by an assert that tried to read a > lucene 2.x index. > We should try to make these consistent, on the other hand we should be > careful about keeping the overhead of slice() relatively low (it would be > nice to use this api in more places). > And such things should be tested elsewhere, e.g. BaseDirectoryTestCase -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin
[
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015663#comment-14015663
]
Joel Bernstein edited comment on SOLR-6088 at 6/2/14 6:05 PM:
--
New patch, added tests for preserving document order when
QueryElevationComponent is used in conjuction with ReRanking.
was (Author: joel.bernstein):
New patch, preserves documents that were elevated by the
QueryElevationComponent.
> Add query re-ranking with the ReRankingQParserPlugin
>
>
> Key: SOLR-6088
> URL: https://issues.apache.org/jira/browse/SOLR-6088
> Project: Solr
> Issue Type: New Feature
> Components: search
>Reporter: Joel Bernstein
> Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch,
> SOLR-6088.patch, SOLR-6088.patch
>
>
> This ticket introduces the ReRankingQParserPlugin which adds query
> Reranking/Rescoring for Solr. It leverages the new RankQuery framework to
> plug-in the new Lucene QueryRescorer.
> See ticket LUCENE-5489 for details on the use case.
> Sample syntax:
> {code}
> q={!rerank mainQuery=$qq reRankQuery=$rqq reRankDocs=200}
> {code}
> In the example above the mainQuery is executed and 200 docs are collected and
> re-ranked based on the results of the reRankQuery.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin
[
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Bernstein updated SOLR-6088:
-
Attachment: SOLR-6088.patch
New patch, preserves documents that were elevated by the
QueryElevationComponent.
> Add query re-ranking with the ReRankingQParserPlugin
>
>
> Key: SOLR-6088
> URL: https://issues.apache.org/jira/browse/SOLR-6088
> Project: Solr
> Issue Type: New Feature
> Components: search
>Reporter: Joel Bernstein
> Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch,
> SOLR-6088.patch, SOLR-6088.patch
>
>
> This ticket introduces the ReRankingQParserPlugin which adds query
> Reranking/Rescoring for Solr. It leverages the new RankQuery framework to
> plug-in the new Lucene QueryRescorer.
> See ticket LUCENE-5489 for details on the use case.
> Sample syntax:
> {code}
> q={!rerank mainQuery=$qq reRankQuery=$rqq reRankDocs=200}
> {code}
> In the example above the mainQuery is executed and 200 docs are collected and
> re-ranked based on the results of the reRankQuery.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5726) Make all resourceDescriptions of indexinputs consistent
Robert Muir created LUCENE-5726: --- Summary: Make all resourceDescriptions of indexinputs consistent Key: LUCENE-5726 URL: https://issues.apache.org/jira/browse/LUCENE-5726 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir This was weakly tested in a roundabout way by an assert that tried to read a lucene 2.x index. We should try to make these consistent, on the other hand we should be careful about keeping the overhead of slice() relatively low (it would be nice to use this api in more places). And such things should be tested elsewhere, e.g. BaseDirectoryTestCase -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 23107 - Still Failing!
Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/23107/ 2 tests failed. FAILED: org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes Error Message: got exc message: Format version is not supported (resource: MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). This version of Lucene only supports indexes created with release 3.0 and later. Stack Trace: java.lang.AssertionError: got exc message: Format version is not supported (resource: MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). This version of Lucene only supports indexes created with release 3.0 and later. at __randomizedtesting.SeedInfo.seed([BDA188B98AFB198D:465ABA9A3A60BED1]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes(TestBackwardsCompatibility.java:331) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:745) FAILED: org.apache.lucene.index.TestBackwardsCompatib
[jira] [Commented] (LUCENE-4371) consider refactoring slicer to indexinput.slice
[
https://issues.apache.org/jira/browse/LUCENE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015659#comment-14015659
]
ASF subversion and git services commented on LUCENE-4371:
-
Commit 1599284 from [~rcmuir] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1599284 ]
LUCENE-4371: remove bogus and bogusly placed assert
> consider refactoring slicer to indexinput.slice
> ---
>
> Key: LUCENE-4371
> URL: https://issues.apache.org/jira/browse/LUCENE-4371
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Robert Muir
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-4371.patch, LUCENE-4371.patch, LUCENE-4371.patch,
> LUCENE-4371.patch, LUCENE-4371.patch
>
>
> From LUCENE-4364:
> {quote}
> In my opinion, we should maybe check, if we can remove the whole Slicer in
> all Indexinputs? Just make the slice(...) method return the current
> BufferedIndexInput-based one. This could be another issue, once this is in.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 23106 - Still Failing!
I'm looking into this. I broke it, but im upset tests passed twice on my machine... On Mon, Jun 2, 2014 at 1:45 PM, wrote: > Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/23106/ > > 2 tests failed. > FAILED: > org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes > > Error Message: > got exc message: Format version is not supported (resource: > MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). This version > of Lucene only supports indexes created with release 3.0 and later. > > Stack Trace: > java.lang.AssertionError: got exc message: Format version is not supported > (resource: MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). > This version of Lucene only supports indexes created with release 3.0 and > later. > at > __randomizedtesting.SeedInfo.seed([B199F6CA7BAAD1F8:4A62C4E9CB3176A4]:0) > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.assertTrue(Assert.java:43) > at > org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes(TestBackwardsCompatibility.java:331) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > at > com.
[JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 23106 - Still Failing!
Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/23106/ 2 tests failed. FAILED: org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes Error Message: got exc message: Format version is not supported (resource: MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). This version of Lucene only supports indexes created with release 3.0 and later. Stack Trace: java.lang.AssertionError: got exc message: Format version is not supported (resource: MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). This version of Lucene only supports indexes created with release 3.0 and later. at __randomizedtesting.SeedInfo.seed([B199F6CA7BAAD1F8:4A62C4E9CB3176A4]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes(TestBackwardsCompatibility.java:331) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:745) FAILED: org.apache.lucene.index.TestBackwardsCompatib
[jira] [Resolved] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler resolved LUCENE-5722. --- Resolution: Fixed Assignee: Uwe Schindler Thanks Robert for the quick backport of slicer removal! Also thanks for benchmarking - great work together. We can open another issue to maybe specialize the single buffer indexinput more (override readByte() / readBytes()). > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir >Assignee: Uwe Schindler > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, > LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015629#comment-14015629 ] ASF subversion and git services commented on LUCENE-5722: - Commit 1599278 from [~thetaphi] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599278 ] Merged revision(s) 1599276 from lucene/dev/trunk: LUCENE-5722: Optimize ByteBufferIndexInput#seek() by specializing implementations. This improves random access as used by docvalues codecs if used with MMapDirectory. > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, > LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers
[
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-5703:
-
Attachment: LUCENE-5703.patch
I missed that one. Here is an updated patch that removes the setTerm paranoia
from the terms enum of Lucene45DocValuesFormat.
> Don't allocate/copy bytes all the time in binary DV producers
> -
>
> Key: LUCENE-5703
> URL: https://issues.apache.org/jira/browse/LUCENE-5703
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5703.patch, LUCENE-5703.patch
>
>
> Our binary doc values producers keep on creating new {{byte[]}} arrays and
> copying bytes when a value is requested, which likely doesn't help
> performance. This has been done because of the way fieldcache consumers used
> the API, but we should try to fix it in 5.0.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 23105 - Failure!
Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/23105/ 2 tests failed. REGRESSION: org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes Error Message: got exc message: Format version is not supported (resource: MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). This version of Lucene only supports indexes created with release 3.0 and later. Stack Trace: java.lang.AssertionError: got exc message: Format version is not supported (resource: MockIndexInputWrapper(_0.fdx)): 1 (needs to be between 2 and 3). This version of Lucene only supports indexes created with release 3.0 and later. at __randomizedtesting.SeedInfo.seed([7517126C08A92274:8EEC204FB8328528]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.lucene.index.TestBackwardsCompatibility.testUnsupportedOldIndexes(TestBackwardsCompatibility.java:331) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:745) REGRESSION: org.apache.lucene.index.TestBackwards
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015604#comment-14015604 ] ASF subversion and git services commented on LUCENE-5722: - Commit 1599276 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1599276 ] LUCENE-5722: Optimize ByteBufferIndexInput#seek() by specializing implementations. This improves random access as used by docvalues codecs if used with MMapDirectory. > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, > LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4371) consider refactoring slicer to indexinput.slice
[
https://issues.apache.org/jira/browse/LUCENE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015599#comment-14015599
]
ASF subversion and git services commented on LUCENE-4371:
-
Commit 1599275 from [~rcmuir] in branch 'dev/trunk'
[ https://svn.apache.org/r1599275 ]
LUCENE-4371: move CHANGES entry
> consider refactoring slicer to indexinput.slice
> ---
>
> Key: LUCENE-4371
> URL: https://issues.apache.org/jira/browse/LUCENE-4371
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Robert Muir
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-4371.patch, LUCENE-4371.patch, LUCENE-4371.patch,
> LUCENE-4371.patch, LUCENE-4371.patch
>
>
> From LUCENE-4364:
> {quote}
> In my opinion, we should maybe check, if we can remove the whole Slicer in
> all Indexinputs? Just make the slice(...) method return the current
> BufferedIndexInput-based one. This could be another issue, once this is in.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-4371) consider refactoring slicer to indexinput.slice
[
https://issues.apache.org/jira/browse/LUCENE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-4371.
-
Resolution: Fixed
Fix Version/s: 4.9
> consider refactoring slicer to indexinput.slice
> ---
>
> Key: LUCENE-4371
> URL: https://issues.apache.org/jira/browse/LUCENE-4371
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Robert Muir
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-4371.patch, LUCENE-4371.patch, LUCENE-4371.patch,
> LUCENE-4371.patch, LUCENE-4371.patch
>
>
> From LUCENE-4364:
> {quote}
> In my opinion, we should maybe check, if we can remove the whole Slicer in
> all Indexinputs? Just make the slice(...) method return the current
> BufferedIndexInput-based one. This could be another issue, once this is in.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4371) consider refactoring slicer to indexinput.slice
[
https://issues.apache.org/jira/browse/LUCENE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015598#comment-14015598
]
ASF subversion and git services commented on LUCENE-4371:
-
Commit 1599274 from [~rcmuir] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1599274 ]
LUCENE-4371: Replace IndexInputSlicer with IndexInput.slice
> consider refactoring slicer to indexinput.slice
> ---
>
> Key: LUCENE-4371
> URL: https://issues.apache.org/jira/browse/LUCENE-4371
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Robert Muir
> Fix For: 5.0
>
> Attachments: LUCENE-4371.patch, LUCENE-4371.patch, LUCENE-4371.patch,
> LUCENE-4371.patch, LUCENE-4371.patch
>
>
> From LUCENE-4364:
> {quote}
> In my opinion, we should maybe check, if we can remove the whole Slicer in
> all Indexinputs? Just make the slice(...) method return the current
> BufferedIndexInput-based one. This could be another issue, once this is in.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers
[
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015582#comment-14015582
]
Robert Muir commented on LUCENE-5703:
-
{quote}
The term returned by TermsEnum.next() or TermsEnum.term() always comes from
Sorted(Set)DocValues.lookupOrd. It doesn't allocate its own terms.
{quote}
Well only in the base (simplistic) implementation. At least for the default
codec, the codec overrides TermsEnum and implements it special for performance
reasons. and lookupOrd in this case actually uses the termsenum (i think)
> Don't allocate/copy bytes all the time in binary DV producers
> -
>
> Key: LUCENE-5703
> URL: https://issues.apache.org/jira/browse/LUCENE-5703
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5703.patch
>
>
> Our binary doc values producers keep on creating new {{byte[]}} arrays and
> copying bytes when a value is requested, which likely doesn't help
> performance. This has been done because of the way fieldcache consumers used
> the API, but we should try to fix it in 5.0.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4371) consider refactoring slicer to indexinput.slice
[
https://issues.apache.org/jira/browse/LUCENE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015580#comment-14015580
]
Uwe Schindler commented on LUCENE-4371:
---
Thanks! We need this backported for the ByteBufferIndexInput improvements.
> consider refactoring slicer to indexinput.slice
> ---
>
> Key: LUCENE-4371
> URL: https://issues.apache.org/jira/browse/LUCENE-4371
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Robert Muir
> Fix For: 5.0
>
> Attachments: LUCENE-4371.patch, LUCENE-4371.patch, LUCENE-4371.patch,
> LUCENE-4371.patch, LUCENE-4371.patch
>
>
> From LUCENE-4364:
> {quote}
> In my opinion, we should maybe check, if we can remove the whole Slicer in
> all Indexinputs? Just make the slice(...) method return the current
> BufferedIndexInput-based one. This could be another issue, once this is in.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6128) SolrResourceLoader Error messages
[ https://issues.apache.org/jira/browse/SOLR-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015578#comment-14015578 ] Ahmet Arslan commented on SOLR-6128: Hi Omer, These are log messages printed at *warning* level. What is the problem you are facing here? > SolrResourceLoader Error messages > - > > Key: SOLR-6128 > URL: https://issues.apache.org/jira/browse/SOLR-6128 > Project: Solr > Issue Type: Bug > Components: contrib - Solr Cell (Tika extraction) >Affects Versions: 4.8.1 > Environment: Windows Server 2012 R2 >Reporter: Omer Arslan >Priority: Minor > > Solr loaded a deprecated plugin/analysis class [solr.IntField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.LongField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.FloatField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.DoubleField]. Please > consult documentation how to replace it accordingly. > Solr loaded a deprecated plugin/analysis class [solr.DateField]. Please > consult documentation how to replace it accordingly. > No stored data found for /rest/managed > No registered observers for /rest/managed -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers
[
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015569#comment-14015569
]
Adrien Grand commented on LUCENE-5703:
--
The term returned by TermsEnum.next() or TermsEnum.term() always comes from
Sorted(Set)DocValues.lookupOrd. It doesn't allocate its own terms.
> Don't allocate/copy bytes all the time in binary DV producers
> -
>
> Key: LUCENE-5703
> URL: https://issues.apache.org/jira/browse/LUCENE-5703
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5703.patch
>
>
> Our binary doc values producers keep on creating new {{byte[]}} arrays and
> copying bytes when a value is requested, which likely doesn't help
> performance. This has been done because of the way fieldcache consumers used
> the API, but we should try to fix it in 5.0.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5722: -- Attachment: LUCENE-5722.patch New patch that adds a test which checks the implementations returned after getting IndexInput and cloning/slicing. It asserts on random slices, their size and the chunkSize used. I will commit this later! > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, > LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers
[
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015559#comment-14015559
]
Robert Muir commented on LUCENE-5703:
-
Thanks for tackling this! I will help with the reviews, its a tricky one.
I didn't yet look, do SORTED/SORTED_SET TermsEnums also have the new behavior?
This was another source of stupid stuff, I think they should be consistent with
other termsenums (and now lookupOrd).
> Don't allocate/copy bytes all the time in binary DV producers
> -
>
> Key: LUCENE-5703
> URL: https://issues.apache.org/jira/browse/LUCENE-5703
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5703.patch
>
>
> Our binary doc values producers keep on creating new {{byte[]}} arrays and
> copying bytes when a value is requested, which likely doesn't help
> performance. This has been done because of the way fieldcache consumers used
> the API, but we should try to fix it in 5.0.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers
[
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-5703:
-
Attachment: LUCENE-5703.patch
Here is a patch that switches BinaryDocValues to the discussed API, as well as
Sorted(Set)DocValues.lookupOrd for consistency.
- the default codec as well as memory, direct and disk don't allocate the
byte[] anymore in BinaryDocValues.get.
- the default codec takes advantage of the maximum length of binary terms,
which is exposed in the metadata to never have to resize the BytesRef that
stores the term.
- old codecs (lucene40, lucene42) have moved to the new API but still allocate
the byte[] on the fly
- fixed grouping and comparators to not assume they own the bytes
- removed the two tests from BaseDocValuesFormatTestCase that ensured that
each return value had its own bytes
Tests pass (I ran the whole suite 6 times already) and I'll run benchmarks soon
to make sure that doesn't introduce a performance regression.
> Don't allocate/copy bytes all the time in binary DV producers
> -
>
> Key: LUCENE-5703
> URL: https://issues.apache.org/jira/browse/LUCENE-5703
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5703.patch
>
>
> Our binary doc values producers keep on creating new {{byte[]}} arrays and
> copying bytes when a value is requested, which likely doesn't help
> performance. This has been done because of the way fieldcache consumers used
> the API, but we should try to fix it in 5.0.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6119) TestReplicationHandler attempts to remove open folders
[
https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Thacker updated SOLR-6119:
Attachment: SOLR-6119.patch
{noformat}
[junit4] FAILURE 3.05s J1 | TestReplicationHandler.doTestBackup <<<
[junit4]> Throwable #1: java.lang.AssertionError: expected:<1> but
was:<2>
[junit4]>at
__randomizedtesting.SeedInfo.seed([227BE1FA9CDC834F:63F0C19FBB627000]:0)
[junit4]>at
org.apache.solr.handler.TestReplicationHandler.doTestBackup(TestReplicationHandler.java:1513)
[junit4]>at java.lang.Thread.run(Thread.java:745)
{noformat}
The file names created are - q and qjxiogjoksvtiijnlhhm and the test conditon
{code} if (name.startsWith("snapshot." + backupName)) { {code} should have
been equals.
Patch which fixes this.
Updating the patch against branch_4x.
> TestReplicationHandler attempts to remove open folders
> --
>
> Key: SOLR-6119
> URL: https://issues.apache.org/jira/browse/SOLR-6119
> Project: Solr
> Issue Type: Bug
>Reporter: Dawid Weiss
>Priority: Minor
> Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch,
> SOLR-6119.patch, SOLR-6119.patch
>
>
> TestReplicationHandler has a weird logic around the 'snapDir' variable. It
> attempts to remove snapshot folders, even though they're not closed yet. My
> recent patch uncovered the bug but I don't know how to fix it cleanly -- the
> test itself seems to be very fragile (for example I don't understand the
> 'namedBackup' variable which is always set to true, yet there are
> conditionals around it).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (LUCENE-4371) consider refactoring slicer to indexinput.slice
[
https://issues.apache.org/jira/browse/LUCENE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir reopened LUCENE-4371:
-
Reopen for 4.9 backport.
> consider refactoring slicer to indexinput.slice
> ---
>
> Key: LUCENE-4371
> URL: https://issues.apache.org/jira/browse/LUCENE-4371
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Robert Muir
> Fix For: 5.0
>
> Attachments: LUCENE-4371.patch, LUCENE-4371.patch, LUCENE-4371.patch,
> LUCENE-4371.patch, LUCENE-4371.patch
>
>
> From LUCENE-4364:
> {quote}
> In my opinion, we should maybe check, if we can remove the whole Slicer in
> all Indexinputs? Just make the slice(...) method return the current
> BufferedIndexInput-based one. This could be another issue, once this is in.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5725) More Like This: necessary improvement to run mlt on multi-field values.
[ https://issues.apache.org/jira/browse/LUCENE-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Ksikes updated LUCENE-5725: Attachment: LUCENE-5725.patch > More Like This: necessary improvement to run mlt on multi-field values. > --- > > Key: LUCENE-5725 > URL: https://issues.apache.org/jira/browse/LUCENE-5725 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Alex Ksikes >Priority: Minor > Attachments: LUCENE-5725.patch, LUCENE-5725.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6026) Also check work-queue while processing a REQUESTSTATUS Collection API Call
[ https://issues.apache.org/jira/browse/SOLR-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015505#comment-14015505 ] ASF subversion and git services commented on SOLR-6026: --- Commit 1599254 from [~anshumg] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599254 ] SOLR-6026: Also check work-queue while processing a REQUESTSTATUS Collection API Call (Merging r1599248 from trunk) > Also check work-queue while processing a REQUESTSTATUS Collection API Call > -- > > Key: SOLR-6026 > URL: https://issues.apache.org/jira/browse/SOLR-6026 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.8 >Reporter: Anshum Gupta > Fix For: 4.8.1 > > Attachments: SOLR-6026.patch, SOLR-6026.patch > > > REQUESTSTATUS API call should check for the following: > * work-queue (submitted task) > * running-map (running task/in progress) > * completed-map > * failure-map > Right now it checks everything but the work-queue. Add that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015472#comment-14015472 ] Uwe Schindler commented on LUCENE-5722: --- I will just add some more test to TestMultiMMap so it checks that the correct instances are returned (using instanceof). I just want to be sure, the 2 factory methods are creating the right impl classes for every combination of slices and master indexinputs. > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6026) Also check work-queue while processing a REQUESTSTATUS Collection API Call
[ https://issues.apache.org/jira/browse/SOLR-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015471#comment-14015471 ] ASF subversion and git services commented on SOLR-6026: --- Commit 1599248 from [~anshumg] in branch 'dev/trunk' [ https://svn.apache.org/r1599248 ] SOLR-6026: Also check work-queue while processing a REQUESTSTATUS Collection API Call > Also check work-queue while processing a REQUESTSTATUS Collection API Call > -- > > Key: SOLR-6026 > URL: https://issues.apache.org/jira/browse/SOLR-6026 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.8 >Reporter: Anshum Gupta > Fix For: 4.8.1 > > Attachments: SOLR-6026.patch, SOLR-6026.patch > > > REQUESTSTATUS API call should check for the following: > * work-queue (submitted task) > * running-map (running task/in progress) > * completed-map > * failure-map > Right now it checks everything but the work-queue. Add that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers
[
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-5703:
-
Fix Version/s: 4.9
> Don't allocate/copy bytes all the time in binary DV producers
> -
>
> Key: LUCENE-5703
> URL: https://issues.apache.org/jira/browse/LUCENE-5703
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.9, 5.0
>
>
> Our binary doc values producers keep on creating new {{byte[]}} arrays and
> copying bytes when a value is requested, which likely doesn't help
> performance. This has been done because of the way fieldcache consumers used
> the API, but we should try to fix it in 5.0.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6129) DateFormatTransformer doesn't resolve dateTimeFormat
[ https://issues.apache.org/jira/browse/SOLR-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron LaBella updated SOLR-6129: Attachment: SOLR-6129.patch > DateFormatTransformer doesn't resolve dateTimeFormat > > > Key: SOLR-6129 > URL: https://issues.apache.org/jira/browse/SOLR-6129 > Project: Solr > Issue Type: Bug > Components: contrib - DataImportHandler >Affects Versions: 4.8.1 >Reporter: Aaron LaBella > Fix For: 4.9 > > Attachments: SOLR-6129.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > The DateFormatTransformer doesn't use a VariableResolver to resolve the > format, which could potentially be passed in via a dataimport request > parameter. The attached patch is tested and working. Please include in 4.9. > Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6129) DateFormatTransformer doesn't resolve dateTimeFormat
Aaron LaBella created SOLR-6129: --- Summary: DateFormatTransformer doesn't resolve dateTimeFormat Key: SOLR-6129 URL: https://issues.apache.org/jira/browse/SOLR-6129 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 4.8.1 Reporter: Aaron LaBella Fix For: 4.9 The DateFormatTransformer doesn't use a VariableResolver to resolve the format, which could potentially be passed in via a dataimport request parameter. The attached patch is tested and working. Please include in 4.9. Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 86708 - Failure!
Unfortunately, the test class did not "know" it failed, so it didnt
print its log of information about where exceptions occurred.
But here is the seed so its not lost:
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestIndexWriter
-Dtests.method=testTwoThreadsInterruptDeadlock
-Dtests.seed=623CD77966D3A70E -Dtests.slow=true -Dtests.locale=ro_RO
-Dtests.timezone=America/Indiana/Vevay -Dtests.file.encoding=UTF-8
[junit4] ERROR 7189s J0 |
TestIndexWriter.testTwoThreadsInterruptDeadlock <<<
[junit4]> Throwable #1: java.lang.Exception: Test abandoned
because suite timeout was reached.
On Sun, Jun 1, 2014 at 5:31 PM, wrote:
> Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/86708/
>
> 2 tests failed.
> FAILED: junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriter
>
> Error Message:
> Suite timeout exceeded (>= 720 msec).
>
> Stack Trace:
> java.lang.Exception: Suite timeout exceeded (>= 720 msec).
> at __randomizedtesting.SeedInfo.seed([623CD77966D3A70E]:0)
>
>
> REGRESSION:
> org.apache.lucene.index.TestIndexWriter.testTwoThreadsInterruptDeadlock
>
> Error Message:
> Test abandoned because suite timeout was reached.
>
> Stack Trace:
> java.lang.Exception: Test abandoned because suite timeout was reached.
> at __randomizedtesting.SeedInfo.seed([623CD77966D3A70E]:0)
>
>
>
>
> Build Log:
> [...truncated 1574 lines...]
>[junit4] Suite: org.apache.lucene.index.TestIndexWriter
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriter
> -Dtests.method=testTwoThreadsInterruptDeadlock -Dtests.seed=623CD77966D3A70E
> -Dtests.slow=true -Dtests.locale=ro_RO -Dtests.timezone=America/Indiana/Vevay
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR 7189s J0 |
> TestIndexWriter.testTwoThreadsInterruptDeadlock <<<
>[junit4]> Throwable #1: java.lang.Exception: Test abandoned because
> suite timeout was reached.
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([623CD77966D3A70E]:0)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64-test-only/checkout/lucene/build/core/test/J0/./temp/lucene.index.TestIndexWriter-623CD77966D3A70E-001
>[junit4] 2> NOTE: test params are: codec=Lucene41,
> sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {e8=IB SPL-LZ(0.3),
> f93=DFR I(n)B2, a6=DFR GL1, d60=DFR GL2, b64=IB SPL-L2, b63=DFR I(ne)1,
> f9=DFR GLZ(0.3), d78=DFR I(n)B3(800.0), f34=IB LL-D1, e87=IB SPL-D1, a69=DFR
> I(ne)3(800.0), e66=IB LL-DZ(0.3), a31=DFR I(ne)BZ(0.3), a63=IB SPL-D2,
> f47=DFR G1, d27=DFR I(F)2, b78=IB LL-D1, d72=DFR I(F)BZ(0.3), c56=LM
> Jelinek-Mercer(0.70), a94=LM Jelinek-Mercer(0.70), d66=DFR
> I(n)LZ(0.3), f38=DFR I(ne)BZ(0.3), f78=DFR GLZ(0.3), c70=IB SPL-D3(800.0),
> c30=LM Jelinek-Mercer(0.70), b60=DFR I(ne)LZ(0.3), e12=IB LL-D2, b84=DFR
> I(n)BZ(0.3), f25=DFR I(n)1, d62=IB LL-L3(800.0), b79=DFR I(F)L3(800.0),
> c43=DFR I(ne)L3(800.0), field=DFR GL2, e32=DFR I(ne)B2, a77=DFR I(F)2, d89=IB
> SPL-L1, e83=DFR I(n)LZ(0.3), e27=DFR I(F)3(800.0), e86=DFR GB1, b2=DFR
> I(ne)B2, f44=IB SPL-D2, f68=DFR I(ne)LZ(0.3), c98=IB LL-D3(800.0), f56=DFR
> I(F)B3(800.0), content1=DFR GL3(800.0), d37=IB SPL-L3(800.0), f37=IB LL-D2,
> b33=DFR G2, b41=LM Jelinek-Mercer(0.10), b42=DFR I(ne)3(800.0), f11=IB
> LL-D2, d92=DFR I(ne)L2, e17=DefaultSimilarity, f64=DFR I(ne)B3(800.0),
> e44=DFR I(F)2, a53=IB LL-D1, f57=DFR I(ne)B2, a91=DFR I(n)B3(800.0), f39=DFR
> G3(800.0), a9=IB LL-L1, b98=IB LL-DZ(0.3), d50=DFR I(n)L1, c88=DFR I(n)L1,
> f69=DFR I(F)2, c86=DFR GBZ(0.3), c13=DFR I(F)B2, a87=IB SPL-L3(800.0),
> a50=DFR I(ne)B2, b7=IB LL-L2, a56=IB LL-D2, d15=DFR I(ne)B2, d12=DFR I(F)B2,
> f13=DFR G3(800.0), e6=IB LL-DZ(0.3), a64=DFR I(ne)1, a80=IB SPL-DZ(0.3),
> d26=DFR I(ne)LZ(0.3), str2=DFR GL1, c=DefaultSimilarity, b54=IB LL-L2,
> d31=DFR I(ne)L3(800.0), a10=IB SPL-D3(800.0), d51=DFR I(n)B2, a66=DFR G1,
> b12=DFR I(F)BZ(0.3), a39=DFR I(F)BZ(0.3), b6=DFR I(F)L3(800.0), d28=IB
> SPL-D2, b35=IB SPL-D3(800.0), f88=DFR GB3(800.0), d47=DFR I(n)2, e72=LM
> Jelinek-Mercer(0.70), e46=DFR I(ne)1, b95=DFR I(F)B1, f83=DFR
> I(n)B3(800.0), b25=DFR GBZ(0.3), b52=IB LL-D1, e67=DFR I(n)BZ(0.3), f80=DFR
> I(F)1, c31=DFR I(n)L3(800.0), a28=DFR I(n)L1, a36=IB SPL-D3(800.0), e64=DFR
> I(n)2, f28=DFR I(ne)B1, a48=DFR I(n)B1, f18=DFR GL1, f8=DFR I(F)B1, d45=DFR
> I(n)L3(800.0), b69=DFR I(n)1, c18=DFR I(F)Z(0.3), c20=DFR I(n)L3(800.0),
> a93=DFR I(F)LZ(0.3), d86=DFR GL2, b90=IB SPL-DZ(0.3), c34=DFR I(ne)B3(800.0),
> a8=DFR I(F)BZ(0.3), a52=DFR I(F)Z(0.3), d56=DFR I(n)L3(800.0), b30=DFR
> I(ne)BZ(0.3), b40=DFR I(F)L1, a70=DFR I(F)B1, a73=DFR I(F)B2, d88=IB
> LL-L3(800.0), d46=DFR GB3(800.0), b87=DFR I(F)2, d75=DFR I(F)1, a14=IB LL-L1,
> d42=DFR GZ(0.3), c60=DFR GBZ(0.3), f42=DFR I(ne)LZ(0.3), f97=LM
> Jelinek-Mercer(0.70), id=DFR I(n)2, c48=DFR GLZ(0.3), f15=DFR G2, c7=IB
> SPL-L2,
[jira] [Commented] (SOLR-6127) Improve Solr's exampledocs data
[
https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015459#comment-14015459
]
Ahmet Arslan commented on SOLR-6127:
I get exceptions with {noformat}Python 2.7.1 (r271:86832, Jun 16 2011,
16:59:05) {noformat} too. Sorry I am python ignorant, may be it is a good idea
to use same/compatible version that can run {{smokeTestRelease.py}} ?
> Improve Solr's exampledocs data
> ---
>
> Key: SOLR-6127
> URL: https://issues.apache.org/jira/browse/SOLR-6127
> Project: Solr
> Issue Type: Improvement
> Components: documentation
>Reporter: Varun Thacker
>Priority: Minor
> Fix For: 5.0
>
> Attachments: freebase_film_dump.py
>
>
> Currently
> - The CSV example has 10 documents.
> - The JSON example has 4 documents.
> - The XML example has 32 documents.
> 1. We should have equal number of documents and the same documents in all the
> example formats
> 2. A data set which is slightly more comprehensive.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015457#comment-14015457 ] Robert Muir commented on LUCENE-5722: - +1 to current patch, this is just more speed on top of previous improvements today. With my sort test, its an additional 7-10% (on top of previous commit which was similar). With a microbenchmark of numericdocvalues the improvement is way more substantial (it seems ~ 25%) In order to continue further, after this one is committed I want to exploit this slice API for packed ints, instead of clone()'ing the whole file in DV we just slice() what we need, remove offset adjustments in the packed ints decoder, and actually get more safety (read past EOF if you screw up instead of reading into another fields packed ints or whatever). In parallel I will begin work on backporting slice() api to 4.x, its baked for a while and I think is good to go. Ill start on this now. > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-6126) MapReduce's GoLive script should support replicas
[ https://issues.apache.org/jira/browse/SOLR-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley closed SOLR-6126. -- Resolution: Not a Problem Oooh, ok. I guess since in this mode of operation no documents are sent to any replica, all that needs to be done is to merge additional segments on each replica (leader or not is irrelevant). Interestingly, this GoLive script looks fairly generic. It's nothing more than a distributed addIndexes() call followed by a distributed commit. There's nothing Hadoop/HDFS oriented about it, even if it's only used in such contexts. > MapReduce's GoLive script should support replicas > - > > Key: SOLR-6126 > URL: https://issues.apache.org/jira/browse/SOLR-6126 > Project: Solr > Issue Type: Improvement > Components: contrib - MapReduce >Reporter: David Smiley > > The GoLive feature of the MapReduce contrib module is pretty cool. But a > comment in there indicates that it doesn't support replicas. Every > production SolrCloud setup I've seen has had replicas! > I wonder what is needed to support this. For GoLive to work, it assumes a > shared file system (be it HDFS or whatever, like a SAN). If perhaps the > replicas in such a system read from the very same network disk location, then > all we'd need to do is send a commit() to replicas; right? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5722: -- Attachment: (was: LUCENE-5722.patch) > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5722: -- Attachment: LUCENE-5722.patch New patch, last one had a bug (merge problem). > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5725) More Like This: necessary improvement to run mlt on multi-field values.
[
https://issues.apache.org/jira/browse/LUCENE-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015406#comment-14015406
]
Simon Willnauer commented on LUCENE-5725:
-
maybe we should just use variable args on the constructor like
{noformat}
public Query like(String fieldName, Reader... readers) {
//...
}
@Deprecated
public Query like(Reader r, String fieldName) throws IOException {
...
}
{noformat}
and can we have a test for it?
> More Like This: necessary improvement to run mlt on multi-field values.
> ---
>
> Key: LUCENE-5725
> URL: https://issues.apache.org/jira/browse/LUCENE-5725
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Alex Ksikes
>Priority: Minor
> Attachments: LUCENE-5725.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5722) Speed up MMapDirectory.seek()
[
https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-5722:
--
Attachment: LUCENE-5722.patch
New patch for current trunk:
- Removed the useless tests which were there for debugging only
- Factored out the factory for creating clone instances. For perf tests there
are now 2 places to modify:
# {{static ByteBufferIndexInput newInstance()}} called from MMapDirectory to
create the instance used to return as IndexInput
# {{protected ByteBufferIndexInput newCloneInstance()}} called when
clones/slices are requested. This one also takes offset into account.
For benchmarking we might comment out some parts of those 2 methods.
> Speed up MMapDirectory.seek()
> -
>
> Key: LUCENE-5722
> URL: https://issues.apache.org/jira/browse/LUCENE-5722
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
>Reporter: Robert Muir
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch,
> LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch
>
>
> For traditional lucene access which is mostly sequential, occasional
> advance(), I think this method gets drowned out in noise.
> But for access like docvalues, its important. Unfortunately seek() is complex
> today because of mapping multiple buffers.
> However, the very common case is that only one map is used for a given clone
> or slice.
> When there is the possibility to use only a single mapped buffer, we should
> instead take advantage of ByteBuffer.slice(), which will adjust the internal
> mmap address and remove the offset calculation. furthermore we don't need the
> shift/mask or even the negative check, as they are then all handled with the
> ByteBuffer api: seek is a one-liner (with try/catch of course to convert
> exceptions).
> This makes docvalues access 20% faster, I havent tested conjunctions or
> anyhting like that.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5725) More Like This: necessary improvement to run mlt on multi-field values.
[ https://issues.apache.org/jira/browse/LUCENE-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Ksikes updated LUCENE-5725: Attachment: LUCENE-5725.patch > More Like This: necessary improvement to run mlt on multi-field values. > --- > > Key: LUCENE-5725 > URL: https://issues.apache.org/jira/browse/LUCENE-5725 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Alex Ksikes >Priority: Minor > Attachments: LUCENE-5725.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5725) More Like This: necessary improvement to run mlt on multi-field values.
Alex Ksikes created LUCENE-5725: --- Summary: More Like This: necessary improvement to run mlt on multi-field values. Key: LUCENE-5725 URL: https://issues.apache.org/jira/browse/LUCENE-5725 Project: Lucene - Core Issue Type: Improvement Reporter: Alex Ksikes Priority: Minor -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015382#comment-14015382 ] ASF subversion and git services commented on LUCENE-5722: - Commit 1599219 from [~thetaphi] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599219 ] Merged revision(s) 1599218 from lucene/dev/trunk: LUCENE-5722: Speed up MMapDirectory.seek() - first small patch for multi-mmap case > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5722: -- Component/s: core/store > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5722: -- Fix Version/s: 5.0 4.9 > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015381#comment-14015381 ] ASF subversion and git services commented on LUCENE-5722: - Commit 1599218 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1599218 ] LUCENE-5722: Speed up MMapDirectory.seek() - first small patch for multi-mmap case > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015378#comment-14015378 ] Uwe Schindler commented on LUCENE-5722: --- OK, I will commit that to 4.x and trunk. Then I will upload a new patch for the big refactoring. > Speed up MMapDirectory.seek() > - > > Key: LUCENE-5722 > URL: https://issues.apache.org/jira/browse/LUCENE-5722 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, > LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch > > > For traditional lucene access which is mostly sequential, occasional > advance(), I think this method gets drowned out in noise. > But for access like docvalues, its important. Unfortunately seek() is complex > today because of mapping multiple buffers. > However, the very common case is that only one map is used for a given clone > or slice. > When there is the possibility to use only a single mapped buffer, we should > instead take advantage of ByteBuffer.slice(), which will adjust the internal > mmap address and remove the offset calculation. furthermore we don't need the > shift/mask or even the negative check, as they are then all handled with the > ByteBuffer api: seek is a one-liner (with try/catch of course to convert > exceptions). > This makes docvalues access 20% faster, I havent tested conjunctions or > anyhting like that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5205) [PATCH] SpanQueryParser with recursion, analysis and syntax very similar to classic QueryParser
[ https://issues.apache.org/jira/browse/LUCENE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015380#comment-14015380 ] Tim Allison commented on LUCENE-5205: - [~rcmuir], would you have any interest/time to pick up work on this again? Is there anything I can do to help? Thank you! > [PATCH] SpanQueryParser with recursion, analysis and syntax very similar to > classic QueryParser > --- > > Key: LUCENE-5205 > URL: https://issues.apache.org/jira/browse/LUCENE-5205 > Project: Lucene - Core > Issue Type: Improvement > Components: core/queryparser >Reporter: Tim Allison > Labels: patch > Fix For: 4.9 > > Attachments: LUCENE-5205-cleanup-tests.patch, > LUCENE-5205-date-pkg-prvt.patch, LUCENE-5205.patch.gz, LUCENE-5205.patch.gz, > LUCENE-5205_dateTestReInitPkgPrvt.patch, > LUCENE-5205_improve_stop_word_handling.patch, > LUCENE-5205_smallTestMods.patch, LUCENE_5205.patch, > SpanQueryParser_v1.patch.gz, patch.txt > > > This parser extends QueryParserBase and includes functionality from: > * Classic QueryParser: most of its syntax > * SurroundQueryParser: recursive parsing for "near" and "not" clauses. > * ComplexPhraseQueryParser: can handle "near" queries that include multiterms > (wildcard, fuzzy, regex, prefix), > * AnalyzingQueryParser: has an option to analyze multiterms. > At a high level, there's a first pass BooleanQuery/field parser and then a > span query parser handles all terminal nodes and phrases. > Same as classic syntax: > * term: test > * fuzzy: roam~0.8, roam~2 > * wildcard: te?t, test*, t*st > * regex: /\[mb\]oat/ > * phrase: "jakarta apache" > * phrase with slop: "jakarta apache"~3 > * default "or" clause: jakarta apache > * grouping "or" clause: (jakarta apache) > * boolean and +/-: (lucene OR apache) NOT jakarta; +lucene +apache -jakarta > * multiple fields: title:lucene author:hatcher > > Main additions in SpanQueryParser syntax vs. classic syntax: > * Can require "in order" for phrases with slop with the \~> operator: > "jakarta apache"\~>3 > * Can specify "not near": "fever bieber"!\~3,10 :: > find "fever" but not if "bieber" appears within 3 words before or 10 > words after it. > * Fully recursive phrasal queries with \[ and \]; as in: \[\[jakarta > apache\]~3 lucene\]\~>4 :: > find "jakarta" within 3 words of "apache", and that hit has to be within > four words before "lucene" > * Can also use \[\] for single level phrasal queries instead of " as in: > \[jakarta apache\] > * Can use "or grouping" clauses in phrasal queries: "apache (lucene solr)"\~3 > :: find "apache" and then either "lucene" or "solr" within three words. > * Can use multiterms in phrasal queries: "jakarta\~1 ap*che"\~2 > * Did I mention full recursion: \[\[jakarta\~1 ap*che\]\~2 (solr~ > /l\[ou\]\+\[cs\]\[en\]\+/)]\~10 :: Find something like "jakarta" within two > words of "ap*che" and that hit has to be within ten words of something like > "solr" or that "lucene" regex. > * Can require at least x number of hits at boolean level: "apache AND (lucene > solr tika)~2 > * Can use negative only query: -jakarta :: Find all docs that don't contain > "jakarta" > * Can use an edit distance > 2 for fuzzy query via SlowFuzzyQuery (beware of > potential performance issues!). > Trivial additions: > * Can specify prefix length in fuzzy queries: jakarta~1,2 (edit distance =1, > prefix =2) > * Can specifiy Optimal String Alignment (OSA) vs Levenshtein for distance > <=2: (jakarta~1 (OSA) vs jakarta~>1(Levenshtein) > This parser can be very useful for concordance tasks (see also LUCENE-5317 > and LUCENE-5318) and for analytical search. > Until LUCENE-2878 is closed, this might have a use for fans of SpanQuery. > Most of the documentation is in the javadoc for SpanQueryParser. > Any and all feedback is welcome. Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
[
https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015375#comment-14015375
]
Robert Muir commented on LUCENE-5722:
-
{quote}
For reference, this is the optimization I had in mind. I don't know if it helps
for the multi-buffer case, but may be worth a try.
The patch may not apply cleanly, its just for demonstartion purposes.
{quote}
I tested this with sorting on 1M and 10M wikipedia index: its a consistent 7%
improvement. +1 to just commit that one, and lets keep iterating on the more
complex refactor!
> Speed up MMapDirectory.seek()
> -
>
> Key: LUCENE-5722
> URL: https://issues.apache.org/jira/browse/LUCENE-5722
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch,
> LUCENE-5722.patch, LUCENE-5722.patch, LUCENE-5722.patch
>
>
> For traditional lucene access which is mostly sequential, occasional
> advance(), I think this method gets drowned out in noise.
> But for access like docvalues, its important. Unfortunately seek() is complex
> today because of mapping multiple buffers.
> However, the very common case is that only one map is used for a given clone
> or slice.
> When there is the possibility to use only a single mapped buffer, we should
> instead take advantage of ByteBuffer.slice(), which will adjust the internal
> mmap address and remove the offset calculation. furthermore we don't need the
> shift/mask or even the negative check, as they are then all handled with the
> ByteBuffer api: seek is a one-liner (with try/catch of course to convert
> exceptions).
> This makes docvalues access 20% faster, I havent tested conjunctions or
> anyhting like that.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6127) Improve Solr's exampledocs data
[ https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015349#comment-14015349 ] Varun Thacker commented on SOLR-6127: - I used python 2.7 You might need to modify the script to run on Python 3x - http://stackoverflow.com/questions/11914472/stringio-in-python3 Yes indeed the current exampledocs don't have the license in the JSON and CSV files while XML do. I guess we should fix that in this issue as well. > Improve Solr's exampledocs data > --- > > Key: SOLR-6127 > URL: https://issues.apache.org/jira/browse/SOLR-6127 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Varun Thacker >Priority: Minor > Fix For: 5.0 > > Attachments: freebase_film_dump.py > > > Currently > - The CSV example has 10 documents. > - The JSON example has 4 documents. > - The XML example has 32 documents. > 1. We should have equal number of documents and the same documents in all the > example formats > 2. A data set which is slightly more comprehensive. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6127) Improve Solr's exampledocs data
[
https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015347#comment-14015347
]
Ahmet Arslan commented on SOLR-6127:
I tried to run it with {noformat}
Python 3.4.0 (v3.4.0:04f714765c13, Mar 15 2014, 23:02:41)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
{noformat}
it complains : {noformat}
Traceback (most recent call last):
File "freebase_film_dump.py", line 5, in
import cStringIO
ImportError: No module named 'cStringIO'
{noformat}
I see that in example documents xml files has licence headers, json and cvs
don't. Does this add Licence headers to XML?
> Improve Solr's exampledocs data
> ---
>
> Key: SOLR-6127
> URL: https://issues.apache.org/jira/browse/SOLR-6127
> Project: Solr
> Issue Type: Improvement
> Components: documentation
>Reporter: Varun Thacker
>Priority: Minor
> Fix For: 5.0
>
> Attachments: freebase_film_dump.py
>
>
> Currently
> - The CSV example has 10 documents.
> - The JSON example has 4 documents.
> - The XML example has 32 documents.
> 1. We should have equal number of documents and the same documents in all the
> example formats
> 2. A data set which is slightly more comprehensive.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5720) Optimize on disk packed integers part 2
[ https://issues.apache.org/jira/browse/LUCENE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-5720. - Resolution: Fixed > Optimize on disk packed integers part 2 > --- > > Key: LUCENE-5720 > URL: https://issues.apache.org/jira/browse/LUCENE-5720 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5720.patch, LUCENE-5720.patch, LUCENE-5720.patch > > > These are heavily optimized for the in-RAM case (for example FieldCache uses > PackedInts.FAST to make it even faster so), but for the docvalues case they > are not: we always essentially use COMPACT, we have only one decoder that > must solve all the cases, even the complicated ones, we use BlockPackedWriter > for all integers (even if they are ordinals), etc. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5720) Optimize on disk packed integers part 2
[ https://issues.apache.org/jira/browse/LUCENE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015339#comment-14015339 ] ASF subversion and git services commented on LUCENE-5720: - Commit 1599182 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599182 ] LUCENE-5720: Optimize DirectPackedReader's decompression > Optimize on disk packed integers part 2 > --- > > Key: LUCENE-5720 > URL: https://issues.apache.org/jira/browse/LUCENE-5720 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5720.patch, LUCENE-5720.patch, LUCENE-5720.patch > > > These are heavily optimized for the in-RAM case (for example FieldCache uses > PackedInts.FAST to make it even faster so), but for the docvalues case they > are not: we always essentially use COMPACT, we have only one decoder that > must solve all the cases, even the complicated ones, we use BlockPackedWriter > for all integers (even if they are ordinals), etc. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5720) Optimize on disk packed integers part 2
[ https://issues.apache.org/jira/browse/LUCENE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015335#comment-14015335 ] ASF subversion and git services commented on LUCENE-5720: - Commit 1599180 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1599180 ] LUCENE-5720: Optimize DirectPackedReader's decompression > Optimize on disk packed integers part 2 > --- > > Key: LUCENE-5720 > URL: https://issues.apache.org/jira/browse/LUCENE-5720 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5720.patch, LUCENE-5720.patch, LUCENE-5720.patch > > > These are heavily optimized for the in-RAM case (for example FieldCache uses > PackedInts.FAST to make it even faster so), but for the docvalues case they > are not: we always essentially use COMPACT, we have only one decoder that > must solve all the cases, even the complicated ones, we use BlockPackedWriter > for all integers (even if they are ordinals), etc. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5720) Optimize on disk packed integers part 2
[ https://issues.apache.org/jira/browse/LUCENE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015313#comment-14015313 ] Adrien Grand commented on LUCENE-5720: -- +1 to the updated patch > Optimize on disk packed integers part 2 > --- > > Key: LUCENE-5720 > URL: https://issues.apache.org/jira/browse/LUCENE-5720 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5720.patch, LUCENE-5720.patch, LUCENE-5720.patch > > > These are heavily optimized for the in-RAM case (for example FieldCache uses > PackedInts.FAST to make it even faster so), but for the docvalues case they > are not: we always essentially use COMPACT, we have only one decoder that > must solve all the cases, even the complicated ones, we use BlockPackedWriter > for all integers (even if they are ordinals), etc. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5720) Optimize on disk packed integers part 2
[ https://issues.apache.org/jira/browse/LUCENE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-5720: Attachment: LUCENE-5720.patch Updated patch: since we are using these for numerics (and in those cases high BPV is common, e.g. floats/doubles/etc), i added 'byte' cases for bpv > 32 (40,48,56,64). I changed the upgrade logic, to try to go to byte first, then nibble. This means it also works like the in-ram one: if you pass FASTEST it always goes to some multiple of a byte. > Optimize on disk packed integers part 2 > --- > > Key: LUCENE-5720 > URL: https://issues.apache.org/jira/browse/LUCENE-5720 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Robert Muir > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5720.patch, LUCENE-5720.patch, LUCENE-5720.patch > > > These are heavily optimized for the in-RAM case (for example FieldCache uses > PackedInts.FAST to make it even faster so), but for the docvalues case they > are not: we always essentially use COMPACT, we have only one decoder that > must solve all the cases, even the complicated ones, we use BlockPackedWriter > for all integers (even if they are ordinals), etc. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - Failure
Hi, I think we should send some message to Rory/Oracle about this ChuckNorrisException - I have not yet done anything. I am a little bit afraid, because RAMUsageEstimator closely works with sun.misc.Unsafe (although not at this part of the code that fails), but they could maybe don't like this dependency of the code. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf > Of Dawid Weiss > Sent: Monday, June 02, 2014 11:03 AM > To: dev@lucene.apache.org > Subject: Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - Failure > > Has anybody contacted Oracle about this? Is this something known? > > Dawid > > On Mon, Jun 2, 2014 at 10:58 AM, Apache Jenkins Server > wrote: > > Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2005/ > > > > 1 tests failed. > > REGRESSION: > > org.apache.lucene.search.join.TestJoinUtil.testMultiValueRandomJoin > > > > Error Message: > > > > > > Stack Trace: > > java.lang.ArrayStoreException: > > at > __randomizedtesting.SeedInfo.seed([42A81A27864842D0:33CDC2C277FFF4D > C]:0) > > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsage > Estimator.java:674) > > at > org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEs > timator.java:420) > > at > org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java: > 333) > > at > org.apache.lucene.search.FieldCache$CacheEntry.estimateSize(FieldCache.j > ava:739) > > at > org.apache.lucene.util.FieldCacheSanityChecker.check(FieldCacheSanityChec > ker.java:104) > > at > org.apache.lucene.util.FieldCacheSanityChecker.checkSanity(FieldCacheSanit > yChecker.java:87) > > at > org.apache.lucene.util.LuceneTestCase.assertSaneFieldCaches(LuceneTestC > ase.java:781) > > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCa > cheSanity.java:55) > > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > fterRule.java:46) > > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 > .evaluate(SystemPropertiesInvariantRule.java:55) > > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleTh > readAndTestName.java:49) > > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > IgnoreAfterMaxFailures.java:65) > > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > .java:48) > > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > ementAdapter.java:36) > > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > run(ThreadLeakControl.java:360) > > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask > (ThreadLeakControl.java:793) > > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadL > eakControl.java:453) > > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran > domizedRunner.java:836) > > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Rando > mizedRunner.java:738) > > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando > mizedRunner.java:772) > > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > mizedRunner.java:783) > > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > fterRule.java:46) > > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl > assName.java:42) > > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 > .evaluate(SystemPropertiesInvariantRule.java:55) > > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > ementAdapter.java:36) > > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > ementAdapter.java:36) > > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > ementAdapter.java:36) > > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss > ertionsRequired.java:43) > > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > .java:48) > > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > IgnoreAfterMaxFailures.java:65) > > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(Tes
[jira] [Created] (SOLR-6128) SolrResourceLoader Error messages
Omer Arslan created SOLR-6128: - Summary: SolrResourceLoader Error messages Key: SOLR-6128 URL: https://issues.apache.org/jira/browse/SOLR-6128 Project: Solr Issue Type: Bug Components: contrib - Solr Cell (Tika extraction) Affects Versions: 4.8.1 Environment: Windows Server 2012 R2 Reporter: Omer Arslan Priority: Minor Solr loaded a deprecated plugin/analysis class [solr.IntField]. Please consult documentation how to replace it accordingly. Solr loaded a deprecated plugin/analysis class [solr.LongField]. Please consult documentation how to replace it accordingly. Solr loaded a deprecated plugin/analysis class [solr.FloatField]. Please consult documentation how to replace it accordingly. Solr loaded a deprecated plugin/analysis class [solr.DoubleField]. Please consult documentation how to replace it accordingly. Solr loaded a deprecated plugin/analysis class [solr.DateField]. Please consult documentation how to replace it accordingly. No stored data found for /rest/managed No registered observers for /rest/managed -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6096) Support Update and Delete on nested documents
[
https://issues.apache.org/jira/browse/SOLR-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015291#comment-14015291
]
Mikhail Khludnev commented on SOLR-6096:
bq. Currently I don't get it why that's not the default.
1. because it might be a backward incompatible behavior.
2. it's not really possible because right now Solr is unaware about parents
filter, and obtain it on every request.
to support what you need we need to add an ability to specify the schema of
nesting docs, somewhere in schema.xml.
> Support Update and Delete on nested documents
> -
>
> Key: SOLR-6096
> URL: https://issues.apache.org/jira/browse/SOLR-6096
> Project: Solr
> Issue Type: Improvement
>Affects Versions: 4.7.2
>Reporter: Thomas Scheffler
> Labels: blockjoin, nested
>
> When using nested or child document. Update and delete operation on the root
> document should also affect the nested documents, as no child can exist
> without its parent :-)
> Example
> {code:xml|title=First Import}
>
> 1
> Article with author
>
> Smith, John
> author
>
>
> {code}
> If I change my mind and the author was not named *John* but *_Jane_*:
> {code:xml|title=Changed name of author of '1'}
>
> 1
> Article with author
>
> Smith, Jane
> author
>
>
> {code}
> I would expect that John is not in the index anymore. Currently he is. There
> might also be the case that any subdocument is removed by an update:
> {code:xml|title=Remove author}
>
> 1
> Article without author
>
> {code}
> This should affect a delete on all nested documents, too. The same way all
> nested documents should be deleted if I delete the root document:
> {code:xml|title=Deletion of '1'}
>
> 1
>
>
> {code}
> This is currently possible to do all this stuff on client side by issuing
> additional request to delete document before every update. It would be more
> efficient if this could be handled on SOLR side. One would benefit on atomic
> update. The biggest plus shows when using "delete-by-query".
> {code:xml|title=Deletion of '1' by query}
>
> title:*
>
>
> {code}
> In that case one would not have to first query all documents and issue
> deletes by those id and every document that are nested.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-SmokeRelease-trunk - Build # 175 - Failure
I committed a fix.
On Mon, Jun 2, 2014 at 4:05 AM, Steve Rowe wrote:
> I'll fix.
> On Jun 2, 2014 1:39 AM, "Apache Jenkins Server"
> wrote:
>
>> Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-trunk/175/
>>
>> No tests ran.
>>
>> Build Log:
>> [...truncated 52964 lines...]
>> prepare-release-no-sign:
>> [mkdir] Created dir:
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/fakeRelease
>> [copy] Copying 431 files to
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/fakeRelease/lucene
>> [copy] Copying 230 files to
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/fakeRelease/solr
>> [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7
>> [exec] NOTE: output encoding is US-ASCII
>> [exec]
>> [exec] Load release URL
>> "file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/lucene/build/fakeRelease/"...
>> [exec]
>> [exec] Test Lucene...
>> [exec] test basics...
>> [exec] get KEYS
>> [exec] 0.1 MB in 0.01 sec (13.2 MB/sec)
>> [exec] check changes HTML...
>> [exec] download lucene-5.0.0-src.tgz...
>> [exec] 27.2 MB in 0.04 sec (664.9 MB/sec)
>> [exec] verify md5/sha1 digests
>> [exec] download lucene-5.0.0.tgz...
>> [exec] 60.6 MB in 0.16 sec (380.3 MB/sec)
>> [exec] verify md5/sha1 digests
>> [exec] download lucene-5.0.0.zip...
>> [exec] 69.9 MB in 0.10 sec (706.5 MB/sec)
>> [exec] verify md5/sha1 digests
>> [exec] unpack lucene-5.0.0.tgz...
>> [exec] verify JAR metadata/identity/no javax.* or java.*
>> classes...
>> [exec] test demo with 1.7...
>> [exec] got 5487 hits for query "lucene"
>> [exec] check Lucene's javadoc JAR
>> [exec] unpack lucene-5.0.0.zip...
>> [exec] verify JAR metadata/identity/no javax.* or java.*
>> classes...
>> [exec] test demo with 1.7...
>> [exec] got 5487 hits for query "lucene"
>> [exec] check Lucene's javadoc JAR
>> [exec] unpack lucene-5.0.0-src.tgz...
>> [exec] Traceback (most recent call last):
>> [exec] File
>> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
>> line 1343, in
>> [exec] main()
>> [exec] File
>> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
>> line 1287, in main
>> [exec] smokeTest(baseURL, svnRevision, version, tmpDir,
>> isSigned, testArgs)
>> [exec] File
>> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
>> line 1325, in smokeTest
>> [exec] unpackAndVerify('lucene', tmpDir, 'lucene-%s-src.tgz' %
>> version, svnRevision, version, testArgs, baseURL)
>> [exec] File
>> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
>> line 637, in unpackAndVerify
>> [exec] verifyUnpacked(project, artifact, unpackPath,
>> svnRevision, version, testArgs, tmpDir, baseURL)
>> [exec] File
>> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/dev-tools/scripts/smokeTestRelease.py",
>> line 708, in verifyUnpacked
>> [exec] raise RuntimeError('%s: unexpected files/dirs in artifact
>> %s: %s' % (project, artifact, l))
>> [exec] RuntimeError: lucene: unexpected files/dirs in artifact
>> lucene-5.0.0-src.tgz: ['ivy-ignore-conflicts.properties']
>>
>> BUILD FAILED
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-trunk/build.xml:387:
>> exec returned: 1
>>
>> Total time: 32 minutes 17 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>>
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
[jira] [Commented] (LUCENE-5442) Build system should sanity check transative 3rd party dependencies
[ https://issues.apache.org/jira/browse/LUCENE-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015273#comment-14015273 ] ASF subversion and git services commented on LUCENE-5442: - Commit 1599138 from [~steve_rowe] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599138 ] LUCENE-5442: smoke tester: 'ivy-ignore-conflicts.properties' is not a foreign invader (merge trunk r1599134) > Build system should sanity check transative 3rd party dependencies > -- > > Key: LUCENE-5442 > URL: https://issues.apache.org/jira/browse/LUCENE-5442 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build >Reporter: Hoss Man >Assignee: Steve Rowe > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5442.patch > > > SOLR-5365 is an example of a bug that croped up because we upgraded a 3rd > party dep (tika) w/o realizing that the version we upgraded too depended on a > newer version of another 3rd party dep (commons-compress) > in a comment in SOLR-5365, Jan suggested that it would be nice if there was > an easy way to spot problems like this ... i asked steve about it, thinking > maybe this is something the maven build could help with, and he mentioned > that there is already an ant task to inspect the ivy transative deps in order > to generate the maven deps and it could be used to help detect this sort of > problem. > opening this issue per steve's request as a reminder to look into this > possibility. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5442) Build system should sanity check transative 3rd party dependencies
[ https://issues.apache.org/jira/browse/LUCENE-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015272#comment-14015272 ] ASF subversion and git services commented on LUCENE-5442: - Commit 1599134 from [~steve_rowe] in branch 'dev/trunk' [ https://svn.apache.org/r1599134 ] LUCENE-5442: smoke tester: 'ivy-ignore-conflicts.properties' is not a foreign invader > Build system should sanity check transative 3rd party dependencies > -- > > Key: LUCENE-5442 > URL: https://issues.apache.org/jira/browse/LUCENE-5442 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build >Reporter: Hoss Man >Assignee: Steve Rowe > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5442.patch > > > SOLR-5365 is an example of a bug that croped up because we upgraded a 3rd > party dep (tika) w/o realizing that the version we upgraded too depended on a > newer version of another 3rd party dep (commons-compress) > in a comment in SOLR-5365, Jan suggested that it would be nice if there was > an easy way to spot problems like this ... i asked steve about it, thinking > maybe this is something the maven build could help with, and he mentioned > that there is already an ant task to inspect the ivy transative deps in order > to generate the maven deps and it could be used to help detect this sort of > problem. > opening this issue per steve's request as a reminder to look into this > possibility. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - Failure
Has anybody contacted Oracle about this? Is this something known? Dawid On Mon, Jun 2, 2014 at 10:58 AM, Apache Jenkins Server wrote: > Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2005/ > > 1 tests failed. > REGRESSION: > org.apache.lucene.search.join.TestJoinUtil.testMultiValueRandomJoin > > Error Message: > > > Stack Trace: > java.lang.ArrayStoreException: > at > __randomizedtesting.SeedInfo.seed([42A81A27864842D0:33CDC2C277FFF4DC]:0) > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsageEstimator.java:674) > at > org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEstimator.java:420) > at > org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java:333) > at > org.apache.lucene.search.FieldCache$CacheEntry.estimateSize(FieldCache.java:739) > at > org.apache.lucene.util.FieldCacheSanityChecker.check(FieldCacheSanityChecker.java:104) > at > org.apache.lucene.util.FieldCacheSanityChecker.checkSanity(FieldCacheSanityChecker.java:87) > at > org.apache.lucene.util.LuceneTestCase.assertSaneFieldCaches(LuceneTestCase.java:781) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:55) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) > at java.lang.Thread.run(Thread.java:724) > > > > > Build Log: > [...truncated 8432 lines...] >[junit4] Suite: org.apache.lucene.search.join.TestJoinUtil >[junit4] 2> *** BEGIN > testMultiValueRandomJoin(org.apache.lucene.search.join.TestJoinUtil): > FieldCache *** >[junit4] 2> > 'ParallelAtomicReader(FieldFilterAtomicReader(reader=SlowCompositeReaderWrapper(FCInvisibleMultiReader(StandardDirectoryReader(segments_4:11:nrt > _4(4.9):c38))), fields=![to, from, value]), > FieldFilterAtomicReader(reader=SlowCompositeReaderWrapper(FCInvisibleMultiReader(StandardDirectoryReader(
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2005 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2005/ 1 tests failed. REGRESSION: org.apache.lucene.search.join.TestJoinUtil.testMultiValueRandomJoin Error Message: Stack Trace: java.lang.ArrayStoreException: at __randomizedtesting.SeedInfo.seed([42A81A27864842D0:33CDC2C277FFF4DC]:0) at org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsageEstimator.java:674) at org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEstimator.java:420) at org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java:333) at org.apache.lucene.search.FieldCache$CacheEntry.estimateSize(FieldCache.java:739) at org.apache.lucene.util.FieldCacheSanityChecker.check(FieldCacheSanityChecker.java:104) at org.apache.lucene.util.FieldCacheSanityChecker.checkSanity(FieldCacheSanityChecker.java:87) at org.apache.lucene.util.LuceneTestCase.assertSaneFieldCaches(LuceneTestCase.java:781) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:55) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:724) Build Log: [...truncated 8432 lines...] [junit4] Suite: org.apache.lucene.search.join.TestJoinUtil [junit4] 2> *** BEGIN testMultiValueRandomJoin(org.apache.lucene.search.join.TestJoinUtil): FieldCache *** [junit4] 2> 'ParallelAtomicReader(FieldFilterAtomicReader(reader=SlowCompositeReaderWrapper(FCInvisibleMultiReader(StandardDirectoryReader(segments_4:11:nrt _4(4.9):c38))), fields=![to, from, value]), FieldFilterAtomicReader(reader=SlowCompositeReaderWrapper(FCInvisibleMultiReader(StandardDirectoryReader(segments_4:11:nrt _4(4.9):c38))), fields=[to, from, value]))'=>'to',class org.apache.lucene.index.DocTermOrds,null=>org.apache.lucene.index.DocTermOrds#446179646 [junit4] 2> *** END testMultiValueRandomJoin(org.apache.lucene.search.join.TestJoinUtil): FieldCache *** [junit4] 2> NOTE: reproduce with: ant test -D
[jira] [Commented] (SOLR-6126) MapReduce's GoLive script should support replicas
[ https://issues.apache.org/jira/browse/SOLR-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015266#comment-14015266 ] wolfgang hoschek commented on SOLR-6126: [~dsmiley] It uses the --zk-host CLI options to fetch the solr URLs of each replica from zk - see extractShardUrls(). This info gets passed via the Options.shardUrls parameter into the go-live phase. In the go-live phase the segments of each shard are explicitly merged via a separate REST merge request per replica into the corresponding replica. The result is that each input segment is explicitly merged N times where N is the replication factor. Each such merge reads from HDFS and writes to HDFS. (BTW, I'll be unreachable on an transatlantic flight very soon) > MapReduce's GoLive script should support replicas > - > > Key: SOLR-6126 > URL: https://issues.apache.org/jira/browse/SOLR-6126 > Project: Solr > Issue Type: Improvement > Components: contrib - MapReduce >Reporter: David Smiley > > The GoLive feature of the MapReduce contrib module is pretty cool. But a > comment in there indicates that it doesn't support replicas. Every > production SolrCloud setup I've seen has had replicas! > I wonder what is needed to support this. For GoLive to work, it assumes a > shared file system (be it HDFS or whatever, like a SAN). If perhaps the > replicas in such a system read from the very same network disk location, then > all we'd need to do is send a commit() to replicas; right? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2613) DIH Cache backed w/bdb-je
[ https://issues.apache.org/jira/browse/SOLR-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015264#comment-14015264 ] Adrian Rogers commented on SOLR-2613: - Is there going to be an update made to this case so that it's compatible with the latest version of SOLR-2943? It looks like a constant was removed or renamed which prevents this from compiling now. > DIH Cache backed w/bdb-je > - > > Key: SOLR-2613 > URL: https://issues.apache.org/jira/browse/SOLR-2613 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Affects Versions: 4.0-ALPHA >Reporter: James Dyer >Priority: Minor > Attachments: SOLR-2613.patch, SOLR-2613.patch, SOLR-2613.patch, > SOLR-2613.patch, SOLR-2613.patch, SOLR-2613.patch, SOLR-2613.patch > > > This is spun out of SOLR-2382, which provides a framework for multiple > cacheing implementations with DIH. This cache implementation is fast & > flexible, supporting persistence and delta updates. However, it depends on > Berkley Database Java Edition so in order to evaluate this and use it you > must download bdb-je from Oracle and accept the license requirements. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6127) Improve Solr's exampledocs data
[ https://issues.apache.org/jira/browse/SOLR-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Thacker updated SOLR-6127: Attachment: freebase_film_dump.py I thought Freebase would be a good place to get data from. [~thetaphi] - Would using the data from freebase ( https://developers.google.com/freebase/faq#rules_for_using_data ) be a licensing issue? If thats not a concern here is a script which fetches 200 rows of film data ( http://www.freebase.com/film ) and dumps it into JSON, XML and CSV. The number of documents can be adjusted. You would need to put in the API KEY for it to run. Any opinions if this is a good idea? > Improve Solr's exampledocs data > --- > > Key: SOLR-6127 > URL: https://issues.apache.org/jira/browse/SOLR-6127 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Varun Thacker >Priority: Minor > Fix For: 5.0 > > Attachments: freebase_film_dump.py > > > Currently > - The CSV example has 10 documents. > - The JSON example has 4 documents. > - The XML example has 32 documents. > 1. We should have equal number of documents and the same documents in all the > example formats > 2. A data set which is slightly more comprehensive. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
