[jira] Commented: (LUCENE-2380) Add FieldCache.getTermBytes, to load term data as byte[]

2010-04-08 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854853#action_12854853 ] Toke Eskildsen commented on LUCENE-2380: Working on LUCENE-2369 I essentially

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-04-06 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854151#action_12854151 ] Toke Eskildsen commented on LUCENE-1990: I did a checkout with {code} sv

[jira] Commented: (LUCENE-2369) Locale-based sort by field with low memory overhead

2010-04-06 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854069#action_12854069 ] Toke Eskildsen commented on LUCENE-2369: A few experiments with the cur

[jira] Commented: (LUCENE-2369) Locale-based sort by field with low memory overhead

2010-04-06 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853886#action_12853886 ] Toke Eskildsen commented on LUCENE-2369: Earwin, it would be great if this ca

[jira] Commented: (LUCENE-2369) Locale-based sort by field with low memory overhead

2010-04-06 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853885#action_12853885 ] Toke Eskildsen commented on LUCENE-2369: Moved from LUCENE-2335 as it re

[jira] Updated: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-04-06 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-2335: --- Comment: was deleted (was: Lotsa devils in the details when you're poking around i

[jira] Commented: (LUCENE-2369) Locale-based sort by field with low memory overhead

2010-04-06 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853884#action_12853884 ] Toke Eskildsen commented on LUCENE-2369: The current implementation acc

[jira] Commented: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-04-06 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853847#action_12853847 ] Toke Eskildsen commented on LUCENE-2335: I've mis-read Michael M

[jira] Created: (LUCENE-2369) Locale-based sort by field with low memory overhead

2010-04-06 Thread Toke Eskildsen (JIRA)
Components: Search Reporter: Toke Eskildsen Priority: Minor The current implementation of locale-based sort in Lucene uses the FieldCache which keeps all sort terms in memory. Beside the huge memory overhead, searching requires comparison of terms with collator.compare every

Changing the subject for a JIRA-issue (Was: [jira] Created: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field

2010-04-06 Thread Toke Eskildsen
The current subject and description of https://issues.apache.org/jira/browse/LUCENE-2335 is obsolete due to new knowledge. Is it possible to change it? If not, what is the policy here? To open a new issue and close the old one? Cc: To Michael McCandless as he is the reporter of the issue. If it

[jira] Commented: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-04-05 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853569#action_12853569 ] Toke Eskildsen commented on LUCENE-2335: Lotsa devils in the details when yo

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-04-01 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852568#action_12852568 ] Toke Eskildsen commented on LUCENE-1990: I've located the bug and fix

[jira] Issue Comment Edited: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-04-01 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852557#action_12852557 ] Toke Eskildsen edited comment on LUCENE-1990 at 4/1/10 9:3

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-04-01 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852557#action_12852557 ] Toke Eskildsen commented on LUCENE-1990: It seem like my unit-testin

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-03-31 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852074#action_12852074 ] Toke Eskildsen commented on LUCENE-1990: In the original proof of concept

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-03-31 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851806#action_12851806 ] Toke Eskildsen commented on LUCENE-1990: I am very happy to hear that, Ro

[jira] Commented: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-03-26 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850174#action_12850174 ] Toke Eskildsen commented on LUCENE-2335: The exposed DirectoryReader is

[jira] Commented: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-03-26 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850055#action_12850055 ] Toke Eskildsen commented on LUCENE-2335: Just a little status on develop

[jira] Commented: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-03-23 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848589#action_12848589 ] Toke Eskildsen commented on LUCENE-2335: I can see that I messed up reading

[jira] Commented: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-03-20 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847766#action_12847766 ] Toke Eskildsen commented on LUCENE-2335: The sort-first-then-resolve-String

[jira] Commented: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache

2010-03-19 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847650#action_12847650 ] Toke Eskildsen commented on LUCENE-2335: I obviously have a share in this

RE: Sorting with little memory: A suggestion

2010-03-19 Thread Toke Eskildsen
MB for a given locale vs. the 10M*log2(10M)/8 = 27MB for a compressed order array. Still, depending on how little space a byte-array will take in flex, using the indexed collator key approach might turn out to be the best choice in a lot of cases as it works really well for incremental updates.

RE: Sorting with little memory: A suggestion

2010-03-19 Thread Toke Eskildsen
that a standard sort is used, which - to my knowledge - loads the Strings into memory. For my quick test, this means a tripling of memory usage for the sort field when indexing collatorKeys? Regards, Toke Eskildsen - To unsubscri

RE: Sorting with little memory: A suggestion

2010-03-19 Thread Toke Eskildsen
se. Currently the seatch API works both ways, providing methods both for simple Locale and for custom sorting, so I guess it would be the same for Exposed. Regards, Toke Eskildsen - To unsubscribe, e-mail: java-dev-unsubscr...@luce

Sorting with little memory: A suggestion

2010-03-19 Thread Toke Eskildsen
topic on the user-list, having an easy alternative to buying more RAM would be nice. I would like to hear if Exposed sounds like a feasible idea to the more seasoned Lucene developers. Regards, Toke Eskildsen - To unsubscribe, e-

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-03-14 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845082#action_12845082 ] Toke Eskildsen commented on LUCENE-1990: Thanks for rounding off, Michael.

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-03-09 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843027#action_12843027 ] Toke Eskildsen commented on LUCENE-1990: Michael McCandless: {quote}

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-03-09 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843028#action_12843028 ] Toke Eskildsen commented on LUCENE-1990: Uwe Schindler: {quote} We should

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-03-01 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839852#action_12839852 ] Toke Eskildsen commented on LUCENE-1990: Some thoughts on avoiding the gen

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-03-01 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: LUCENE-1990-te20100301.patch performance-20100301.txt I&#x

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-02-26 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: generated_performance-te20100226.txt LUCENE-1990

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-02-26 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: performance-te20100226.txt LUCENE-1990-te20100226b.patch I

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-02-26 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: LUCENE-1990-te20100226.patch Now we're getting somewhere. I finally squ

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-02-23 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: LUCENE-1990-te20100223.patch I've renamed most of the classes to short for

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-02-20 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836267#action_12836267 ] Toke Eskildsen commented on LUCENE-1990: I am sorry, but personal issues sa

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-02-11 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: LUCENE-1990-te20100212.patch I've read through the comments on LUCENE-199

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-02-10 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: LUCENE-1990-te20100210.patch Changing the code to use bitsPerValue instead of

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-25 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12804809#action_12804809 ] Toke Eskildsen commented on LUCENE-1990: {quote} I think we should re

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-22 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803872#action_12803872 ] Toke Eskildsen commented on LUCENE-1990: I think Michaels generated code

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-22 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: LUCENE-1990-te20100122.patch I've uploaded a preliminary patch with pac

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-22 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803628#action_12803628 ] Toke Eskildsen commented on LUCENE-1990: Looking at bit patterns and persist

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-20 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802888#action_12802888 ] Toke Eskildsen commented on LUCENE-1990: {quote} Toke: Introducing yet ano

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-19 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802542#action_12802542 ] Toke Eskildsen commented on LUCENE-1990: Introducing yet another leve

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-04 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: (was: ba.zip) > Add unsigned packed int impls in oal.u

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-04 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: LUCENE-1990_PerformanceMeasurements20100104.zip > Add unsigned packed int im

[jira] Updated: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-04 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toke Eskildsen updated LUCENE-1990: --- Attachment: ba.zip I made some small tweaks to improve performance and added long[]-backed

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2010-01-02 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795861#action_12795861 ] Toke Eskildsen commented on LUCENE-1990: The first section if for 1M value

[jira] Commented: (LUCENE-1990) Add unsigned packed int impls in oal.util

2009-12-29 Thread Toke Eskildsen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795228#action_12795228 ] Toke Eskildsen commented on LUCENE-1990: I have very recently done

Re: [jira] Created: (LUCENE-1257) Port to Java5

2008-04-08 Thread Toke Eskildsen
On Tue, 2008-04-08 at 18:48 -0500, robert engels wrote: > That is opposite of my testing:... > > The 'foreach' is consistently faster. The time difference is > independent of the size of the array. What I know about JVM > implementations, the foreach version SHOULD always be faster - > becau

Re: [jira] Created: (LUCENE-1257) Port to Java5

2008-04-04 Thread Toke Eskildsen
On Wed, 2008-04-02 at 09:30 -0400, Mark Miller wrote: > > - replacement of indexed for loops with for each constructs > > Is this always the best idea? Doesn't the for loop construct make an > iterator, which can be much slower than an indexed for loop? Only in the case of iterations over collect