Sorting on primitive types
Hey, I have a question regarding the primitive type definitions and use of those for sorting. I have an ID field in my index of type SortableLongField, and on my test index I have about 2 million documents. When doing a sort=id desc and q=*:* I'm getting out of memory (heap space)... running the instance with 2GB of memory so I wouldn't really think that there should be any big problems here. So I'm wondering if the Trie based field types are less memory expensive than the old SortableXXFields? sorting on the date field (which is a TrieDateField) works fine (and fast)... Any input is highly appreciated! Cheers, Aleksander -- Aleksander M. Stensby Lead Software Developer and System Architect Integrasco A/S www.integrasco.com http://twitter.com/Integrasco http://facebook.com/Integrasco Please consider the environment before printing all or any of this e-mail
Re: Sorting on primitive types
On Mon, Sep 21, 2009 at 3:30 AM, Aleksander Stensby aleksander.sten...@integrasco.com wrote: So I'm wondering if the Trie based field types are less memory expensive than the old SortableXXFields? sorting on the date field (which is a TrieDateField) works fine (and fast)... In general, yes (assuming there are many unique values - your ID field would qualify). SortableXXFields used the StringIndex (the only option in the past)... Trie* fields FieldCache entry use long[maxDoc] for TrieLong and TrieDate. -Yonik http://www.lucidimagination.com
Re: Sorting on primitive types
Perfect, thanks a heap Yonik! Cheers, Aleks On Mon, Sep 21, 2009 at 3:47 PM, Yonik Seeley yo...@lucidimagination.comwrote: On Mon, Sep 21, 2009 at 3:30 AM, Aleksander Stensby aleksander.sten...@integrasco.com wrote: So I'm wondering if the Trie based field types are less memory expensive than the old SortableXXFields? sorting on the date field (which is a TrieDateField) works fine (and fast)... In general, yes (assuming there are many unique values - your ID field would qualify). SortableXXFields used the StringIndex (the only option in the past)... Trie* fields FieldCache entry use long[maxDoc] for TrieLong and TrieDate. -Yonik http://www.lucidimagination.com -- Aleksander M. Stensby Lead Software Developer and System Architect Integrasco A/S E-mail: aleksander.sten...@integrasco.com Tel.: +47 41 22 82 72 www.integrasco.com http://twitter.com/Integrasco http://facebook.com/Integrasco Please consider the environment before printing all or any of this e-mail
