On Tue, Mar 17, 2015 at 9:46 AM, Thomas den Braber <[email protected]> wrote: > Hallo, > > I have done some installations on Windows 2012 with a 32bit Strawberry Perl > (64 bit is not yet ready) and I get frequent error like: > > Not enough storage is available to process this command. > (D:\sites\index\lcyindx10007\seg_3/sort-45.ord).
I don't recall having seen this particular error message before, but I believe that it occurs when `MapViewOfFile` (the Windows analogue to `mmap`) fails. Gory details: To support large files under 32-bit operating systems, we avoid mapping the entire file into memory and instead use a "sliding window" technique where only a portion of the file is mapped. But large indexes with sort caches put pressure on the design because the minimum amount of address space you can map is fairly large (I think 64k as opposed to typically 4k on Unixen), and portions of each sort cache have to be mapped. I speculate that if you have a lot of sortable fields, a lot more of the index is getting mapped into memory than we would like. I also wonder whether the way `MapViewOfFile` handles overlapping regions is optimal. > It looks like this error happens if there are many results returned from a > query that needs to be sorted in a particular order. > > There is enough memory available in the server and I have seen it on > different installations (servers). > I use Strawberry Perl 5.16 (32bit) with modperl 2.08 and Apache 2.2.19 (32 > bit) from apachelounche (compiled with vc2010) > > Do you have any idea how to fix that ? Maybe upgrading to 64bit will help, I > will try that when the 64 bit is ready but that can take some time. It would > be nice if there was an easy fix. I don't recall having tested Lucy under 64-bit Windows and I'm not sure what it would take to make it work. (Moving to 64-bit Unix would probably solve the problem, for whatever that's worth.) In the meantime, another possibility might be to cut down on the number of segments in the index. Try making a copy of a problematic index and calling Indexer#optimize before Indexer#commit, which will cause the consolidation of all segments into one. (This doesn't actually "optimize" much but it might help in this case.) Marvin Humphrey
