Hi John,
After reading your response on a different thread ("Out of memory without
MMAP"), I'm thinking that increasing the fileManager cache might help me
with this problem as well. From the online documentation, I was able to
determine that I can set the cache size in a file named "ibis.rc" that is
located in the same directory where I am running "buildIndex." What should
the format of the configuration parameter be?
I tried:
fileManager.maxBytes = 5GB
But this doesn't seem to make a difference. Can you provide an example
configuration file?
Thanks,
Darryl
On Fri, Jul 25, 2014 at 7:02 PM, Darryl Reeves <[email protected]> wrote:
> Hi John,
>
> I just shared the file with you through my Google Drive account. Let me
> know if you have any problems accessing the file.
>
> The command that I have tried to run is:
>
> fastquery-0.8.2.8/examples/buildIndex -f sample.2.h5 -n k
>
> Thanks,
> Darryl
>
>
> On Thu, Jul 24, 2014 at 10:59 PM, John Wu <[email protected]> wrote:
>
>> Hi, Darryl,
>>
>> Are you able to share the test data with us? It would be useful for us
>> to reproduce the problem.
>>
>> If yes, please provide instructions on reproducing the problem.
>>
>> Thanks.
>>
>> K Wu
>> On Jul 24, 2014 1:25 AM, "Darryl Reeves" <[email protected]> wrote:
>>
>>> Hello,
>>>
>>> I am attempting to index a file consisting of unsigned integer data
>>> using the example program "buildIndex" included with fastquery. This
>>> program works fine for the test data included with the source code.
>>>
>>> However, when I try to index my data, I receive the following memory
>>> allocation error:
>>>
>>> Warning -- fileManager::storage::ctor unable to find 6,572,816,304 bytes
>>> of space in memory
>>> terminate called after throwing an instance of 'ibis::bad_alloc'
>>> what(): storage::ctor(memory) failed
>>>
>>> This error is revealed very early in the execution of the program. By
>>> monitoring the memory being used, I can tell that only 58756 Kb have been
>>> allocated by the program. The server where this is running has no shortage
>>> of RAM available. I am willing to share the dataset that I am using but it
>>> is 3.2 Gb, so it cannot be transferred as part of this message.
>>>
>>> Any help that can you can provide to figure out this issue is
>>> appreciated.
>>>
>>> Thanks,
>>>
>>> --
>>> Darryl Reeves
>>> Ph.D. Candidate
>>> Mason Lab
>>> Weill Cornell Medical College of Cornell University
>>> Tri-Institutional Program in Computational Biology and Medicine
>>>
>>> _______________________________________________
>>> FastBit-users mailing list
>>> [email protected]
>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>>
>>>
>> _______________________________________________
>> FastBit-users mailing list
>> [email protected]
>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>
>>
>
>
> --
> Darryl Reeves
> Ph.D. Candidate
> Mason Lab
> Weill Cornell Medical College of Cornell University
> Tri-Institutional Program in Computational Biology and Medicine
>
--
Darryl Reeves
Ph.D. Candidate
Mason Lab
Weill Cornell Medical College of Cornell University
Tri-Institutional Program in Computational Biology and Medicine
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
