Hi, Darryl, Thanks for your patience.
What program are using using? One of our programs or your own program? With most of the example programs we provide, there is an option -c, which is for you to specify the configuration file. John On 7/28/14 3:50 PM, Darryl Reeves wrote: > My apologies. I should have continued reading the documentation. I found a > sample configuration file here: > > http://crd-legacy.lbl.gov/~kewu/fastbit/doc/dataLoading.html#samplerc > > Unfortunately, even when I set the value to 1000000Mb > (fileManager.maxBytes=1000000Mb), I still get the same error originally > reported. > > > On Mon, Jul 28, 2014 at 1:29 PM, Darryl Reeves <[email protected]> wrote: > >> Hi John, >> >> After reading your response on a different thread ("Out of memory without >> MMAP"), I'm thinking that increasing the fileManager cache might help me >> with this problem as well. From the online documentation, I was able to >> determine that I can set the cache size in a file named "ibis.rc" that is >> located in the same directory where I am running "buildIndex." What should >> the format of the configuration parameter be? >> >> I tried: >> >> fileManager.maxBytes = 5GB >> >> But this doesn't seem to make a difference. Can you provide an example >> configuration file? >> >> Thanks, >> Darryl >> >> >> On Fri, Jul 25, 2014 at 7:02 PM, Darryl Reeves <[email protected]> wrote: >> >>> Hi John, >>> >>> I just shared the file with you through my Google Drive account. Let me >>> know if you have any problems accessing the file. >>> >>> The command that I have tried to run is: >>> >>> fastquery-0.8.2.8/examples/buildIndex -f sample.2.h5 -n k >>> >>> Thanks, >>> Darryl >>> >>> >>> On Thu, Jul 24, 2014 at 10:59 PM, John Wu <[email protected]> wrote: >>> >>>> Hi, Darryl, >>>> >>>> Are you able to share the test data with us? It would be useful for us >>>> to reproduce the problem. >>>> >>>> If yes, please provide instructions on reproducing the problem. >>>> >>>> Thanks. >>>> >>>> K Wu >>>> On Jul 24, 2014 1:25 AM, "Darryl Reeves" <[email protected]> wrote: >>>> >>>>> Hello, >>>>> >>>>> I am attempting to index a file consisting of unsigned integer data >>>>> using the example program "buildIndex" included with fastquery. This >>>>> program works fine for the test data included with the source code. >>>>> >>>>> However, when I try to index my data, I receive the following memory >>>>> allocation error: >>>>> >>>>> Warning -- fileManager::storage::ctor unable to find 6,572,816,304 >>>>> bytes of space in memory >>>>> terminate called after throwing an instance of 'ibis::bad_alloc' >>>>> what(): storage::ctor(memory) failed >>>>> >>>>> This error is revealed very early in the execution of the program. By >>>>> monitoring the memory being used, I can tell that only 58756 Kb have been >>>>> allocated by the program. The server where this is running has no shortage >>>>> of RAM available. I am willing to share the dataset that I am using but it >>>>> is 3.2 Gb, so it cannot be transferred as part of this message. >>>>> >>>>> Any help that can you can provide to figure out this issue is >>>>> appreciated. >>>>> >>>>> Thanks, >>>>> >>>>> -- >>>>> Darryl Reeves >>>>> Ph.D. Candidate >>>>> Mason Lab >>>>> Weill Cornell Medical College of Cornell University >>>>> Tri-Institutional Program in Computational Biology and Medicine >>>>> >>>>> _______________________________________________ >>>>> FastBit-users mailing list >>>>> [email protected] >>>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>>> >>>>> >>>> _______________________________________________ >>>> FastBit-users mailing list >>>> [email protected] >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> >>>> >>> >>> >>> -- >>> Darryl Reeves >>> Ph.D. Candidate >>> Mason Lab >>> Weill Cornell Medical College of Cornell University >>> Tri-Institutional Program in Computational Biology and Medicine >>> >> >> >> >> -- >> Darryl Reeves >> Ph.D. Candidate >> Mason Lab >> Weill Cornell Medical College of Cornell University >> Tri-Institutional Program in Computational Biology and Medicine >> > > > > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
