Thanks Justin and Teryl. Justin, I think you are right. After reading this paper http://crd-legacy.lbl.gov/~kewu/ps/LBNL-59952.pdf, things are a bit more clear to me. FastBit needs to store the data in its own binary form to do vertical partitioning and support the binning option. Teryl, thanks for the suggestion. Yes, it should be possible to create a conceptual RAM disk. I was just wondering if FastBit actually has that option. It does not look like it. I might actually implement this. John, once you have a moment, it would be great to get your confirmation/blessing of this small discussion. Thanks!Mazen
Date: Sat, 4 Jul 2015 14:03:50 -0400 From: [email protected] To: [email protected] Subject: Re: [FastBit-users] Does FastBit need to use filesystem? Have you tried making a RAM disk to store the results? That way you can keep everything in memory and still support fastbits format. Cheers Teryl On Jul 4, 2015 2:00 PM, "Justin Swanhart" <[email protected]> wrote: Hi, John can correct me if I'm wrong but I think it is because the indexes can be binned, so the actual value must be retrieved for the column. --justin Sent from my iPhone On Jul 4, 2015, at 9:47 AM, Mazen Kachmar <[email protected]> wrote: Dear John,Thank you in advance for your time. I am bit of a novice in this area, so my apologies for the quality of my question. I studied FastBit quite a bit in the last couple of days and stepped through the tcapi example in particular. Let me refine my question below a bit. It looks like FastBit needs to serialize the index information into the filesystem. This is fine. However, I also noticedthat it also serializes a transformation of the original data as well (I was looking at fastbit_add_values). Is this true?If so, why is FastBit duplicating the original data (in a different form)? In other words, why can't FastBit use the compressed index information to map to the original data? Thank you,Mazen From: [email protected] To: [email protected] Date: Fri, 3 Jul 2015 12:27:53 +0000 Subject: [FastBit-users] Does FastBit need to use filesystem? Hi,I was reading on https://sdm.lbl.gov/~kewu/fastbit/doc/quickstart.html that FastBit partitions the data in a directory in the filesystem. Is this the only way to use FastBit? In other words, can FastBit be used as an in-memory data base? My data is in memory and I don't wish to serialize it to the hard disk. Thanks,Mazen _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
