Hello John Thank you for your answer. I have in mind to use FastBit in a way similar to what you present in your recent SC'11 paper (Parallel Index and Query for Large Scale Data Analysis), which I found very interesting.
I'm working on a framework for in-situ visualization of large scale simulations. It allows accessing data being generated by simulations directly in the memory of computation nodes, thus bypassing the filesystem. So integrating FastQuery in the process would simply be a matter of implementing the right driver behind what you call the "Array I/O interface" in the paper. Yet I noticed that FastQuery is not released yet (or I didn't search deep enough?) so I was thinking I could do this "from scratch" starting from FastBit. Apparently not. I'm interested if you eventually have a pre-release version of your code (is FastQuery the code you are mentioning?) and need a beta tester. Thanks, Matthieu 2011/11/17 K. John Wu <[email protected]> > Hi, Matthieu, > > You could use FastBit to do what you would like, but it will take some > work. > > - FastBit does not require you to explicitly generate x, y, and z, but > it does want you to put temperature in a file of its own named > temperature. It is possible to keep temperature in your own data file > if it is in HDF5, netCDF, or ADIOS BP format > <http://www.mcs.anl.gov/events/workshops/iasds11/papers/PID2005987.pdf> > > - once you have the data file temperature, you need to tell FastBit > that the data values are on a regular mesh. This is done with a > key-value pair in the header portion of the metadata file -part.txt. > Here is an example > > meshShape = (100, 100, 100) > > - you can issue queries of type ibis::meshQuery on the data partitions > with mesh shape defined. You can find more information about the > meshQuery class at > <http://lbl.gov/~kwu/fastbit/doc/html/classibis_1_1meshQuery.html>. > > > As you might guessed from the first item mentioned above, we are > working on something else to make your task a little easier. > Unfortunately, we have not got the official permission to release the > source code yet. We are happy to get a few beta testers or > collaborators if you are interested. > > John > > > > On 11/17/11 3:13 AM, Matthieu Dorier wrote: > > Hello, > > > > I'm new to FastBit and I have troubles understanding how to use its > > C++ API. I would like to do the following: > > I have a 3D array of floats representing a physical value (for > > instance "temperature") along a 3D grid. > > I would like to be able to answer queries like "give me all the > > coordinates (x,y,z) where temperature > T", without having to > > transform the original data into a list of (x,y,z,temperature) tuples, > > without making a copy of these data, without writing anything to a > > file (the resulting index could eventually be stored in a file, but I > > don't want it to be done automatically). In other words, I would like > > to simply wrap my data (along with other information if required) into > > a structure from which FastBit can work. > > How can I do that? > > > > Thanks > > > > -- > > Matthieu Dorier > > ENS Cachan, Brittany (Computer Science dpt.) > > IRISA Rennes, Office E324 > > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/ > > <http://perso.eleves.bretagne.ens-cachan.fr/%7Emdori307/wiki/> > > > > > > _______________________________________________ > > FastBit-users mailing list > > [email protected] > > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > -- Matthieu Dorier ENS Cachan, Brittany (Computer Science dpt.) IRISA Rennes, Office E324 http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
