Hi, Otis, I could be totally wrong, but my understanding of HBase is that it simply scans all the records to answer a query. Vertica seems to offering its query capability alongside a mapreduce system <http://www.vertica.com/mapreduce>. Both Aster and GreenPlum use the basic idea of mapreduce, I know that Aster does not actually use hadoop, GreenPlum probably doesn't use hadoop either, but I am not sure. So there is plenty of freedom for you to innovate.
If you are concerned about how your work might be affected by FastBit, here is my thought on it. -- If you change things in directory src (i.e., the core of FastBit), we would like to have the changes in a patch so we can make the same changes to our code. If you do give us this patch, we assume agree to use the LGPL license for your contribution and your name will be appended to the list of contributors in the file AUTHORS. -- If you contribute some utilities built on top of FastBit software or alongside of FastBit software, we will place it in the directory contrib. For such software, you are free to choose your own license. -- If you build an extensive system that won't easily fit in the contrib directory or want to distribute your own code, we would be happy to acknowledge your work and direct interested users to it. The only thing we would like to ask is that you direct your users to download FastBit from us -- the organization that funds this work wants to know how many users are using the software (which we use the number of downloads as a proxy). Hope this helps. John PS: One way we acknowledge our users is to list the applications in the web page <http://crd.lbl.gov/~kewu/fastbit/applications.html>. At this point, most of the projects we know about are academic research efforts, one of which, TrixX-BMI is being commercialized. More serious commercial efforts are often less publicized, but occasionally we can get some hints of such uses, for example, from this resume <http://www.cs.unc.edu/~praveen/resume/PraveenResume.pdf> we can surmise that Yahoo! is using FastBit to handle some aspects of advertisements. Based on the information available in the same section of the resume, FastBit is apparently used to handle quite a lot of data. As a rule only those projects that we have permission to publicize are listed in the applications page. On 12/17/2009 7:49 PM, OG wrote: > Hi, > > Yes, HDFS does not allow random I/O access. > But what about HBase (a column-oriented database that reads and writes data > to/from HDFS)? Maybe there is room for Fastbit there! > > > Otis -- > Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch > > > > ----- Original Message ---- >> From: K. John Wu<[email protected]> >> To: FastBit Users<[email protected]> >> Sent: Thu, December 17, 2009 4:42:38 PM >> Subject: Re: [FastBit-users] Technical question >> >> Hi, Andrew, >> >> Thanks for the information. For those who are interested in reading >> more about hadoop and vertica, there are more information at >> . >> >> In the same vain, data warehousing vendors like aster and greenplum >> are also trying to take advantage of mapreduce paradigm as well, see >> and >> . >> >> Here is an interesting article that lists who else is latching on to >> the same thing >> >> >> John >> >> >> On 12/16/2009 12:42 PM, Andrew Olson wrote: >>> Dear FastBit users, >>> I don't have any great ideas, but I attended Hadoop World NYC in October >>> where >> there was a presentation on integrating Vertica and Hadoop. The slide show >> is >> at the following link. >>> http://www.slideshare.net/cloudera/hw09-hadoop-vertica >>> Vertica is a commercial column based database that is designed for >>> analytics. >> They mention software they have developed for moving data between vertica and >> hadoop (map reduce). Vertica is used for rapid queries and map reduce for >> massive computations. There may be some ideas on Vertica's website for >> parallelizing FastBit in a more sophisticated manner. >>> >>> Andrew >>> >>> On Dec 16, 2009, at 11:00 AM, K. John Wu wrote: >>> >>>> Dear Hyeongu Son, >>>> >>>> We are very glad that you are using FastBit. Regarding the use of >>>> FastBit in a parallel environment, we have done a number of tests with >>>> a relatively straightforward setup by having each processor working on >>>> a different data partition. This may not be ideal if you have a large >>>> number of machines in your cluster, but it has been shown to work >>>> quite well and the user has to do only a minimal amount of programming. >>>> >>>> In order to do a more thorough integration with a system like Hadoop, >>>> FastBit probably has to be put underneath the Hadoop run-time system. >>>> As far as I know, HDFS does not support random I/O accesses, if this >>>> is indeed the case, then it would require a lot of work to put FastBit >>>> on top of HDFS -- something like writing a intermediate layer, say >>>> based on FUSION, to translate random I/O accesses into HDFS calls. >>>> >>>> I am copying this message to the FastBit mailing list in hope of >>>> someone else might have a better suggestion.. >>>> >>>> John >>>> >>>> >>>> >>>> On 12/16/2009 5:35 AM, Hyeongu Son wrote: >>>>> Hello, this is Hyeongu Son who is Chungnam National Univ. in South Korea. >>>>> I have a technical question >>>>> Can I use the FastBit such as parallel DBMS and HDFS in cluster >>>>> environment? >>>>> I use HDFS(Hadoop Distributed File System), now. However, it is hard to >>>>> employ data compression such as binary file. >>>>> According to HDFS architecture, I can use it in serveral servers such as >>>>> one file system. >>>>> I read research papers and some document, but I could not find that >>>>> FastBit is used in cluster system. >>>>> How can I use FastBit in cluster system if it can be used? >>>>> Thank you for reading my questions. >>>>> your faithfully >>>>> Hyeongu Son >>>>> >>>>> >>>>> -- >>>>> Hyeongu Son >>>>> Office Room No. 424, >>>>> Building No. Eng.2 >>>>> Dept. of Compuer Science and Engineering, >>>>> Chungnam National University >>>>> Daejeon, Republic of Korea >>>> _______________________________________________ >>>> FastBit-users mailing list >>>> [email protected] >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>> >>> _______________________________________________ >>> FastBit-users mailing list >>> [email protected] >>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
