Hi, Andrew, Thanks for the information. For those who are interested in reading more about hadoop and vertica, there are more information at <http://www.vertica.com/mapreduce>.
In the same vain, data warehousing vendors like aster and greenplum are also trying to take advantage of mapreduce paradigm as well, see <http://www.asterdata.com/resources/mapreduce.php> and <http://www.greenplum.com/technology/mapreduce/>. Here is an interesting article that lists who else is latching on to the same thing <http://news.idg.no/cw/art.cfm?id=94984D7A-1A64-67EA-E4C8E778D73F6D97> John On 12/16/2009 12:42 PM, Andrew Olson wrote: > Dear FastBit users, > I don't have any great ideas, but I attended Hadoop World NYC in October > where there was a presentation on integrating Vertica and Hadoop. The slide > show is at the following link. > http://www.slideshare.net/cloudera/hw09-hadoop-vertica > Vertica is a commercial column based database that is designed for analytics. > They mention software they have developed for moving data between vertica > and hadoop (map reduce). Vertica is used for rapid queries and map reduce > for massive computations. There may be some ideas on Vertica's website for > parallelizing FastBit in a more sophisticated manner. > > Andrew > > On Dec 16, 2009, at 11:00 AM, K. John Wu wrote: > >> Dear Hyeongu Son, >> >> We are very glad that you are using FastBit. Regarding the use of >> FastBit in a parallel environment, we have done a number of tests with >> a relatively straightforward setup by having each processor working on >> a different data partition. This may not be ideal if you have a large >> number of machines in your cluster, but it has been shown to work >> quite well and the user has to do only a minimal amount of programming. >> >> In order to do a more thorough integration with a system like Hadoop, >> FastBit probably has to be put underneath the Hadoop run-time system. >> As far as I know, HDFS does not support random I/O accesses, if this >> is indeed the case, then it would require a lot of work to put FastBit >> on top of HDFS -- something like writing a intermediate layer, say >> based on FUSION, to translate random I/O accesses into HDFS calls. >> >> I am copying this message to the FastBit mailing list in hope of >> someone else might have a better suggestion.. >> >> John >> >> >> >> On 12/16/2009 5:35 AM, Hyeongu Son wrote: >>> Hello, this is Hyeongu Son who is Chungnam National Univ. in South Korea. >>> I have a technical question >>> Can I use the FastBit such as parallel DBMS and HDFS in cluster environment? >>> I use HDFS(Hadoop Distributed File System), now. However, it is hard to >>> employ data compression such as binary file. >>> According to HDFS architecture, I can use it in serveral servers such as >>> one file system. >>> I read research papers and some document, but I could not find that >>> FastBit is used in cluster system. >>> How can I use FastBit in cluster system if it can be used? >>> Thank you for reading my questions. >>> your faithfully >>> Hyeongu Son >>> >>> >>> -- >>> Hyeongu Son >>> Office Room No. 424, >>> Building No. Eng.2 >>> Dept. of Compuer Science and Engineering, >>> Chungnam National University >>> Daejeon, Republic of Korea >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
