Andy, I didn't try this feature. But I know that Yahoo had a performance record with this file format. I came across a file system included in hadoop code (probably that one) when searching the source code. Luckily I found it: org.apache.hadoop.fs.InMemoryFileSystem But if you have a lot of big files, this approach won't be suitable I think.
Maybe someone can give further info. 2009/4/2 andy2005cst <[email protected]>: > > thanks for your reply. Let me explain more clearly, since Map Reduce is just > one step of my program, I need to use the output of reduce for furture > computation, so i do not need to want to wirte the output into disk, but > wanna to get the collection or list of the output in RAM. if it directly > wirtes into disk, I have to read it back into RAM again. > you have mentioned a special file format, will you please show me what is > it? and give some example if possible. > > thank you so much. > > > Rasit OZDAS wrote: >> >> Hi, hadoop is normally designed to write to disk. There are a special file >> format, which writes output to RAM instead of disk. >> But I don't have an idea if it's what you're looking for. >> If what you said exists, there should be a mechanism which sends output as >> objects rather than file content across computers, as far as I know there >> is >> no such feature yet. >> >> Good luck. >> >> 2009/4/2 andy2005cst <[email protected]> >> >>> >>> I need to use the output of the reduce, but I don't know how to do. >>> use the wordcount program as an example if i want to collect the >>> wordcount >>> into a hashtable for further use, how can i do? >>> the example just show how to let the result onto disk. >>> myemail is : [email protected] >>> looking forward your help. thanks a lot. >>> -- >>> View this message in context: >>> http://www.nabble.com/HELP%3A-I-wanna-store-the-output-value-into-a-list-not-write-to-the-disk-tp22844277p22844277.html >>> Sent from the Hadoop core-user mailing list archive at Nabble.com. >>> >>> >> >> >> -- >> M. Raşit ÖZDAŞ >> >> > > -- > View this message in context: > http://www.nabble.com/HELP%3A-I-wanna-store-the-output-value-into-a-list-not-write-to-the-disk-tp22844277p22848070.html > Sent from the Hadoop core-user mailing list archive at Nabble.com. > > -- M. Raşit ÖZDAŞ
