Or put it in MySQL, or in S3, or...or... so my point was that you need a recipient that transcends the JVMs ;)
So it is doable and pretty normal to output in tables the result of MRs that map other tables, we have dozens of those here at StumbleUpon. But if it fits in a single HashMap in a single JVM, my guess is that the output is very small hence this is an operation done for live clients and not suitable for MR. J-D On Tue, Apr 6, 2010 at 4:34 AM, Michael Segel <michael_se...@hotmail.com> wrote: > > > J-D, > > There's an alternative... > > He could write a M/R that takes the input from a scan() , do something, > reduce() and then output the reduced set back to hbase in the form of a temp > table. > (Even an in memory temp table) and then at the end pull the data out in to a > hash table? > > In theory this should be possible, but I haven't had time to play with in > memory tables.... > > No? > > > Thx > > -Mike > >> Date: Mon, 5 Apr 2010 09:57:02 -0700 >> Subject: Re: hbase mapreduce scan >> From: jdcry...@apache.org >> To: hbase-user@hadoop.apache.org >> >> You want to put the result in a HashMap? MapReduce is a batch >> processing framework that runs multiple parallel JVMs over a cluster >> of machines so I don't see how you could simply output in a HashMap >> (unless you don't mind outputting on disk, then reading it back into a >> HashMap). >> >> So I will guess that you want to do a live query against HBase, here >> MR won't help you since that is meant for bulk processing which >> usually takes more than a minute. >> >> What you want to use is a Scan, using HTable. The unit tests have tons >> of example on how to use a scanner, look in the >> org.apache.hadoop.hbase.client package, so will find what you need. >> The main client package also contains some examples >> http://hadoop.apache.org/hbase/docs/r0.20.3/api/org/apache/hadoop/hbase/client/package-summary.html >> >> J-D >> >> On Sun, Apr 4, 2010 at 11:18 AM, Jürgen Jakobitsch <jakobits...@punkt.at> >> wrote: >> > hi, >> > >> > i'm totally new to hbase and mapreduce and could really need some >> > pointer into the right direction for the following situation. >> > >> > i managed to run a basic mapreduce example - analog to Export.java >> > in the hbase.mapreduce package. >> > >> > what i need to achieve is the following : >> > >> > do a map/reduce scan on a hbase table and put the results >> > into a HashMap. >> > >> > could someone point me to an example. >> > >> > any help really appreciated >> > >> > wkr turnguard.com/turnguard >> > >> > -- >> > punkt. netServices >> > ______________________________ >> > Jürgen Jakobitsch >> > Codeography >> > >> > Lerchenfelder Gürtel 43 Top 5/2 >> > A - 1160 Wien >> > Tel.: 01 / 897 41 22 - 29 >> > Fax: 01 / 897 41 22 - 22 >> > >> > netServices http://www.punkt.at >> > >> > > > _________________________________________________________________ > Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1