Hello J-D I have a similar requirement as that presented by the original poster, i.e updating a totals count without having to push the entire data set through the Mapper again.
Are you advising against calling incrementColumnValue on a mapper's HTable instance because the operation is not idempotent or are there other reasons? It is even suggested in the docs: http://hbase.apache.org/book/mapreduce.example.html (section 7.2.6). Do you know of any "count-exactly-once" implementations on top of Hadoop Map/Reduce? Thanks, /David On Tue, Jun 19, 2012 at 6:55 PM, Jean-Daniel Cryans <[email protected]>wrote: > This question was answered here already: > > http://mail-archives.apache.org/mod_mbox/hbase-user/201101.mbox/%[email protected]%3E > > Counters are not idempotent, this can be hard to manage. > > J-D > > On Mon, Jun 18, 2012 at 5:49 PM, Sid Kumar <[email protected]> wrote: > > Hi everyone, > > > > I have a use case in HBase that I was wondering if someone may have > > stumbled upon. I am maintaining an ad impressions table with columns that > > are counters for certain metrics. I started using the > incrementColumnValue > > method part of the HTable API to update these metrics and that works > great. > > I was wondering if this function could be used from a MapReduce job. > > The TableOutputFormat supports only Delete and Put operations. Using the > > Incremental counters saves me from doing any aggregations in my Map > Reduce > > code. Ideally i would like to just call this function in my mapper and > > wouldn't even need a Reducer. > > Has anyone run into this use case? I would also love to know if there > > are any better alternatives of solving this too. Any info would be great. > > > > Thanks > > Sid >
