Thanks for that info did not thank about it that way but good reason.
Billy
Hey,
It is not possible, nor alas would it be a good idea. Speculative
execution can cause jobs to run twice with the "results" discarded
from one. The hbase output format doesnt really have a good way to
'discard' results, since we are outputting to a table not to a file
that can be tossed.
Futhermore, failures will cause job reruns, and the ICV is not exactly
what you'd call idempotent. You can instantiate HTable and call ICV
directly yourself in either the map or reduce phase, but again, not
recommended.
You can also summarize your data and use a secondary process to
execute a roll up of ICVs... if the number isnt too massive this might
be acceptable.
On Tue, Jan 11, 2011 at 4:07 PM, Billy Pearson
<[email protected]> wrote:
Is there a way to make a mapreduce job and use incrementColumnValue in
place
of Put?
I am trying to move a job over from thrift and have to be able to use
incrementColumnValue
as a output but I can not seams to work it out with out calling HTable
every
map.
small example would be nice if anyone uses it now
Billy
