Turns out to be a major screw-up on my behalf :) Passed line number as key instead of actual row key, ofcourse resulting in reducing and ordering 1 line at a time. Woops..
It's working fine now :) thanks for your time.. Menno On do, 2010-01-21 at 20:35 +0100, Menno Luiten wrote: > Manually added the "kv.clone()"-patch and it did not solve the issue, > unfortunately. > > Will most certainly dig deeper as to why this is happening and will post > the results here or in JIRA. > > Thanks > > On do, 2010-01-21 at 10:47 -0800, stack wrote: > > On Thu, Jan 21, 2010 at 9:48 AM, Menno Luiten <mlui...@artifix.net> wrote: > > > > > ...Then I use the KeyValueSortReducer as Reducer, and > > > limit myself to 1 reducer only, to prevent writing a custom Partitioner. > > > > > > > > This all sounds like you are setting things up properly. > > > > > > > > > java.io.IOException: Added a key not lexically larger than previous > > > key=im2.jpgaspectcolor/blue1---some-color/p.995�������, > > > lastkey=im2.jpgaspectcolor/cyan---some-color/p.995������� > > > > > > This is a complaint that comes up out of hfile if keys are not in strictly > > increasing lexicographical order. > > > > > > > > > I thought the KeyValueSortReducer would sort the output to prevent these > > > errors from occuring. > > > > > > Yes. Me too. There is a bug in hbase-0.20.2 KVSR -- hbase-2101 -- but it > > would not explain what you are seeing, as far as I can tell. > > > > > > > Am I doing something wrong or should I write my > > > own comparator which compares actual row,column pairs? > > > > > > You shouldn't have to. See the comparator passed to the TreeMap used > > internally in KVSR. It should do the compare properly. > > > > I'm not sure whats going wrong. You seem to have it set up right yet keys > > are going into hfile out of order. Any chance of your digging in to figure > > what is going awry? > > > > Thanks, > > St.Ack > >
