I've made another gist for this rather than clutter up the mail with code snippets: https://gist.github.com/4002132
I basically just changed all instances of Pair<GenericRecord, Integer> in the reducer with just GenericRecord. I also changed the output schema that gets set in the Main function. When I run this, I get a run time error that's also included in the above gist: "java.lang.IllegalArgumentException: Not a Pair schema:" The pom.xml file I'm using is also in this gist, in case I'm screwing up a version somewhere. My intent is to be running on CDH4 using MRv1 and Avro 1.7.1, and as far as I can tell from the pom.xml I'm doing just that. Could be mistaken. Thanks again for your time. On Thu, Nov 1, 2012 at 5:49 PM, Dave Beech <[email protected]> wrote: > Hi Brian > > I don't think the output from the reducer should be a Pair. You said > you got an error when you didn't use a Pair here - what was it? > > Cheers, > Dave > > On 1 November 2012 22:09, Brian Derickson <[email protected]> wrote: > > I've been pulling my hair out over this all day, and I'm hoping this is > > something simple I'm overlooking. > > > > The relevant portions of my code, the schema I'm using, and the stack > trace > > are at https://gist.github.com/3996847. > > > > I'm using Hadoop 0.20.2 and Avro 1.7.1 as part of CDH4. > > > > To briefly describe what I'm doing: the mapper (not included in the > gist) is > > taking a bam file and spitting out some information. The key is the > > chromosome and position colon delimited and the value is an integer. > > > > The reducer is summing up all the integers at a particular position and > > creating a Pair object containing a record using the schema included in > my > > gist. The second portion of the pair is an integer that I don't care > > about... if I didn't use a Pair here, I'd get an error. If this is > something > > I could do differently, please correct me. :) > > > > Every time this is run, I get the stack trace included in the gist. I've > run > > out of things to try to fix this... I'd really really appreciate any > help I > > can get. Thanks! > > > > > > >
