On Fri, Mar 28, 2014 at 1:22 AM, Gabriel Reid <[email protected]>wrote:
> Hi Jeremy, > > On Thu, Mar 27, 2014 at 3:26 PM, Jeremy Lewi <[email protected]> wrote: > > Hi > > > > I'm hitting the exception pasted below when using AvroPathPerKeyTarget. > > Interestingly, my code works just fine when I run on a small dataset > using > > the LocalJobTracker. However, when I run on a large dataset using a > hadoop > > cluster I hit the exception. > > > > Have you ever been able to successfully use the AvroPathPerKeyTarget > on a real cluster, or is this the first try with it? > > I'm wondering if this could be a problem that's always been around (as > the integration test for AvroPathPerKeyTarget also runs in the local > jobtracker), or if this could be something new. > +1-- Jeremy, if you force the job to run w/a single reducer on the cluster (i.e., via groupByKey(1)), does it work? > > - Gabriel >
