> Since we call waitForCompletion() (which calls submit() internally) in > GiraphJob#run(), we cannot override those methods. Why can't we just call super.waitForCompletion() and override them to our users? It seems there are some advantages to GiraphJob being-a Job (easier integration with schedulers, Pig, etc.), but I'm not sure if it outweighs the inheritance issues that may be reaised...
On Wed, Feb 8, 2012 at 9:57 PM, David Garcia <dgar...@potomacfusion.com> wrote: > Not tonight. But I might be able to tomorrow. > > Sent from my HTC Inspire™ 4G on AT&T > > > ----- Reply message ----- > From: "Avery Ching" <ach...@apache.org> > To: "giraph-user@incubator.apache.org" <giraph-user@incubator.apache.org> > Subject: Giraph Architecture bug in > Date: Wed, Feb 8, 2012 8:17 pm > > > > AFAIK we don't have any SOP for opening issues. Maybe I'll take a crack > at this one tonight if I find some time, unless you were planning to > work on it David. > > Avery > > On 2/8/12 5:46 PM, David Garcia wrote: >> I opened up >> >> * GIRAPH-144<https://issues.apache.org/jira/browse/GIRAPH-144> >> >> >> I apologize if I didn't do it up according to project SOP's. I haven't >> had time to read it thoroughly. >> >> -David >> >> >> On 2/8/12 7:29 PM, "David Garcia"<dgar...@potomacfusion.com> wrote: >> >>> Yeah, I'll write something up. >>> >>> >>> On 2/8/12 7:26 PM, "Avery Ching"<ach...@apache.org> wrote: >>> >>>> Since we call waitForCompletion() (which calls submit() internally) in >>>> GiraphJob#run(), we cannot override those methods. A better fix would >>>> probably be to use composition rather than inheritance (i.e. >>>> >>>> public class GiraphJob { >>>> Job internalJob; >>>> } >>>> >>>> and expose the methods we would like as necessary. There are other >>>> methods we don't want the user to call, (i.e. setMapperClass(), etc.). >>>> David, can you please open an issue for this? >>>> >>>> Avery >>>> >>>> On 2/8/12 5:17 PM, David Garcia wrote: >>>>> This is a very subtle bug. GiraphJob inherits from >>>>> org.apache.mapreduce.Job. However, the methods submit() and >>>>> waitForCompletion() are not overridden. I assumed that they were >>>>> implemented, so when I called either one of these methods, the >>>>> framework >>>>> started up identity mappers/reducers. A simple fix is to throw >>>>> unsupported operation exceptions or to implement these methods. >>>>> Perhaps >>>>> this has been done already? >>>>> >>>>> -David >>>>> >>>>> On 2/7/12 7:46 PM, "David Garcia"<dgar...@potomacfusion.com> wrote: >>>>> >>>>>> I am running into a weird error that I haven't seen yet (I suppose >>>>>> I've >>>>>> been lucky). I see the following in the logging: >>>>>> >>>>>> org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop >>>>>> library for your platform... using builtin-java classes where >>>>>> applicable >>>>>> >>>>>> >>>>>> In the job definition, the property "mapreduce.map.class" is not even >>>>>> defined. For Giraph, this is usually set to >>>>>> "mapreduce.map.class=org.apache.giraph.graph.GraphMapper" >>>>>> >>>>>> I'm building my project with hadoop 0.20.204. >>>>>> >>>>>> When I build the GiraphProject myself (and run my own tests with the >>>>>> projects dependencies), I have no problems. The main difference is >>>>>> that >>>>>> I'm using a Giraph dependency in my work project. All input is >>>>>> welcome. >>>>>> Thx!! >>>>>> >>>>>> -David >>>>>> >
