HI , I will be using Giraph to produce a poc at work and i am very new to hadoop . Please provide with suggestion on how to go about learning giraph
-----Original Message----- From: "Ameya Vilankar" <[email protected]> Sent: Friday, January 10, 2014 10:00pm To: [email protected] Subject: Re: Writing my own aggregator.. Your are welcome. If you don't mind me asking, what are you using Giraph for? Class project or on the job? On Fri, Jan 10, 2014 at 4:22 AM, Jyoti Yadav <[mailto:[email protected]] [email protected]> wrote: Thanks a lot Ameya...It really worked..:). On Fri, Jan 10, 2014 at 2:32 PM, Jyoti Yadav <[mailto:[email protected]] [email protected]> wrote: ---------- Forwarded message ---------- From: Ameya Vilankar <[mailto:[email protected]] [email protected]> Date: Fri, Jan 10, 2014 at 1:43 PM Subject: Re: Writing my own aggregator.. To: Jyoti Yadav <[mailto:[email protected]] [email protected]> This should solve it I think. If it doesn't email me the error. // MyArrayWritable.java package org.apache.giraph.examples.utils; import java.io.*; import org.apache.hadoop.io.Writable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.WritableComparator; import java.util.Arrays; import java.util.*; public class MyArrayWritable implements Writable { private ArrayList<Long> arraylist; public MyArrayWritable() { arraylist = new ArrayList<Long>(); } public MyArrayWritable(long toAdd) { arraylist = new ArrayList<Long>(); arraylist.add(toAdd); } public ArrayList<Long> get_arraylist() { return arraylist; } public void set_arraylist(ArrayList al) { this.arraylist = al; } @Override public void readFields(DataInput in) throws IOException { int size = in.readInt(); arraylist = new ArrayList<Long>(size); for(int i = 0; i < size; i++) { arraylist.add(in.readLong()); } } @Override public void write(DataOutput out) throws IOException { out.writeInt(arraylist.size()); for(int i = 0; i < arraylist.size(); i++) { out.writeLong(arraylist.get(i)); } } @Override public String toString() { return "output is "+ Long.toString(item) + "\n"; } } 2.MyArrayAggregator.java package org.apache.giraph.examples.utils; import org.apache.giraph.aggregators.BasicAggregator; import java.util.*; public class MyArrayAggregator extends BasicAggregator<MyArrayWritable> { @Override public void aggregate(MyArrayWritable value) { getAggregatedValue().get_arraylist().addAll(value); } @Override public MyArrayWritable createInitialValue() { return new MyArrayWritable(); } } On Fri, Jan 10, 2014 at 1:27 AM, Jyoti Yadav <[mailto:[email protected]] [email protected]> wrote: Hi Ameya.. I am badly stuck while implementing my custom aggregator.. In my program i want to send each vertex id to master. For that i took an arraylist, in which each vertex is adding its own id.while running the program,each vertex calls aggregate() function..As per my observation it is working fine in vertex compute method.But while retrieving back in master compute function.arraylist is not reflected back to master compute function. I am attaching two files below..You are requested to please check it once.. 1.MyArrayWritable.java package org.apache.giraph.examples.utils; import java.io.*; import org.apache.hadoop.io.Writable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.WritableComparator; import java.util.Arrays; import java.util.*; public class MyArrayWritable implements Writable { private long item; private ArrayList<Long> arraylist=new ArrayList<Long>(5); public MyArrayWritable() { item=0; //arraylist=new ArrayList<Long>(5); arraylist.add(item); } public MyArrayWritable(long item1) { item=item1; //arraylist=new ArrayList<Long>(5); arraylist.add(item); } public ArrayList<Long> get_arraylist() { return arraylist; } public void set_arraylist(ArrayList al) { //this.arraylist=new ArrayList<Long>(al); this.arraylist=al; } public long get_item(){return item;} @Override public void readFields(DataInput in) throws IOException { item=in.readLong(); int size=arraylist.size(); size=in.readInt(); arraylist=new ArrayList<Long>(5); for(int i=0;i<size;i++) { arraylist.add(in.readLong()); } } @Override public void write(DataOutput out) throws IOException { out.writeLong(item); out.writeInt(arraylist.size()); for(int i=0;i<arraylist.size();i++) { out.writeLong(arraylist.get(i)); } } @Override public String toString() { return "output is "+ Long.toString(item) + "\n"; } } 2.MyArrayAggregator.java package org.apache.giraph.examples.utils; import org.apache.giraph.aggregators.BasicAggregator; import java.util.*; public class MyArrayAggregator extends BasicAggregator<MyArrayWritable> { @Override public void aggregate(MyArrayWritable value) { ArrayList<Long> al=new ArrayList<Long>(); (getAggregatedValue().get_arraylist()).add(value.get_item()); al=getAggregatedValue().get_arraylist(); getAggregatedValue().set_arraylist(al); } @Override public MyArrayWritable createInitialValue() { return new MyArrayWritable(); } } Thanks in advance ... Jyoti ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address******** ******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential information and intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. The recipient should check this email and any attachments for the presence of viruses. THBS reserves the right to monitor and review the content of all messages sent to or from this e-mail address.********
