Thank you,

I will try distributed cache.

When I use distributed cache, the patches I have written will be
unnecassary ?


On 20 February 2012 03:44, David Garcia <dgar...@potomacfusion.com> wrote:

> so, if that's the case, it's possible that the Tasktracker process doesn't
> have the job on it's classpath.  Although you have added the jar to "a"
> classpath, I'm not certain that the Tasktracker will have it.  There are
> several ways to address this.  1.) you could bring Hadoop down, and then
> adjust hadoop-env.sh to export the HADOOP_CLASSPATH environment variable to
> include your jar.   This variable is commented out by default.  If you are
> running in distributed mode, this means that you will have to copy this jar
> to ever single machine...and probably change this script on every single
> machine too...unless you are using something like condor (or puppet if
> you're hard core serious), this is a serious pain...and for changing MR
> jobs, totally overkill.  My personal preference is to use the Distributed
> cache, and copy your jar to a location in hdfs:
> http://hadoop.apache.org/common/docs/r0.18.3/mapred_tutorial.html#DistributedCache
>
> hope this helps.
> ________________________________________
> From: yavuz gokirmak [ygokir...@gmail.com]
> Sent: Sunday, February 19, 2012 2:19 AM
> To: giraph-user@incubator.apache.org
> Subject: Re: how to use SimplePageRankVertex
>
> I am using pseudo distribudet cluster
>
> On 19 February 2012 02:00, David Garcia <dgar...@potomacfusion.com<mailto:
> dgar...@potomacfusion.com>> wrote:
> Are you submitting this job to a pseudo distributed cluster or a fully
> distributed cluster?
>
>
> Sent from my HTC Inspireā„¢ 4G on AT&T
>
> ----- Reply message -----
> From: "yavuz gokirmak" <ygokir...@gmail.com<mailto:ygokir...@gmail.com>>
> To: "giraph-user@incubator.apache.org<mailto:
> giraph-user@incubator.apache.org>" <giraph-user@incubator.apache.org
> <mailto:giraph-user@incubator.apache.org>>
> Subject: how to use SimplePageRankVertex
> Date: Sat, Feb 18, 2012 2:04 pm
>
>
>
> Thank you for advices,
>
> I have a few more questions.
>
> I have created a class named INTPageRankVertex which is similar to
> SimplePageRankVertex and generated a jar holding only
> INTPageRankVertex.java.
>
> Later, try to run with giraph command as below but get classpath errors:
>
> giraph INTPageRankVertex.jar org.test.INTPageRankVertex \
> -ip /user/hdfs/pagerankinput/graph.input \
> -op /user/hdfs/pagerankoutput/ \
> -w 1  \
> -if org.test.INTPageRankVertex.INTPageRankVertexInputFormat \
> -of org.test.INTPageRankVertex.INTPageRankVertexOutputFormat \
>
>
> First I get,
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.test.INTPageRankVertex
>
> in bin/giraph user jar is added to classpath on line 58
> 58. CLASSPATH=${USER_JAR}
>
> but CLASSPATH is overwritten on line 87
> 87.         CLASSPATH=`mvn dependency:build-classpath | grep -v "[INFO]"`
>
> changing line 87 as below solves my first problem. Does this patch is
> valid?
> 87.         CLASSPATH=$CLASSPATH:`mvn dependency:build-classpath | grep -v
> "[INFO]"`
>
> After changing line 87 I get a different classpath error:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/giraph/graph/LongDoubleFloatDoubleVertex
>
> And I solved this problem by adding below line
> 113. CLASSPATH=$CLASSPATH:$JAR
>
> Does these patches are necessary or I am doing something wrong while
> running my code..
>
> best regards..
>
>
> On 18 February 2012 18:37, Avery Ching <ach...@apache.org<mailto:
> ach...@apache.org>> wrote:
> IntIntNullIntTextInputFormat in the examples package (extending
> TextVertexInputFormat as David suggests) is very similar to what you need I
> think, although the types might be different for your application.  You can
> start with that perhaps.
>
> Avery
>
>
> On 2/18/12 7:48 AM, David Garcia wrote:
> The easiest thing to do is to extend text vertex or/and textvertext input
> format and/or the record reader.  The record reader will give you the
> vertices you want.  Look at the record reader for textvertexinputformat.
>  It's an innerclass on this format class.
>
> Sent from my HTC Inspireā„¢ 4G on AT&T
>
> ----- Reply message -----
> From: "yavuz gokirmak" <ygokir...@gmail.com><mailto:ygokir...@gmail.com>
> To: "giraph-user@incubator.apache.org"<mailto:
> giraph-user@incubator.apache.org> <giraph-user@incubator.apache.org
> ><mailto:giraph-user@incubator.apache.org>
> Subject: how to use SimplePageRankVertex
> Date: Sat, Feb 18, 2012 9:08 am
>
>
>
> Hi,
>
> I am planning to use giraph for network analysis. First I am trying to
> fully understand SimplePageRankVertex implementation and modify in order to
> serve my needs.
>
> I have a question about example,
> What is the expected input format for SimplePageRankVertex, I couldn't
> understand the input format although  SimplePageRankVertexReader class has
> few lines.
>
>
> My input file is contains of rows such as:
> usera, userb
> usera, userc
> userc, usera
> userb, userc
> userc, userb
> .
> .
> .
> Each row represents a relation between two users,
> "usera,userb" means that "usera is clicked userb's profile"
>
> Is it possible to make social network analysis over such kind of data
> using giraph?
> I will be glad if you can give advices..
>
> thanks in advance
> best regards
> ygokirmak
>
>
>
>

Reply via email to