Glad to hear about the progress. I am probably going to work on my algo
soon and try to run it with very big graphs. I plan to do some profiling and
experimental modifications to Giraph code as well to get it to scale. (Don't
worry, I am not committing anything :)). Will keep you updated.


On Sep 27, 2011, at 4:33 PM, Jake Mannix wrote:

> 
> 
> On Tue, Sep 27, 2011 at 1:29 PM, Aapo Kyrola <akyr...@cs.cmu.edu> wrote:
> 
> 
> The code is still in draft stage, but I attached it.
> 
> Cool, thanks.  The best place to usually attach it is to a JIRA ticket which 
> describes what
> the code does, usually.
>  
> it is actually quite optimized code with regards to data serialization etc.
> 
> It is just that Giraph currently takes a lot of memory (I guess it is the 
> RPC), which makes it difficult
> to run algos like this that cannot use a combiner.
> 
>   https://issues.apache.org/jira/browse/GIRAPH-28 
> 
> is tracking some work toward reducing the memory footprint, but it requires 
> some work on 
> 
>   https://issues.apache.org/jira/browse/GIRAPH-36
> 
> before it'll work as well as it can.  Similarly, the RPC (including memory 
> overhead) is being improved in explorations in 
> 
>   https://issues.apache.org/jira/browse/GIRAPH-12 
> 
> and 
> 
>   https://issues.apache.org/jira/browse/GIRAPH-37
> 
> Watch those spaces for upcoming improvements!
> 
>   -jake
> 
> 
> Aapo
> 
> On Sep 27, 2011, at 4:14 PM, Jake Mannix wrote:
> 
>> 
>> 
>> On Tue, Sep 27, 2011 at 12:31 PM, Aapo Kyrola <akyr...@cs.cmu.edu> wrote:
>> 
>> I have written a very simple Belief Propagation algorithm for binary
>> variables.
>> 
>> Not ready for prime-time either, though :).
>> 
>> That's awesome!   Where's the code? :)
>> 
>> It shouldn't be construed that "not ready for prime-time" is *bad*, in case 
>> that's what it looked like I was saying.
>> 
>> More examples the better, so we can see where the bottlenecks are, and move 
>> them toward productionalized stage!
>>  
>>   -jake
>>  
>> 
>> Aapo
>> 
>> On Sep 27, 2011, at 3:03 PM, Jake Mannix wrote:
>> 
>>> Not really.  It's really really early, and they're in the "examples" stage 
>>> - nothing is
>>> really productionized.  There's things like PageRank, finding shortest 
>>> path, but 
>>> nothing is really ready for prime time yet.
>>> 
>>> On Tue, Sep 27, 2011 at 11:56 AM, Josh Patterson <j...@cloudera.com> wrote:
>>> Is there a list of known algorithms that have been implemented on the
>>> Giraph framework?
>>> 
>>> JP
>>> 
>>> --
>>> Twitter: @jpatanooga
>>> Solution Architect @ Cloudera
>>> hadoop: http://www.cloudera.com
>>> 
>> 
>> Aapo Kyrola
>> Ph.D. student, http://www.cs.cmu.edu/~akyrola
>> 
>> 
> 
> Aapo Kyrola
> Ph.D. student, http://www.cs.cmu.edu/~akyrola
> 
> 
> 

Aapo Kyrola
Ph.D. student, http://www.cs.cmu.edu/~akyrola

Reply via email to