Thanks for your help Aapo. Definitely fee free to open/tackle issues on the JIRA as you find them. =)

Avery

On 9/27/11 5:05 PM, Aapo Kyrola wrote:

Glad to hear about the progress. I am probably going to work on my algo
soon and try to run it with very big graphs. I plan to do some profiling and experimental modifications to Giraph code as well to get it to scale. (Don't
worry, I am not committing anything :)). Will keep you updated.


On Sep 27, 2011, at 4:33 PM, Jake Mannix wrote:



On Tue, Sep 27, 2011 at 1:29 PM, Aapo Kyrola <akyr...@cs.cmu.edu <mailto:akyr...@cs.cmu.edu>> wrote:



    The code is still in draft stage, but I attached it.


Cool, thanks. The best place to usually attach it is to a JIRA ticket which describes what
the code does, usually.

    it is actually quite optimized code with regards to data
    serialization etc.

    It is just that Giraph currently takes a lot of memory (I guess
    it is the RPC), which makes it difficult
    to run algos like this that cannot use a combiner.


https://issues.apache.org/jira/browse/GIRAPH-28

is tracking some work toward reducing the memory footprint, but it requires some work on

https://issues.apache.org/jira/browse/GIRAPH-36

before it'll work as well as it can. Similarly, the RPC (including memory overhead) is being improved in explorations in

https://issues.apache.org/jira/browse/GIRAPH-12

and

https://issues.apache.org/jira/browse/GIRAPH-37

Watch those spaces for upcoming improvements!

  -jake


    Aapo

    On Sep 27, 2011, at 4:14 PM, Jake Mannix wrote:



    On Tue, Sep 27, 2011 at 12:31 PM, Aapo Kyrola
    <akyr...@cs.cmu.edu <mailto:akyr...@cs.cmu.edu>> wrote:


        I have written a very simple Belief Propagation algorithm
        for binary
        variables.

        Not ready for prime-time either, though :).


    That's awesome!   Where's the code? :)

    It shouldn't be construed that "not ready for prime-time" is
    *bad*, in case that's what it looked like I was saying.

    More examples the better, so we can see where the bottlenecks
    are, and move them toward productionalized stage!
      -jake


        Aapo

        On Sep 27, 2011, at 3:03 PM, Jake Mannix wrote:

        Not really.  It's really really early, and they're in the
        "examples" stage - nothing is
        really productionized.  There's things like PageRank,
        finding shortest path, but
        nothing is really ready for prime time yet.

        On Tue, Sep 27, 2011 at 11:56 AM, Josh Patterson
        <j...@cloudera.com <mailto:j...@cloudera.com>> wrote:

            Is there a list of known algorithms that have been
            implemented on the
            Giraph framework?

            JP

            --
            Twitter: @jpatanooga
            Solution Architect @ Cloudera
            hadoop: http://www.cloudera.com <http://www.cloudera.com/>



        Aapo Kyrola
        Ph.D. student, http://www.cs.cmu.edu/~akyrola
        <http://www.cs.cmu.edu/%7Eakyrola>



    Aapo Kyrola
    Ph.D. student, http://www.cs.cmu.edu/~akyrola
    <http://www.cs.cmu.edu/%7Eakyrola>




Aapo Kyrola
Ph.D. student, http://www.cs.cmu.edu/~akyrola <http://www.cs.cmu.edu/%7Eakyrola>


Reply via email to