Thanks for your help Aapo. Definitely fee free to open/tackle issues on
the JIRA as you find them. =)
Avery
On 9/27/11 5:05 PM, Aapo Kyrola wrote:
Glad to hear about the progress. I am probably going to work on my algo
soon and try to run it with very big graphs. I plan to do some
profiling and
experimental modifications to Giraph code as well to get it to scale.
(Don't
worry, I am not committing anything :)). Will keep you updated.
On Sep 27, 2011, at 4:33 PM, Jake Mannix wrote:
On Tue, Sep 27, 2011 at 1:29 PM, Aapo Kyrola <[email protected]
<mailto:[email protected]>> wrote:
The code is still in draft stage, but I attached it.
Cool, thanks. The best place to usually attach it is to a JIRA
ticket which describes what
the code does, usually.
it is actually quite optimized code with regards to data
serialization etc.
It is just that Giraph currently takes a lot of memory (I guess
it is the RPC), which makes it difficult
to run algos like this that cannot use a combiner.
https://issues.apache.org/jira/browse/GIRAPH-28
is tracking some work toward reducing the memory footprint, but it
requires some work on
https://issues.apache.org/jira/browse/GIRAPH-36
before it'll work as well as it can. Similarly, the RPC (including
memory overhead) is being improved in explorations in
https://issues.apache.org/jira/browse/GIRAPH-12
and
https://issues.apache.org/jira/browse/GIRAPH-37
Watch those spaces for upcoming improvements!
-jake
Aapo
On Sep 27, 2011, at 4:14 PM, Jake Mannix wrote:
On Tue, Sep 27, 2011 at 12:31 PM, Aapo Kyrola
<[email protected] <mailto:[email protected]>> wrote:
I have written a very simple Belief Propagation algorithm
for binary
variables.
Not ready for prime-time either, though :).
That's awesome! Where's the code? :)
It shouldn't be construed that "not ready for prime-time" is
*bad*, in case that's what it looked like I was saying.
More examples the better, so we can see where the bottlenecks
are, and move them toward productionalized stage!
-jake
Aapo
On Sep 27, 2011, at 3:03 PM, Jake Mannix wrote:
Not really. It's really really early, and they're in the
"examples" stage - nothing is
really productionized. There's things like PageRank,
finding shortest path, but
nothing is really ready for prime time yet.
On Tue, Sep 27, 2011 at 11:56 AM, Josh Patterson
<[email protected] <mailto:[email protected]>> wrote:
Is there a list of known algorithms that have been
implemented on the
Giraph framework?
JP
--
Twitter: @jpatanooga
Solution Architect @ Cloudera
hadoop: http://www.cloudera.com <http://www.cloudera.com/>
Aapo Kyrola
Ph.D. student, http://www.cs.cmu.edu/~akyrola
<http://www.cs.cmu.edu/%7Eakyrola>
Aapo Kyrola
Ph.D. student, http://www.cs.cmu.edu/~akyrola
<http://www.cs.cmu.edu/%7Eakyrola>
Aapo Kyrola
Ph.D. student, http://www.cs.cmu.edu/~akyrola
<http://www.cs.cmu.edu/%7Eakyrola>
