[
https://issues.apache.org/jira/browse/GIRAPH-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claudio Martella updated GIRAPH-616:
------------------------------------
Attachment: GIRAPH-616.diff
Re-ran experiments, this time with bigger graph and more workers. The results
are more stable. Had to increase heap-space for in-memory to achieve good
performance. With same memory as with OOC it would still reach the end of
computation, but would run slower supposedly due to higher pressure on GC.
PR, 50M vertices, 100 edges each, 60 workers.
GIRAPH-616 in-memory
13/04/11 23:45:16 INFO mapred.JobClient: Total (milliseconds)=582072
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 3 (milliseconds)=47174
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 4 (milliseconds)=49080
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 10 (milliseconds)=1648
13/04/11 23:45:16 INFO mapred.JobClient: Setup (milliseconds)=20144
13/04/11 23:45:16 INFO mapred.JobClient: Shutdown (milliseconds)=198
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 7 (milliseconds)=50510
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 9 (milliseconds)=46757
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 0 (milliseconds)=53758
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 8 (milliseconds)=54245
13/04/11 23:45:16 INFO mapred.JobClient: Input superstep
(milliseconds)=47222
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 6 (milliseconds)=64889
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 5 (milliseconds)=52122
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 2 (milliseconds)=46064
13/04/11 23:45:16 INFO mapred.JobClient: Superstep 1 (milliseconds)=48257
GIRAPH-616 isStatic=true maxPartitionsInMemory=2
13/04/11 23:14:52 INFO mapred.JobClient: Total (milliseconds)=644252
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 3 (milliseconds)=52543
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 4 (milliseconds)=54847
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 10 (milliseconds)=16012
13/04/11 23:14:52 INFO mapred.JobClient: Setup (milliseconds)=20257
13/04/11 23:14:52 INFO mapred.JobClient: Shutdown (milliseconds)=242
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 7 (milliseconds)=52789
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 9 (milliseconds)=52341
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 0 (milliseconds)=51049
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 8 (milliseconds)=56641
13/04/11 23:14:52 INFO mapred.JobClient: Input superstep
(milliseconds)=47426
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 6 (milliseconds)=53835
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 5 (milliseconds)=58083
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 2 (milliseconds)=62154
13/04/11 23:14:52 INFO mapred.JobClient: Superstep 1 (milliseconds)=66027
GIRAPH-616 isStatic=false maxPartitionsInMemory=2
13/04/11 23:02:10 INFO mapred.JobClient: Giraph Timers
13/04/11 23:02:10 INFO mapred.JobClient: Total (milliseconds)=764215
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 3 (milliseconds)=72673
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 4 (milliseconds)=62751
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 10 (milliseconds)=25774
13/04/11 23:02:10 INFO mapred.JobClient: Setup (milliseconds)=25106
13/04/11 23:02:10 INFO mapred.JobClient: Shutdown (milliseconds)=54
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 7 (milliseconds)=64634
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 9 (milliseconds)=67493
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 0 (milliseconds)=49969
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 8 (milliseconds)=73192
13/04/11 23:02:10 INFO mapred.JobClient: Input superstep
(milliseconds)=53890
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 6 (milliseconds)=69300
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 5 (milliseconds)=60797
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 2 (milliseconds)=64752
13/04/11 23:02:10 INFO mapred.JobClient: Superstep 1 (milliseconds)=73824
> Decouple vertices and edges in DiskBackedPartitionStore and avoid writing
> back edges when the algorithm does not change topology.
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: GIRAPH-616
> URL: https://issues.apache.org/jira/browse/GIRAPH-616
> Project: Giraph
> Issue Type: Improvement
> Reporter: Claudio Martella
> Assignee: Claudio Martella
> Attachments: GIRAPH-616.diff, GIRAPH-616.diff
>
>
> Many algorithms work on a static graph. In these cases, when running
> out-of-core graph we end up writing back the edges that have not changed
> since we read them. By decoupling vertices and edges, we can write back only
> the freshly computed vertex values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
