[ 
https://issues.apache.org/jira/browse/GIRAPH-57?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169147#comment-13169147
 ] 

Avery Ching commented on GIRAPH-57:
-----------------------------------


Added a new request called putVertexIdMessagesList to reduce the total number 
of RPCs.  Performance has improved a lot.  The number of messages per call is 
configurable with 'giraph.maxMessages\
PerFlushPut'.  Here are some examples using RandomMessageBenchmark:

For large messages (10000 bytes) the improvement is decent, here's the old code:

2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Outputing statistics for superstep 5
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total bytes sent : 60000000000
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total bytes sent : 300000000000
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total messages : 6000000
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total messages : 30000000
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total millis : 928660
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total millis : 4782825
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 workers : 5
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second = 308.0807775955409
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second = 299.0934175114864
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second = 32304.61094480219
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second = 31362.21793605244
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second / worker = 61.616155519108176
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second / worker = 59.818683502297276
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second / worker = 6460.922188960438
2011-12-13 22:08:01,746 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second / worker = 6272.443587210488

And the new code with giraph.maxMessagesPerFlushPut=1000

2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Outputing statistics for superstep 5
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total bytes sent : 60000000000
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total bytes sent : 300000000000
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total messages : 6000000
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total messages : 30000000
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total millis : 769012
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total millis : 3816559
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 workers : 5
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second = 372.0387912306635
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second = 374.8170733399837
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second = 39011.09475534842
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second = 39302.41874945468
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second / worker = 74.4077582461327
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second / worker = 74.96341466799674
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second / worker = 7802.218951069684
2011-12-13 21:47:25,873 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second / worker = 7860.4837498909355

Performance improvement of 
(374.8170733399837-299.0934175114864)/299.0934175114864=.2531
About 25% more bytes / second

When the messages are smaller (50 bytes), the effect can be much more dramatic. 
 I think many applications (i.e. PageRank, shortest paths, etc.) will have 
smaller messages and actually see this effect.

Old code:

2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Outputing statistics for superstep 5
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total bytes sent : 600000000
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total bytes sent : 3000000000
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total messages : 12000000
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total messages : 60000000
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total millis : 280977
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total millis : 1456479
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 workers : 5
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second = 10.182409767414237
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second = 9.821710265711864
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second = 213540.61008552302
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second = 205976.19327158167
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second / worker = 2.0364819534828476
2011-12-13 22:17:39,232 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second / worker = 1.964342053142373
2011-12-13 22:17:39,233 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second / worker = 42708.1220171046
2011-12-13 22:17:39,233 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second / worker = 41195.23865431633

New code with giraph.maxMessagesPerFlushPut=5000

2011-12-13 22:37:43,610 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Outputing statistics for superstep 5
2011-12-13 22:37:43,610 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total bytes sent : 600000000
2011-12-13 22:37:43,610 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total bytes sent : 3000000000
2011-12-13 22:37:43,610 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total messages : 12000000
2011-12-13 22:37:43,610 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total messages : 60000000
2011-12-13 22:37:43,610 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 superstep total millis : 23975
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 total millis : 123956
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 workers : 5
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second = 119.33359537930136
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second = 115.40477868028776
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second = 2502606.882168926
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second = 2420213.6241892283
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep megabytes / second / worker = 23.86671907586027
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total megabytes / second / worker = 23.080955736057554
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Superstep messages / second / worker = 500521.3764337852
2011-12-13 22:37:43,611 INFO 
org.apache.giraph.benchmark.RandomMessageBenchmark$RandomMessageBenchmarkWorkerContext:
 Total messages / second / worker = 484042.72483784566

Performance improvement of about 
(115.40477868028776-9.821710265711864)/9.821710265711864=10.7499
About 1075% more bytes / second


                
> Provide PutMsgs RPC call
> ------------------------
>
>                 Key: GIRAPH-57
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-57
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Jakob Homan
>            Assignee: Avery Ching
>
> Right now messages are sent to a vertex one at a time.  It would be good to 
> have a putMsgs call that could send messages to multiple vertices (all hosted 
> on the same worker).  We'd save a huge number of individual RPC calls at the 
> expense of having smaller calls with larger payloads.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Reply via email to