[jira] [Commented] (GIRAPH-37) Implement Netty-backed rpc solution
[ https://issues.apache.org/jira/browse/GIRAPH-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269304#comment-13269304 ] Bo Wang commented on GIRAPH-37: --- This is really good news. RPC doesn't seems very scalable. Look forward to the Netty implementation. > Implement Netty-backed rpc solution > --- > > Key: GIRAPH-37 > URL: https://issues.apache.org/jira/browse/GIRAPH-37 > Project: Giraph > Issue Type: New Feature >Reporter: Jakob Homan >Assignee: Jakob Homan > Attachments: GIRAPH-37-wip.patch > > > GIRAPH-12 considered replacing the current Hadoop based rpc method with > Netty, but didn't went in another direction. I think there is still value in > this approach, and will also look at Finagle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated GIRAPH-185: --- Attachment: (was: GIRAPH-185.patch) > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch, GIRAPH-185.patch, GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated GIRAPH-185: --- Attachment: GIRAPH-185.patch Sorry for replying late. I've been pretty busy last week with midterms and deadlines. I did a few tests on a 24-core 32GB DRAM machine. Quite astonishing, the performance of the new version (ConcurrentHashMap + ConcurrentLinkedQueue) is kind of slower than the original version. I ran the PageRankBenchmark with "-e 100 -s 10 -V 1". Results are as following. (original) #cores superstep.exec.time 12 11580 8 10651 4 12944 2 19334 1 20782 (ConcurrentHashMap + ConcurrentLinkedQueue) #cores superstep.exec.time 12 11639 8 11527 4 13653 2 20354 1 27825 I think the overhead came from the lock/unlock on the get(), especially when reading messages sent in the previous stage where no lock is needed since it's sequential. To verify my assumption, I changed the ConcurrentLinkedQueue back to ArrayList and wrap it with synchronized, then the performance does improve. (ConcurrentHashMap) #cores superstep.exec.time 12 11380 8 11618 4 12834 2 18417 1 22952 In comparison, it's more or less the same as the original version. Then I ran with another set of parameters "-e 1000 -s 10 -V 1000". It seems the fine grain lock does help. (original) #cores superstep.exec.time 12 97101 6 11321 (ConcurrentHashMap) #cores superstep.exec.time 12 92848 (4.4%) 6 10834 (4.3%) I attached the new patch (ConcurrentHashMap version) for the review. Claudio, thanks for offering to help. You may checkout a clean version and apply the patch ($ patch -p0 -i GIRAPH-185.patch) to test it on the cluster. > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch, GIRAPH-185.patch, GIRAPH-185.patch, > GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated GIRAPH-185: --- Attachment: GIRAPH-185.patch > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch, GIRAPH-185.patch, GIRAPH-185.patch, > GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262709#comment-13262709 ] Bo Wang commented on GIRAPH-185: I did a small test (2 machines, each with 2 cores) and the performance change is relatively small. I am now scheduling a time to run it on a larger environment so that there are more conflicts. There are always too many concurrent jobs on the cluster and I need to wait for a time slot. Sorry for having your waiting. > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch, GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261830#comment-13261830 ] Bo Wang commented on GIRAPH-185: I checked the source and found the same thing. I think LinkedList should be ok in terms of space. ArrayList also has to keep empty space in the array to future insertion. Should we close this issue? > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch, GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: Improve concurrency of putMsg / putMsgList
> On 2012-04-24 20:53:33, Avery Ching wrote: > > http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java, > > lines 776-777 > > <https://reviews.apache.org/r/4852/diff/1/?file=104059#file104059line776> > > > > Bo, I'm a little leery about converting the List and ArrayList to > > LinkedList and ConcurrentLinkedList. I believe that linked list's will use > > more memory than the array list due to the double links (forward and > > backward). Also, is ConcurrentLinkedList supposted to outperform a > > synchronized ArrayList? I haven't seen much on that. > > > > The concurrenthashmap changes look good. Avery, thanks for the comments. I just measured the sizes of these classes and below are an estimation. java.util.ArrayList: 149 bytes java.util.LinkedList: 101 bytes java.util.concurrent.ConcurrentLinkedQueue: 118 bytes The tool I was using is a program from the link below. http://www.javapractices.com/topic/TopicAction.do?Id=83 In terms of performance, here is a benchmark. http://www.javacodegeeks.com/2010/09/java-best-practices-queue-battle-and.html In its test #1 (adding element), ConcurrentLinkedQueue performed slightly better than LinkedList. In test #3 (iterator), LinkedList outperformed ConcurrentLinkedQueue. I think the most time consuming part is add, while iteration is also heavily used but no concurrent accesses. - Bo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4852/#review7185 --- On 2012-04-24 06:11:38, Bo Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/4852/ > --- > > (Updated 2012-04-24 06:11:38) > > > Review request for giraph. > > > Summary > --- > > Use ConcurrentHashMap and ConcurrentLinkedQueue to allow concurrent assess to > message map. The concurrencyLevel of ConcurrentHashMap uses the default > value. There may be some performance gain by tuning this value. > > > This addresses bug GIRAPH-185. > https://issues.apache.org/jira/browse/GIRAPH-185 > > > Diffs > - > > > http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java > 1328747 > > Diff: https://reviews.apache.org/r/4852/diff > > > Testing > --- > > > Thanks, > > Bo > >
[jira] [Created] (GIRAPH-188) Merge inMessages and vertex message list
Bo Wang created GIRAPH-188: -- Summary: Merge inMessages and vertex message list Key: GIRAPH-188 URL: https://issues.apache.org/jira/browse/GIRAPH-188 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.2.0 Reporter: Bo Wang Assignee: Bo Wang Fix For: 0.2.0 Currently received messages will firstly be stored in transientInMessages and then merged and moved to inMessages before finally copied to vertex message list. The copy from inMessages to vertex message list can be avoided to save time and space. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260749#comment-13260749 ] Bo Wang commented on GIRAPH-185: Thanks for looking at this, Claudio. I may do some perf tests and post the results. I also found inMessages and vertex message list can be merged to save memory and time. I will create a Jira and work on it. > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch, GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: Improve concurrency of putMsg / putMsgList
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4852/ --- Review request for giraph. Summary --- Use ConcurrentHashMap and ConcurrentLinkedQueue to allow concurrent assess to message map. The concurrencyLevel of ConcurrentHashMap uses the default value. There may be some performance gain by tuning this value. This addresses bug GIRAPH-185. https://issues.apache.org/jira/browse/GIRAPH-185 Diffs - http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1328747 Diff: https://reviews.apache.org/r/4852/diff Testing --- Thanks, Bo
[jira] [Updated] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated GIRAPH-185: --- Attachment: GIRAPH-185.patch Thanks for keeping up in the communication, Avery. I used putIfAbsent in the new version (thanks Claudio for the suggestion). Here comes the new patch. > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch, GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
Thanks for the opinion, Avery. Yep, the space for ConcurrenHashMap is not a concern. I estimate the space overhead for those empty entries is not too large though. An empty entry is around 69 bytes, just the size of a couple of messages. Statistically speaking, most vertices will receive one or more messages, for example, in PageRank. Actually, each Vertex object also has an internal messageList structure of the same size, whether it receives a message or not. With pre-population the time for entry creation and insertion can be saved as well as the time spent on garbage collection. Do you think it's worth the trade-off? If not, I am pretty open to using ConcurrentHashMap. Thanks, Bo On Mon, Apr 23, 2012 at 12:08 PM, Avery Ching (JIRA) wrote: > >[ > https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259841#comment-13259841] > > Avery Ching commented on GIRAPH-185: > > > Since we only allocate one ConcurrentHashMap per worker, the empty > overhead isn't a concern. If however, the per element memory cost of a > entry into the concurrent hash map is much more expensive then I would > definitely be worried. We can also tune the concurrency level (default 16) > to a reasonable tradeoff. > > > Improve concurrency of putMsg / putMsgList > > -- > > > > Key: GIRAPH-185 > > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > > Project: Giraph > > Issue Type: Improvement > > Components: graph > >Affects Versions: 0.2.0 > >Reporter: Bo Wang > >Assignee: Bo Wang > > Fix For: 0.2.0 > > > > Attachments: GIRAPH-185.patch > > > > Original Estimate: 2h > > Remaining Estimate: 2h > > > > Currently in putMsg / putMsgList, a synchronized closure is used to > protect the whole transientInMessages when adding the new message. This > lock prevents other concurrent calls to putMsg/putMsgList and increases the > response time. We should use fine-grain locks to allow high concurrency in > message communication. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators: > https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa > For more information on JIRA, see: http://www.atlassian.com/software/jira > > >
[jira] [Commented] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259798#comment-13259798 ] Bo Wang commented on GIRAPH-185: Thanks for the comments, Avery and Claudio. I am measuring the performance, but I found the #Sent messages (reported by counter) is always zero in both the original and changed version. Is that a bug? In the output of Quick Start Guide, it is also zero. I think ConcurrentHashMap is a good way to go. But for the current approach, I think the memory overhead won't be much since most of the vertices will receive messages except for those isolated ones and the one whose neighbors are all inactive. In comparison, ConcurrentHashMap is much larger than HashMap. An empty ConcurrentHashMap takes around 1673 bytes while an empty HashMap only takes 137 bytes. Another reason to use pre-population is to avoid the time spent on allocating new message lists and inserting them into the map. We just need to clear the list before the next super-step. What's your opinion? Thanks, Bo > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 > Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: Change coarse-grain lock to fine-grain lock by pre-populating the message map so that each entry for a vertex can be locked individually and the hazard condition is also avoided.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4840/ --- Review request for giraph. Summary --- Improve concurrency of putMsg / putMsgList. This addresses bug GIRAPH-185. https://issues.apache.org/jira/browse/GIRAPH-185 Diffs - http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1328747 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/WorkerCommunications.java 1328747 http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 1328747 Diff: https://reviews.apache.org/r/4840/diff Testing --- All test passed. Wait for review. Thanks. Thanks, Bo
[jira] [Updated] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Wang updated GIRAPH-185: --- Attachment: GIRAPH-185.patch Fixed the problem and wait for review. > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 0.2.0 > > Attachments: GIRAPH-185.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Assigned] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
Thanks for the comments, Claudio and Avery. I will work on it. On Tue, Apr 17, 2012 at 2:35 PM, Avery Ching (Assigned) (JIRA) < j...@apache.org> wrote: > > [ > https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel] > > Avery Ching reassigned GIRAPH-185: > ------ > >Assignee: Bo Wang > > > Improve concurrency of putMsg / putMsgList > > -- > > > > Key: GIRAPH-185 > > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > > Project: Giraph > > Issue Type: Improvement > > Components: graph > > Affects Versions: 0.2.0 > >Reporter: Bo Wang > >Assignee: Bo Wang > > Fix For: 0.2.0 > > > > Original Estimate: 2h > > Remaining Estimate: 2h > > > > Currently in putMsg / putMsgList, a synchronized closure is used to > protect the whole transientInMessages when adding the new message. This > lock prevents other concurrent calls to putMsg/putMsgList and increases the > response time. We should use fine-grain locks to allow high concurrency in > message communication. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators: > https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa > For more information on JIRA, see: http://www.atlassian.com/software/jira > > >
[jira] [Created] (GIRAPH-186) Improve concurrency of putVertexList
Improve concurrency of putVertexList Key: GIRAPH-186 URL: https://issues.apache.org/jira/browse/GIRAPH-186 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.2.0 Reporter: Bo Wang Fix For: 0.2.0 It's pretty similar to GIRAPH-185. The whole inPartitionVertexMap is locked when there is a call to it. We should allow multiple calls adding different partitions to the same worker at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
[ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255933#comment-13255933 ] Bo Wang commented on GIRAPH-185: putVertexIdMessagesList can be improved together > Improve concurrency of putMsg / putMsgList > -- > > Key: GIRAPH-185 > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.2.0 >Reporter: Bo Wang > Fix For: 0.2.0 > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently in putMsg / putMsgList, a synchronized closure is used to protect > the whole transientInMessages when adding the new message. This lock prevents > other concurrent calls to putMsg/putMsgList and increases the response time. > We should use fine-grain locks to allow high concurrency in message > communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
Improve concurrency of putMsg / putMsgList -- Key: GIRAPH-185 URL: https://issues.apache.org/jira/browse/GIRAPH-185 Project: Giraph Issue Type: Improvement Components: graph Affects Versions: 0.2.0 Reporter: Bo Wang Fix For: 0.2.0 Currently in putMsg / putMsgList, a synchronized closure is used to protect the whole transientInMessages when adding the new message. This lock prevents other concurrent calls to putMsg/putMsgList and increases the response time. We should use fine-grain locks to allow high concurrency in message communication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
