Thanks for the opinion, Avery. Yep, the space for ConcurrenHashMap is not a concern. I estimate the space overhead for those empty entries is not too large though. An empty entry is around 69 bytes, just the size of a couple of messages. Statistically speaking, most vertices will receive one or more messages, for example, in PageRank. Actually, each Vertex object also has an internal messageList structure of the same size, whether it receives a message or not. With pre-population the time for entry creation and insertion can be saved as well as the time spent on garbage collection.
Do you think it's worth the trade-off? If not, I am pretty open to using ConcurrentHashMap. Thanks, Bo On Mon, Apr 23, 2012 at 12:08 PM, Avery Ching (JIRA) <j...@apache.org>wrote: > > [ > https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259841#comment-13259841] > > Avery Ching commented on GIRAPH-185: > ------------------------------------ > > Since we only allocate one ConcurrentHashMap per worker, the empty > overhead isn't a concern. If however, the per element memory cost of a > entry into the concurrent hash map is much more expensive then I would > definitely be worried. We can also tune the concurrency level (default 16) > to a reasonable tradeoff. > > > Improve concurrency of putMsg / putMsgList > > ------------------------------------------ > > > > Key: GIRAPH-185 > > URL: https://issues.apache.org/jira/browse/GIRAPH-185 > > Project: Giraph > > Issue Type: Improvement > > Components: graph > > Affects Versions: 0.2.0 > > Reporter: Bo Wang > > Assignee: Bo Wang > > Fix For: 0.2.0 > > > > Attachments: GIRAPH-185.patch > > > > Original Estimate: 2h > > Remaining Estimate: 2h > > > > Currently in putMsg / putMsgList, a synchronized closure is used to > protect the whole transientInMessages when adding the new message. This > lock prevents other concurrent calls to putMsg/putMsgList and increases the > response time. We should use fine-grain locks to allow high concurrency in > message communication. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators: > https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa > For more information on JIRA, see: http://www.atlassian.com/software/jira > > >
