sounds cool, we can adhere to just the queue implementation in HAMA-559 @Suraj I do it a little different in my buffer, instead of keeping a byte[] array in memory. I allocate space offheap for every new msg added to an in-memory queue. This in-memory backingQueue just keeps reference to the space allocated offheap from where byte[] repr of Writables can be obtained and spilled to disk. (on exceeding some threshold).
The Reading part I am still developing. I didn't exactly understand by what you meant by keeping a byte array service. I don't think I require this in my design. I think you are referring to your design where this byte array service will pick the relevant portion from the in-memory byte array and read, write stuff into some Writable. Am I correct? Also in my design, the threshold is some number for now. Didn't really think of putting some additional library to measure my in-memory backingQueue size. Does this appear ugly design-wise? -- Regards, Apurv Verma On Wed, Oct 17, 2012 at 10:46 PM, Thomas Jungblut (JIRA) <[email protected]>wrote: > > [ > https://issues.apache.org/jira/browse/HAMA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478047#comment-13478047] > > Thomas Jungblut commented on HAMA-644: > -------------------------------------- > > Let's keep the queue implementation in HAMA-559, this issue will improve > the abstraction arround. > > > Messaging Scalability Improvement > > ---------------------------------- > > > > Key: HAMA-644 > > URL: https://issues.apache.org/jira/browse/HAMA-644 > > Project: Hama > > Issue Type: Improvement > > Components: messaging > > Affects Versions: 0.5.0 > > Reporter: Thomas Jungblut > > Fix For: 0.6.0 > > > > Attachments: HAMA-644_1.patch, HAMA-644_2.patch, HAMA-644_3.patch > > > > > > more details follow soon > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira >
