If we use Spilling or Disk queues for outgoing queue, some memory consumption can be saved but we can't avoid deserializing and bundling messages overhead. According to my test, disk-based outgoing queue doesn't help much in memory usage.
So, in my opinion, we should add outgoing messages in bundle object directly, and should consider about byte array bundle, or disk-based bundle. We can add something like OutgoingMessageManager. Does it make sense? On Wed, Jan 15, 2014 at 2:32 PM, Edward J. Yoon <edwardy...@apache.org> wrote: > As I described here, > http://wiki.apache.org/hama/Architecture#Communication_and_Synchronization_Process > > Currently, we puts the messages to be sent into outgoing queue. Then > we puts the messages of outgoing queue into BSPMessageBundle object > again. > > This is huge overhead, and very inefficient in memory usage. Why > should we do like that? > > Please let me know if i missing something important. > > -- > Best Regards, Edward J. Yoon > @eddieyoon -- Best Regards, Edward J. Yoon @eddieyoon
