----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9288/ -----------------------------------------------------------
(Updated Feb. 4, 2013, 2:22 p.m.) Review request for hama. Summary (updated) ----------------- [HAMA-721] Patch for Combiner implementation on spilling queue. Fixes for performance in spilling queue and added buffers for synchronous spills and bytebuffer streams to reuse ByteBuffer. Description ------- The combiner is implemented in CombineSpilledDataProcessor. Apart from this, the patch contains synchronous spilling buffers. Also due to an oversight, reads from spilling buffer was expensive both in memory and CPU. It is now faster and uses less resources than previous version. Diffs ----- http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/Constants.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/BufferedReadStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ByteBufferInputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ByteBufferOutputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/CombineSpilledDataProcessor.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DirectByteBufferInputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DirectByteBufferOutputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DualChannelByteBufferStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DuplexByteArrayChannel.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/PreFetchCache.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ReadIndexStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ReusableByteBuffer.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpillWriteIndexStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledByteBuffer.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledDataInputBuffer.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledDataProcessor.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledDataReadStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpillingDataOutputBuffer.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SyncFlushByteBufferOutputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SyncReadByteBufferInputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/WriteSpilledDataProcessor.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/DiskQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MemoryQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MessageQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SortedMessageQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SpillingQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/test/java/org/apache/hama/bsp/message/TestMessageIO.java 1441692 Diff: https://reviews.apache.org/r/9288/diff/ Testing ------- Added unit test cases in TestMessageIO and passes mvn install. Thanks, Suraj Menon
