----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9288/ -----------------------------------------------------------
Review request for hama. Description ------- The combiner is implemented in CombineSpilledDataProcessor. Apart from this, the patch contains synchronous spilling buffers. Also due to an oversight, reads from spilling buffer was expensive both in memory and CPU. It is now faster and uses less resources than previous version. Diffs ----- http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/Constants.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/BufferedReadStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ByteBufferInputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ByteBufferOutputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/CombineSpilledDataProcessor.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DirectByteBufferInputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DirectByteBufferOutputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DualChannelByteBufferStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/DuplexByteArrayChannel.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/PreFetchCache.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ReadIndexStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/ReusableByteBuffer.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpillWriteIndexStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledByteBuffer.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledDataInputBuffer.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledDataProcessor.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpilledDataReadStatus.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SpillingDataOutputBuffer.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SyncFlushByteBufferOutputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/SyncReadByteBufferInputStream.java PRE-CREATION http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/io/WriteSpilledDataProcessor.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/DiskQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MemoryQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MessageQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SortedMessageQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SpillingQueue.java 1441692 http://svn.apache.org/repos/asf/hama/trunk/core/src/test/java/org/apache/hama/bsp/message/TestMessageIO.java 1441692 Diff: https://reviews.apache.org/r/9288/diff/ Testing ------- Added unit test cases in TestMessageIO and passes mvn install. Thanks, Suraj Menon
