----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21826/#review43764 -----------------------------------------------------------
Have you done any benchmarks with these changes? I would think that you should see higher CPU usage with less disk I/O, and, overall, quicker ingest. It would be good to see some before and after numbers that show this. core/src/main/java/org/apache/accumulo/core/conf/Property.java <https://reviews.apache.org/r/21826/#comment78049> Should this be tagged with the @Experimental annotation? server/tserver/src/main/java/org/apache/accumulo/tserver/log/DecompressingInputStream.java <https://reviews.apache.org/r/21826/#comment78054> Same comment as elsewhere, can we use what Hadoop has for compression to abstract out the specifics of Snappy? server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java <https://reviews.apache.org/r/21826/#comment78051> Could we leverage the Compression stuff in Hadoop rather than invoke the snappy lib directly? server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java <https://reviews.apache.org/r/21826/#comment78058> It looks like this sync invocation got dropped. Did I just miss it moving somewhere? - Josh Elser On May 22, 2014, 7:57 p.m., John McNamee wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/21826/ > ----------------------------------------------------------- > > (Updated May 22, 2014, 7:57 p.m.) > > > Review request for accumulo. > > > Repository: accumulo > > > Description > ------- > > The goal was to find a way to increase the write speed in the DfsLogger using > compression. > This patch uses the Snappy compression library and was the most optimum way > that I could find to increase write performance. > The option to use compression has been made configurable. > Recovery has been modified to uncompress the WAL if needed. > > > Diffs > ----- > > core/src/main/java/org/apache/accumulo/core/conf/Property.java 62b0a33 > > server/tserver/src/main/java/org/apache/accumulo/tserver/log/DecompressingInputStream.java > PRE-CREATION > server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java > bfa51d3 > > server/tserver/src/test/java/org/apache/accumulo/tserver/log/DecompressingInputStreamTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/21826/diff/ > > > Testing > ------- > > There is a unit test for the DecryptingInputStream.java, and I ran continuous > Ingest/ continuous verify with the agitator on. > > > Thanks, > > John McNamee > >
