> On May 22, 2014, 4:28 p.m., Josh Elser wrote: > > core/src/main/java/org/apache/accumulo/core/conf/Property.java, line 270 > > <https://reviews.apache.org/r/21826/diff/1/?file=588464#file588464line270> > > > > Should this be tagged with the @Experimental annotation?
I view @Experimental to denote features that are more invasive, optional, and may not work as expected, not simply to denote things that users can experiment with. @Experimental hides these configuration items from a typical user's view, making the target audience developers, rather than end users. While it is currently experimental as a patch, I think the expectation here is that when this feature is ready for merging into a branch, the feature will be targeted to end users and made available for them to tweak, so @Experimental doesn't quite apply. > On May 22, 2014, 4:28 p.m., Josh Elser wrote: > > server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java, > > line 241 > > <https://reviews.apache.org/r/21826/diff/1/?file=588466#file588466line241> > > > > Could we leverage the Compression stuff in Hadoop rather than invoke > > the snappy lib directly? I agree, this feature could be improved by allowing more compression types in the configuration, rather than simply enable/disable, and that should probably be done before merging in. However, there are special considerations with other compression types. Snappy allows you to take a byte array and make another byte array. The other types work with streams, and you'd have to ensure the stream is flushed before sync is called. Snappy's byte[]-to-byte[] is a bit easier to experiment with, as a first pass. - Christopher ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21826/#review43764 ----------------------------------------------------------- On May 22, 2014, 3:57 p.m., John McNamee wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/21826/ > ----------------------------------------------------------- > > (Updated May 22, 2014, 3:57 p.m.) > > > Review request for accumulo. > > > Repository: accumulo > > > Description > ------- > > The goal was to find a way to increase the write speed in the DfsLogger using > compression. > This patch uses the Snappy compression library and was the most optimum way > that I could find to increase write performance. > The option to use compression has been made configurable. > Recovery has been modified to uncompress the WAL if needed. > > > Diffs > ----- > > core/src/main/java/org/apache/accumulo/core/conf/Property.java 62b0a33 > > server/tserver/src/main/java/org/apache/accumulo/tserver/log/DecompressingInputStream.java > PRE-CREATION > server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java > bfa51d3 > > server/tserver/src/test/java/org/apache/accumulo/tserver/log/DecompressingInputStreamTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/21826/diff/ > > > Testing > ------- > > There is a unit test for the DecryptingInputStream.java, and I ran continuous > Ingest/ continuous verify with the agitator on. > > > Thanks, > > John McNamee > >
