Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 3:21 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Fix truncated line Bugs: AURORA-722 https://issues.apache.org/jira/browse/AURORA-722 Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing (updated) --- ./gradlew build Microbenchmark with a scheduler backup from my production environment. Before: ## Deflater.DEFAULT_COMPRESSION, (No BufferedOutputStream), DeflaterOutputStream(512B) deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% After: # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream(256KiB), DeflaterOutputStream(256KiB) deflate: 11807ms (4857ms speedup) total: 12379ms (4840ms speedup) compression ratio: 647270930/92690409 = 698.31% (0% improvement) Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 3:12 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Update Testing Done to reflect current code Bugs: AURORA-722 https://issues.apache.org/jira/browse/AURORA-722 Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs (updated) - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing (updated) --- ./gradlew build Microbenchmark with a scheduler backup from my production environment. Before: ## Deflater.DEFAULT_COMPRESSION, (No BufferedOutputStream), DeflaterOutputStream( deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% After: # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream(256KiB), DeflaterOutputStream(256KiB) deflate: 11807ms (4857ms speedup) total: 12379ms (4840ms speedup) compression ratio: 647270930/92690409 = 698.31% (0% improvement) Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 2:24 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Add bug link. Bugs: AURORA-722 https://issues.apache.org/jira/browse/AURORA-722 Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 2:24 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Add explanatory comment. Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs (updated) - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
> On Sept. 25, 2014, 2:23 a.m., Bill Farner wrote: > > Ship It! Actually - can you include a comment with a very brief explanation of the findings? Wouldn't want somebody to remove the seemingly-unnecessary stream wrapper. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/#review54500 --- On Sept. 25, 2014, 1:36 a.m., Kevin Sweeney wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26012/ > --- > > (Updated Sept. 25, 2014, 1:36 a.m.) > > > Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. > > > Repository: aurora > > > Description > --- > > Use BufferedOutputStream for deflater > > > Diffs > - > > src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java > 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b > > Diff: https://reviews.apache.org/r/26012/diff/ > > > Testing > --- > > ./gradlew build > > Also put together a microbenchmark with a scheduler backup from my production > environment - using a BufferedOutputStream gets a 4sec speedup with no > reduction in compression ratio. ¯_(?)_/¯ > > # Deflater.BEST_SPEED, No buffer > deflate: 11745ms > total:12734ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, buf = 512KiB > deflate: 11015ms > total:12001ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream > deflate: 6885ms > total:7694ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB > deflate: 6752ms > total:7585ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.DEFAULT_COMPRESSION > deflate: 16664ms > total:17219ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB > deflate: 16516ms > total:17103ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream > deflate: 12241ms > total:12788ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB > deflate: 11548ms > total:12108ms > compression ratio: 647270930/92690409 = 698.31% > > > Thanks, > > Kevin Sweeney > >
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/#review54500 --- Ship it! Ship It! - Bill Farner On Sept. 25, 2014, 1:36 a.m., Kevin Sweeney wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26012/ > --- > > (Updated Sept. 25, 2014, 1:36 a.m.) > > > Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. > > > Repository: aurora > > > Description > --- > > Use BufferedOutputStream for deflater > > > Diffs > - > > src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java > 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b > > Diff: https://reviews.apache.org/r/26012/diff/ > > > Testing > --- > > ./gradlew build > > Also put together a microbenchmark with a scheduler backup from my production > environment - using a BufferedOutputStream gets a 4sec speedup with no > reduction in compression ratio. ¯_(?)_/¯ > > # Deflater.BEST_SPEED, No buffer > deflate: 11745ms > total:12734ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, buf = 512KiB > deflate: 11015ms > total:12001ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream > deflate: 6885ms > total:7694ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB > deflate: 6752ms > total:7585ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.DEFAULT_COMPRESSION > deflate: 16664ms > total:17219ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB > deflate: 16516ms > total:17103ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream > deflate: 12241ms > total:12788ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB > deflate: 11548ms > total:12108ms > compression ratio: 647270930/92690409 = 698.31% > > > Thanks, > > Kevin Sweeney > >
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/#review54494 --- Ship it! Ship It! - David McLaughlin On Sept. 25, 2014, 1:36 a.m., Kevin Sweeney wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26012/ > --- > > (Updated Sept. 25, 2014, 1:36 a.m.) > > > Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. > > > Repository: aurora > > > Description > --- > > Use BufferedOutputStream for deflater > > > Diffs > - > > src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java > 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b > > Diff: https://reviews.apache.org/r/26012/diff/ > > > Testing > --- > > ./gradlew build > > Also put together a microbenchmark with a scheduler backup from my production > environment - using a BufferedOutputStream gets a 4sec speedup with no > reduction in compression ratio. ¯_(?)_/¯ > > # Deflater.BEST_SPEED, No buffer > deflate: 11745ms > total:12734ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, buf = 512KiB > deflate: 11015ms > total:12001ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream > deflate: 6885ms > total:7694ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB > deflate: 6752ms > total:7585ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.DEFAULT_COMPRESSION > deflate: 16664ms > total:17219ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB > deflate: 16516ms > total:17103ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream > deflate: 12241ms > total:12788ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB > deflate: 11548ms > total:12108ms > compression ratio: 647270930/92690409 = 698.31% > > > Thanks, > > Kevin Sweeney > >
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/#review54492 --- Ship it! Ship It! - Maxim Khutornenko On Sept. 25, 2014, 1:36 a.m., Kevin Sweeney wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26012/ > --- > > (Updated Sept. 25, 2014, 1:36 a.m.) > > > Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. > > > Repository: aurora > > > Description > --- > > Use BufferedOutputStream for deflater > > > Diffs > - > > src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java > 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b > > Diff: https://reviews.apache.org/r/26012/diff/ > > > Testing > --- > > ./gradlew build > > Also put together a microbenchmark with a scheduler backup from my production > environment - using a BufferedOutputStream gets a 4sec speedup with no > reduction in compression ratio. ¯_(?)_/¯ > > # Deflater.BEST_SPEED, No buffer > deflate: 11745ms > total:12734ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, buf = 512KiB > deflate: 11015ms > total:12001ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream > deflate: 6885ms > total:7694ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB > deflate: 6752ms > total:7585ms > compression ratio: 647270930/121616834 = 532.22% > > # Deflater.DEFAULT_COMPRESSION > deflate: 16664ms > total:17219ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB > deflate: 16516ms > total:17103ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream > deflate: 12241ms > total:12788ms > compression ratio: 647270930/92690409 = 698.31% > > # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB > deflate: 11548ms > total:12108ms > compression ratio: 647270930/92690409 = 698.31% > > > Thanks, > > Kevin Sweeney > >
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 24, 2014, 6:36 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- +jsmith Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- Review request for Aurora, David McLaughlin and Bill Farner. Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
