Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 2:24 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Add explanatory comment. Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs (updated) - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 2:24 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Add bug link. Bugs: AURORA-722 https://issues.apache.org/jira/browse/AURORA-722 Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 3:12 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Update Testing Done to reflect current code Bugs: AURORA-722 https://issues.apache.org/jira/browse/AURORA-722 Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs (updated) - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing (updated) --- ./gradlew build Microbenchmark with a scheduler backup from my production environment. Before: ## Deflater.DEFAULT_COMPRESSION, (No BufferedOutputStream), DeflaterOutputStream( deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% After: # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream(256KiB), DeflaterOutputStream(256KiB) deflate: 11807ms (4857ms speedup) total: 12379ms (4840ms speedup) compression ratio: 647270930/92690409 = 698.31% (0% improvement) Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 3:21 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- Fix truncated line Bugs: AURORA-722 https://issues.apache.org/jira/browse/AURORA-722 Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing (updated) --- ./gradlew build Microbenchmark with a scheduler backup from my production environment. Before: ## Deflater.DEFAULT_COMPRESSION, (No BufferedOutputStream), DeflaterOutputStream(512B) deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% After: # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream(256KiB), DeflaterOutputStream(256KiB) deflate: 11807ms (4857ms speedup) total: 12379ms (4840ms speedup) compression ratio: 647270930/92690409 = 698.31% (0% improvement) Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 24, 2014, 6:36 p.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Changes --- +jsmith Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/#review54492 --- Ship it! Ship It! - Maxim Khutornenko On Sept. 25, 2014, 1:36 a.m., Kevin Sweeney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 1:36 a.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/#review54494 --- Ship it! Ship It! - David McLaughlin On Sept. 25, 2014, 1:36 a.m., Kevin Sweeney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 1:36 a.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
Re: Review Request 26012: Use BufferedOutputStream for deflater
On Sept. 25, 2014, 2:23 a.m., Bill Farner wrote: Ship It! Actually - can you include a comment with a very brief explanation of the findings? Wouldn't want somebody to remove the seemingly-unnecessary stream wrapper. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/#review54500 --- On Sept. 25, 2014, 1:36 a.m., Kevin Sweeney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26012/ --- (Updated Sept. 25, 2014, 1:36 a.m.) Review request for Aurora, David McLaughlin, Joe Smith, and Bill Farner. Repository: aurora Description --- Use BufferedOutputStream for deflater Diffs - src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 45cf7ecf52582ba3fd7bbc8b4a981e396793da4b Diff: https://reviews.apache.org/r/26012/diff/ Testing --- ./gradlew build Also put together a microbenchmark with a scheduler backup from my production environment - using a BufferedOutputStream gets a 4sec speedup with no reduction in compression ratio. ¯_(?)_/¯ # Deflater.BEST_SPEED, No buffer deflate: 11745ms total:12734ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, buf = 512KiB deflate: 11015ms total:12001ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream deflate: 6885ms total:7694ms compression ratio: 647270930/121616834 = 532.22% # Deflater.BEST_SPEED, BufferedOutputStream at 512KiB deflate: 6752ms total:7585ms compression ratio: 647270930/121616834 = 532.22% # Deflater.DEFAULT_COMPRESSION deflate: 16664ms total:17219ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, buf size = 512KiB deflate: 16516ms total:17103ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream deflate: 12241ms total:12788ms compression ratio: 647270930/92690409 = 698.31% # Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB deflate: 11548ms total:12108ms compression ratio: 647270930/92690409 = 698.31% Thanks, Kevin Sweeney
