[
https://issues.apache.org/jira/browse/IO-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brett Lounsbury resolved IO-650.
--------------------------------
Fix Version/s: (was: 2.6)
2.7
Resolution: Fixed
[~ggregory] resolved this issue as part of IO-666
> Improve IOUtils performance by increasing DEFAULT_BUFFER_SIZE
> -------------------------------------------------------------
>
> Key: IO-650
> URL: https://issues.apache.org/jira/browse/IO-650
> Project: Commons IO
> Issue Type: Improvement
> Affects Versions: 1.0
> Reporter: Brett Lounsbury
> Priority: Major
> Fix For: 2.7
>
>
> IOUtils has a 4096B default buffer size that is used by copy() methods (and
> will be used for contentEquals methods when IO-649 is pulled). This number
> should be updated to 8192 for a few reasons:
> # It has a big improvement in performance in my micro-benchmark. I tested
> both copy() and contentEquals() with 4K and 8K buffers. This was done on a
> Late 2019 Macbook Pro (2.8GHz i7) with a 128MB file loaded into the OS buffer
> cache. See below for the test harness used. Past 8K performance does
> improve but it begins to experience some diminishing returns and could lead
> to excessive memory allocation.
> # It mirrors the default buffer size of java.io.Buffered* classes. This
> makes buffer sizing consistent regardless of if it is being done internally
> in the method or externally via a Buffered*. These classes are used
> internally in IOUtils as well so the buffer size is not unreasonable.
>
> For copy():
> |*Metric*|*8K Buffer Millisecond*|*4K Buffer Millisecond*|*Performance
> Improvement*|
> |*AVG*|44.2853417|64.2600679|0.31084197|
> |*P50*|42.692406|62.371984|0.31551951|
> |*P90*|49.5538826|68.4303876|0.27584975|
> |*P99*|62.8831473|89.759114|0.29942326|
> |*P100*|102.563615|177.143364|0.42101351|
>
> For contentEquals() with IO-649:
> |*Metric*|*8K Buffer Millisecond*|*4K Buffer Millisecond*|*Performance
> Improvement*|
> |*AVG*|81.5009567|128.497828|0.36574059|
> |*P50*|78.517749|124.191476|0.36776861|
> |*P90*|89.9172708|136.779763|0.34261276|
> |*P99*|125.814333|183.881989|0.31578762|
> |*P100*|308.936585|559.611217|0.44794426|
> {color:#000080}public static void {color}main(String[] args)
> {color:#000080}throws {color}Exception {
> NullOutputStream nos =
> NullOutputStream.{color:#660e7a}NULL_OUTPUT_STREAM{color};
> {color:#000080}for {color}({color:#000080}int {color}i =
> {color:#0000ff}0{color}; i < {color:#0000ff}1000{color}; i++) {
> InputStream fis = {color:#000080}new
> {color}FileInputStream({color:#000080}new
> {color}File({color:#008000}"/tmp/random_data"{color}));
> {color:#000080}long {color}start = System.nanoTime();
> IOUtils.copy(fis, nos);
> {color:#000080}long {color}defaultCopyTime = System.nanoTime() -
> start;
> fis = {color:#000080}new {color}FileInputStream({color:#000080}new
> {color}File({color:#008000}"/tmp/random_data"{color}));
> start = System.nanoTime();
> IOUtils.copy(fis, nos, {color:#0000ff}8192{color});
> {color:#000080}long {color}bufferSizeSpecifiedCopyTime =
> System.nanoTime() - start;
> System.{color:#660e7a}out{color}.println(bufferSizeSpecifiedCopyTime
> + {color:#008000}"{color}{color:#000080}\t{color}{color:#008000}" {color}+
> defaultCopyTime);
> }
> }
> {color:#000080}public static void {color}main(String[] args)
> {color:#000080}throws {color}Exception {
> {color:#000080} for {color}({color:#000080}int {color}i =
> {color:#0000ff}0{color}; i < {color:#0000ff}1000{color}; i++) {
> InputStream fis = {color:#000080}new
> {color}FileInputStream({color:#000080}new
> {color}File({color:#008000}"/tmp/random_data"{color}));
> InputStream fis2 = {color:#000080}new
> {color}FileInputStream({color:#000080}new
> {color}File({color:#008000}"/tmp/random_data"{color}));
> {color:#000080}long {color}start = System.nanoTime();
> IOUtils.contentEquals(fis, fis2);
> {color:#000080}long {color}defaultContentEqualsTime =
> System.nanoTime() - start;
> fis = {color:#000080}new {color}FileInputStream({color:#000080}new
> {color}File({color:#008000}"/tmp/random_data"{color}));
> fis2 = {color:#000080}new {color}FileInputStream({color:#000080}new
> {color}File({color:#008000}"/tmp/random_data"{color}));
> start = System.nanoTime();
> IOUtils.contentEquals(fis, fis2, {color:#0000ff}8192{color});
> {color:#000080}long {color}bufferSizeSpecifiedContentEqualsTime =
> System.nanoTime() - start;
>
> System.{color:#660e7a}out{color}.println(bufferSizeSpecifiedContentEqualsTime
> + {color:#008000}"{color}{color:#000080}\t{color}{color:#008000}" {color}+
> defaultContentEqualsTime);
> }
> }
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
