Martin, Alan

Please help review the change for #6975829

http://cr.openjdk.java.net/~sherman/6975829/webrev/ <http://cr.openjdk.java.net/%7Esherman/6975829/webrev/>

This is again a problem triggered by the change we putback years ago for
6206933: GZipOutputStream/InputStream goes critical(calls JNI_Get*Critical) and causes slowness

in which we switch from using the JNI_Get*Critical to access the input bits
to using GetByteArrayregion(...) (which invloves array copy)

The test case attached in the bug report exaggeratedly use file size of the
.gz file as the buf size param for the GZIPInputStream() constructor.

        File f = new File(TESTFILE);
DataInputStream dis = new DataInputStream(new GZIPInputStream(new FileInputStream(f), (int)f.length()))

and then start to read in the data in very small chunk via dis, which is a disaster.

While it is possible to workaround the issue easily by
a) to use a smaller buf size, such as the defualt 512
b) wrap the GZIPInputStream with a BufferedInputStream

It still appears the memory/buffer handling code in Inflater.c can be changed to improve the performance in use cases similar to above mentioned scenario. The proposed change is to copy no more than the size of the "user's buffer". It's an de-compression, so you can't consume more bytes than you can write out in most cases. The attached test case shows the performance of gzip-ing the un-compressed rt.jar by using different buffer size.

Again, we still need to address the "fundamental performance" issue caused by moving away from JNI_Get*Critical for #6206933. But that will be a separate project.

-Sherman


-----------------------------------------------------------------------------
    private static int BUFSIZE = 1024 * 1024;

    public static void main(String[] args) throws Exception {

        File f = new File(args[0]);

        //gzip it into memory, yeah we have enough memoery:-)
ByteArrayOutputStream baos = new ByteArrayOutputStream((int)f.length() / 2);
        GZIPOutputStream gout = new GZIPOutputStream(baos);
        FileInputStream in = new FileInputStream(args[0]);
        byte[] copyBuf = new byte[BUFSIZE];
        int n;
        while ((n = in.read(copyBuf)) != -1) {
            gout.write(copyBuf, 0, n);
        }
        in.close();
        gout.close();

        byte[] gsrc = baos.toByteArray();

        for (int i = 1; i <= 128; i <<= 1 ) {
            copyBuf = new byte[BUFSIZE / i];
            long time = System.currentTimeMillis();
            for (int j = 0; j < 20; j++) {
                ByteArrayInputStream bais = new ByteArrayInputStream(gsrc);
                GZIPInputStream gin = new GZIPInputStream(bais, BUFSIZE);
                while ((gin.read(copyBuf)) != -1) {}
                gin.close();
            }
System.out.printf( "buf[1/%d]: %d ms%n", i, (System.currentTimeMillis() - time)/ 20);
        }
    }
------------------------------------------------------------------------

(1) latest jdk7
sher...@sherman-linux:/home/sherman/TL/dis$ /net/koori/onestop/jdk/7/latest/binaries/linux-i586/bin/java Inf /tmp/rt.jar
buf[1/1]: 579 ms
buf[1/2]: 640 ms
buf[1/4]: 613 ms
buf[1/8]: 683 ms
buf[1/16]: 873 ms
buf[1/32]: 1247 ms
buf[1/64]: 1958 ms
buf[1/128]: 3379 ms

(2) with proposed fix
sher...@sherman-linux:/home/sherman/TL/dis$ java Inf /tmp/rt.jarbuf[1/1]: 514 ms
buf[1/2]: 546 ms
buf[1/4]: 460 ms
buf[1/8]: 413 ms
buf[1/16]: 426 ms
buf[1/32]: 448 ms
buf[1/64]: 436 ms
buf[1/128]: 448 ms

Reply via email to