Hi Sherman, Martin,
On 05/23/2016 08:28 PM, Xueming Shen wrote:
My apology. I meant to say "to have Inflater to be aware of the
ObjectPool...".
The proposed change switches from the finalizer to the ObjectPool to
clean up
the native resource for Inflater. It appears to be a bigger change.
Which has
a default 32/maxCacheSize and 1 secod keepAliveTime setting. It might have
a big impact to existing applications on how to use the Inflater. It
would be
fine to have the ZipFile to arrange how to use the Inflater based on
its use
scenario, but I'm concerned if it's the correct approach to change the
default
behavior of Inflater, which might have nothing to do with ZipFile.
sherman
It might have a big impact to existing applications, yes. A positive one!
On 05/23/2016 09:21 PM, Martin Buchholz wrote:
....
I am still contending that zstream caching is not worth much. The
best I could do with a synthetic benchmark designed to make caching
win was 25% better performance. Martin's rule is if you can't even
come up with a friendly benchmark that gives you a 2x win, give up on
your optimization! So performance wise, there is no big impact here
on any applications, except to fix excessive memory retention and
OOME.
Here's the 1st test I did. The micro benchmark Martin created and is
attached to the issue 8156484. Just modified a bit to issue a heap dump
at the end:
public class Benchmark60kMulti {
private static void dumpHistogram() {
// dump a heap histogram
try {
new ProcessBuilder(
System.getProperty("java.home") + "/bin/jmap",
"-histo:live",
String.valueOf(ProcessHandle.current().getPid())
).redirectOutput(ProcessBuilder.Redirect.to(
new File("histo_live.txt"))
).start().waitFor();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) throws Throwable {
final int REPS = 20;
final int THREADS = 8;
final byte[] data = "123456789".getBytes("UTF-8");
final File file = new File("60k.zip");
if (!file.exists()) {
try (FileOutputStream fos = new FileOutputStream(file);
BufferedOutputStream bos = new BufferedOutputStream(fos);
ZipOutputStream zos = new ZipOutputStream(bos)) {
for (int i = 0; i < 60_000; i++) {
ZipEntry ze = new ZipEntry(i + ".txt");
ze.setMethod(ZipEntry.DEFLATED);
ze.setSize(0);
ze.setCrc(0);
zos.putNextEntry(ze);
zos.write(data);
}
}
}
Path target = Paths.get("60k.zip");
for (int i = 0; i < THREADS; i++) {
Path link = Paths.get("symlink-" + i + ".zip");
if (!link.toFile().exists())
Files.createSymbolicLink(link, target);
}
ExecutorService pool = Executors.newFixedThreadPool(THREADS);
ArrayList<Future<?>> futures = new ArrayList<>();
for (int i = 0; i < THREADS; i++) {
final Path link = Paths.get("symlink-" + i + ".zip");
final Runnable task = () -> {
try (ZipFile zipFile = new ZipFile(link.toFile())) {
for (int j = 0; j < REPS; j++) {
byte[] buf = new byte[100];
Enumeration<? extends ZipEntry> entries =
zipFile.entries();
while (entries.hasMoreElements()) {
ZipEntry zentry = entries.nextElement();
try (InputStream is =
zipFile.getInputStream(zentry)) {
is.read(buf);
}
}
}
} catch (Throwable t) { throw new Error(t); }
};
futures.add(pool.submit(task));
}
pool.shutdown();
for (Future future : futures) future.get();
dumpHistogram();
pool.awaitTermination(1, TimeUnit.DAYS);
}
}
"time" Linux command and a top-10 heap dump of live objects:
real 0m22.197s
user 1m34.968s
sys 0m12.577s
num #instances #bytes class name (module)
-------------------------------------------------------
1: 3621046 704046712 [B (java.base@9-internal)
2: 2410133 96405320 java.lang.ref.Finalizer
(java.base@9-internal)
3: 1572755 75492240
java.util.zip.ZipFile$ZipFileInputStream (java.base@9-internal)
4: 1204735 67465160
java.util.zip.ZipFile$ZipFileInflaterInputStream (java.base@9-internal)
5: 4597 147104 java.util.HashMap$Node
(java.base@9-internal)
6: 5875 141000 java.lang.String (java.base@9-internal)
7: 1140 138656 java.lang.Class (java.base@9-internal)
8: 2681 85792
java.util.concurrent.ConcurrentHashMap$Node (java.base@9-internal)
9: 1938 84760 [Ljava.lang.Object;
(java.base@9-internal)
10: 636 61056 [Ljava.util.HashMap$Node;
(java.base@9-internal)
With webrev.03 applied:
real 0m6.734s
user 0m32.937s
sys 0m13.760s
num #instances #bytes class name (module)
-------------------------------------------------------
1: 6903 483592 [B (java.base@9-internal)
2: 4598 147136 java.util.HashMap$Node
(java.base@9-internal)
3: 1177 143112 java.lang.Class (java.base@9-internal)
4: 5936 142464 java.lang.String (java.base@9-internal)
5: 2681 85792
java.util.concurrent.ConcurrentHashMap$Node (java.base@9-internal)
6: 1943 84608 [Ljava.lang.Object;
(java.base@9-internal)
7: 2 65568 [Ljava.util.concurrent.ForkJoinTask;
(java.base@9-internal)
8: 637 61136 [Ljava.util.HashMap$Node;
(java.base@9-internal)
9: 11 33696 [C (java.base@9-internal)
10: 637 30576 java.util.HashMap
(java.base@9-internal)
I think this is more than 2x win in speed, don't you? Not to mention memory.
Regards, Peter
