Re: RFR: 8175561: Memory churn in jimage code affects startup after resource encapsulation changes

[Mandy Chung]

> On Feb 23, 2017, at 2:57 PM, Claes Redestad  wrote:
> 
> Hi,
> 
> various resource encapsulation changes in 9+148 meant an uptick in
> footprint and startup times for certain applications.
> 
> While some of this regression is hard to avoid[1] (opening readers,
> touching more memory mapped pages etc), a large part is due to simple
> java allocation churn, some of which can be optimized away by reducing
> the number of objects we create when scanning modules for resources.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8175561
> Webrev: http://cr.openjdk.java.net/~redestad/8175561/jdk.01/
> 


ImageStringsReader.java
   Should the hashCode methods taking byte[] parameter be removed?

ImageLocation.java
   I suggest to add the comment to describe the expect format comparing to the 
name. something like  “//[/][.]” and a comment for 
each if-statement what segment is checking.

I have carefully reviewed the change which attempts to inline the code and 
write to avoid object allocation.  It looks correct to me.  

Since jimage is a sensitive area, I suggest to run PIT and do the automated 
hotspot testing.

Mandy

hg: jigsaw/jake/jdk: jlink ExcludeVMPlugin should use ResourcePoolModule.osName()

[sundararajan . athijegannathan]

Changeset: 2b20fe8e7938
Author:sundar
Date:  2017-02-24 22:22 +0530
URL:   http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/2b20fe8e7938

jlink ExcludeVMPlugin should use ResourcePoolModule.osName()

! 
src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/ExcludeVMPlugin.java
! test/ProblemList.txt

Re: Extending java.base module

[Volker Simonis]

On Wed, Feb 15, 2017 at 5:20 PM, Volker Simonis
 wrote:
>
> On Wed, Feb 15, 2017 at 5:16 PM, Alan Bateman  wrote:
> > On 15/02/2017 16:01, Daniel Fuchs wrote:
> >
> >>
> >> In that specific case it's not java.base that depends
> >> on java.security.jgss, but the application itself.
> >>
> >> So I would expect the application code to either require
> >> java.security.jgss, or some higher level module for that
> >> itself requires java.security.jgss, or jlink to be run with
> >> command line options that explicitly add java.security.jgss
> >> to the image.
> >
> > java.security.jgss exports an API so it will be resolved by default when the
> > initial class is loaded from the class path. In addition, it provides a
> > SecurityProvider implementation and so will be resolved because java.base
> > `uses java.security.Provider`. For the jlink case then you are right, it
> > needs someone to know that the application might need to do SPNEGO
> > authentication.
> >
> > In any case, it's an example of how not to do things, and hopefully it will
> > be cleaned up at some point.
> >
>

While digging deeper into this topic, I found other such examples (and
you can easily find more by searching for "Class.forName" in the base
module :)

sun.text.bidi.BidiBase$TextAttributeConstants (which is in the base
module) has the following static initializer:

// Make sure to load the AWT's TextAttribute class before using the
constants, if any.
static {
try {
Class.forName("java.awt.font.TextAttribute", true, null);
} catch (ClassNotFoundException e) {}
}
static final JavaAWTFontAccess jafa = SharedSecrets.getJavaAWTFontAccess();

which is clearly another implicit dependency from java.base to
java.desktop which contains java.awt.font.TextAttribute

There's yet another such implicit dependency from java.base to jdk.net
in sun.net.ext.ExtendedSocketOptions which has the following static
initializer:

static {
try {
// If the class is present, it will be initialized which
// triggers registration of the extended socket options.
Class c = Class.forName("jdk.net.ExtendedSocketOptions");
} catch (ClassNotFoundException e) {
// the jdk.net module is not present => no extended socket options
instance = new NoExtendedSocketOptions();
}
}

So I'm wondering if these all are examples of "how not to do things"
which will be cleaned up eventually or are if there other reasons that
this cleanup hasn't been done until now. I can imagine that if all the
current, implicit dependencies would have been resolved by using
services, service binding could lead to a much bigger dependency graph
for the base module which would probably include most of the modules
in the jdk. Not sure if this could have an negative performance impact
on startup time?

Wouldn't it be a good idea to have a list of all this implicit
dependencies somewhere (preferable in JBS) with a short comment for
each of them (like "should be fixed by using services", "will leave as
is because it's required too early during initialization", "should
remove dependency by refactoring the code", etc..)

Thank you and best regards,
Volker

>
> Daniel, Alan, thanks for the clarification. I didn't wanted to blame
> anybody - just looking for good arguments to prevent such code in our
> version of the JDK :)
>
> > -Alan
> >
> >

Re: RFR: 8175385: ServiceLoader$LazyClassPathLookupIterator scans boot and platform modules for services

[Claes Redestad]

On 02/24/2017 09:50 AM, Alan Bateman wrote:



On 23/02/2017 23:19, Claes Redestad wrote:

Hi,

related to the same regression as JDK-8175561, there are now some cases
where we unnecessarily scan platform modules for services.  This patch
allows us to skip the platform and boot classloaders in certain cases,
which speeds up loading and enumerating the built-in services.

Bug: https://bugs.openjdk.java.net/browse/JDK-8175561
Webrev: http://cr.openjdk.java.net/~redestad/8175385/jdk.01/

Looks good.


Thanks!

A minor nit is to move BootLoader hasClassPath() so it's co-located 
with the other public static methods.


Moved it up to just after packages() (updated in-place).

/Claes



-Alan

hg: jigsaw/jake/hotspot: 2 new changesets

[alan . bateman]

Changeset: 750ef08b1278
Author:lana
Date:  2017-02-23 16:21 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/750ef08b1278

Added tag jdk-9+158 for changeset 4e78f3093522

! .hgtags

Changeset: ae7703cdbb50
Author:alanb
Date:  2017-02-24 11:07 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/ae7703cdbb50

Merge

! .hgtags

hg: jigsaw/jake/jaxp: 8 new changesets

[alan . bateman]

Changeset: 5621e47022e5
Author:joehw
Date:  2017-02-14 10:28 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/5621e47022e5

8169450: StAX parse error if there is a newline in xml declaration
Reviewed-by: clanger, dfuchs, lancea

! 
src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XMLDocumentScannerImpl.java
+ test/javax/xml/jaxp/unittest/parsers/BaseParsingTest.java

Changeset: f329c1a1a1ea
Author:fyuan
Date:  2017-02-15 11:43 +0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/f329c1a1a1ea

8174025: Regression in XML Transform caused by JDK-8087303
Reviewed-by: joehw, dfuchs

! 
src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/ToStream.java
! 
src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/dom3/DOM3TreeWalker.java
! test/javax/xml/jaxp/unittest/common/prettyprint/PrettyPrintTest.java
+ test/javax/xml/jaxp/unittest/common/prettyprint/generate-catalog.xsl
! test/javax/xml/jaxp/unittest/common/prettyprint/htmltest1.xml
+ test/javax/xml/jaxp/unittest/common/prettyprint/htmltest7.out
+ test/javax/xml/jaxp/unittest/common/prettyprint/htmltest7.xml
! test/javax/xml/jaxp/unittest/common/prettyprint/nodetest2.out
+ test/javax/xml/jaxp/unittest/common/prettyprint/nodetest2ls.out
+ 
test/javax/xml/jaxp/unittest/common/prettyprint/simple-entity-resolver-config-transformed.xml
+ 
test/javax/xml/jaxp/unittest/common/prettyprint/simple-entity-resolver-config.xml
- test/javax/xml/jaxp/unittest/common/prettyprint/xmltest5.out
+ test/javax/xml/jaxp/unittest/common/prettyprint/xmltest5ls.out
+ test/javax/xml/jaxp/unittest/common/prettyprint/xmltest5xslt.out
- test/javax/xml/jaxp/unittest/common/prettyprint/xmltest7.out
+ test/javax/xml/jaxp/unittest/common/prettyprint/xmltest7ls.out
+ test/javax/xml/jaxp/unittest/common/prettyprint/xmltest7xslt.out
! test/javax/xml/jaxp/unittest/common/prettyprint/xmltest8.out
! test/javax/xml/jaxp/unittest/common/prettyprint/xmltest8.xml
! test/javax/xml/jaxp/unittest/dom/ls/LSSerializerTest.java

Changeset: 7d070d3e794b
Author:aefimov
Date:  2017-02-16 04:11 +0300
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/7d070d3e794b

8173390: Investigate SymbolTable in SAXParser
Reviewed-by: dfuchs, joehw

! 
src/java.xml/share/classes/com/sun/org/apache/xerces/internal/parsers/XML11Configuration.java
+ test/javax/xml/jaxp/unittest/sax/SymbolTableResetTest.java

Changeset: 97698c9cd095
Author:aefimov
Date:  2017-02-16 13:19 +0300
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/97698c9cd095

8174735: Update JAX-WS RI integration to latest version
Reviewed-by: alanb, mchung, lancea
Contributed-by: roman.grigori...@oracle.com

! src/java.xml/share/classes/module-info.java

Changeset: 83b9302a43f9
Author:fyuan
Date:  2017-02-16 21:39 +0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/83b9302a43f9

8175043: Multiple jaxp tests failing across platforms
Reviewed-by: weijun, joehw

! test/javax/xml/jaxp/functional/org/w3c/dom/ptests/NodeTest.java
! test/javax/xml/jaxp/functional/test/astro/DocumentLSTest.java
! test/javax/xml/jaxp/unittest/parsers/Bug6341770.java
! test/javax/xml/jaxp/unittest/sax/Bug7057778Test.java
! test/javax/xml/jaxp/unittest/stream/Bug6688002Test.java
! test/javax/xml/jaxp/unittest/stream/XMLEventWriterTest/ReaderToWriterTest.java
! test/javax/xml/jaxp/unittest/stream/XMLStreamWriterTest/WriterTest.java
! test/javax/xml/jaxp/unittest/transform/Bug4693341Test.java
! test/javax/xml/jaxp/unittest/transform/Bug4892774.java
! test/javax/xml/jaxp/unittest/transform/Bug6216226Test.java
! test/javax/xml/jaxp/unittest/transform/CR6935697Test.java
! test/javax/xml/jaxp/unittest/transform/XSLTFunctionsTest.java
! test/javax/xml/jaxp/unittest/transform/util/TransformerUtil.java
! test/javax/xml/jaxp/unittest/validation/CR6708840Test.java
! test/javax/xml/jaxp/unittest/validation/ValidatorTest.java

Changeset: 60e670a65e07
Author:lana
Date:  2017-02-16 18:28 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/60e670a65e07

Merge

- test/javax/xml/jaxp/unittest/common/prettyprint/xmltest5.out
- test/javax/xml/jaxp/unittest/common/prettyprint/xmltest7.out

Changeset: bee6f114a47a
Author:lana
Date:  2017-02-23 16:21 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/bee6f114a47a

Added tag jdk-9+158 for changeset 60e670a65e07

! .hgtags

Changeset: 1ed81018fce2
Author:alanb
Date:  2017-02-24 11:07 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/1ed81018fce2

Merge

! .hgtags
! src/java.xml/share/classes/module-info.java
- test/javax/xml/jaxp/unittest/common/prettyprint/xmltest5.out
- test/javax/xml/jaxp/unittest/common/prettyprint/xmltest7.out

hg: jigsaw/jake/nashorn: 2 new changesets

[alan . bateman]

Changeset: d75af059cff6
Author:lana
Date:  2017-02-23 16:21 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/d75af059cff6

Added tag jdk-9+158 for changeset 13ae2480a4c3

! .hgtags

Changeset: 46ba0efe400d
Author:alanb
Date:  2017-02-24 11:07 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/46ba0efe400d

Merge

! .hgtags

hg: jigsaw/jake/langtools: 26 new changesets

[alan . bateman]

Changeset: 39f31cef8e6a
Author:rfield
Date:  2017-02-13 08:50 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/39f31cef8e6a

8174797: jshell tool: invalid module path crashes tool
8174796: jshell tool: regression: user home (tilde) not translated
Reviewed-by: jlahoda

! src/jdk.jshell/share/classes/jdk/internal/jshell/tool/JShellTool.java
! test/jdk/jshell/ToolBasicTest.java
! test/jdk/jshell/ToolSimpleTest.java

Changeset: c73b98b79148
Author:darcy
Date:  2017-02-13 10:29 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/c73b98b79148

8173945: Add methods for Elements.getAll{Type, Package, Module}Elements
Reviewed-by: jlahoda, jjg

! src/java.compiler/share/classes/javax/lang/model/util/Elements.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/model/JavacElements.java
+ test/tools/javac/processing/model/util/elements/TestAllFoos.java
+ test/tools/javac/processing/model/util/elements/modules/m1/module-info.java
+ test/tools/javac/processing/model/util/elements/modules/m1/pkg/C.java
+ 
test/tools/javac/processing/model/util/elements/modules/m1/pkg/package-info.java
+ test/tools/javac/processing/model/util/elements/modules/m2/module-info.java
+ test/tools/javac/processing/model/util/elements/modules/m2/pkg/C.java
+ 
test/tools/javac/processing/model/util/elements/modules/m2/pkg/package-info.java

Changeset: b87052a3056f
Author:darcy
Date:  2017-02-13 11:51 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/b87052a3056f

8174854: Fix two javax.annotation.processing javadoc link issues
Reviewed-by: mchung

! 
src/java.compiler/share/classes/javax/annotation/processing/AbstractProcessor.java
! src/java.compiler/share/classes/javax/annotation/processing/Processor.java

Changeset: ab6d375bab80
Author:rfield
Date:  2017-02-13 12:14 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/ab6d375bab80

8174857: jshell tool: /help /set truncation -- confusing indentation
Reviewed-by: jjg

! 
src/jdk.jshell/share/classes/jdk/internal/jshell/tool/resources/l10n.properties

Changeset: 0837e68b5689
Author:darcy
Date:  2017-02-13 12:59 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/0837e68b5689

8174860: Fix bad javadoc link in javax.tools.JavaFileManager
Reviewed-by: jjg

! src/java.compiler/share/classes/javax/tools/JavaFileManager.java

Changeset: 44e21b2f6370
Author:rfield
Date:  2017-02-13 17:04 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/44e21b2f6370

8174862: JShell tests: new JDK-8174797 testInvalidClassPath fails on Windows
Reviewed-by: jlahoda

! test/jdk/jshell/ToolSimpleTest.java

Changeset: 6967e22a5ddd
Author:mli
Date:  2017-02-13 18:19 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/6967e22a5ddd

8174695: Fix @since in module-info.java in dev/langtools repo
Reviewed-by: mcimadamore

! src/java.compiler/share/classes/module-info.java
! src/jdk.compiler/share/classes/module-info.java
! src/jdk.javadoc/share/classes/module-info.java
! src/jdk.jdeps/share/classes/module-info.java

Changeset: 457e4a27daad
Author:sadayapalam
Date:  2017-02-14 14:24 +0530
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/457e4a27daad

8170691: fill in @bug number for test
Reviewed-by: darcy

! test/tools/javac/modules/AllDefaultTest.java

Changeset: 3cf5dd848c5b
Author:anazarov
Date:  2017-02-14 16:18 +0300
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/3cf5dd848c5b

8170404: Improve negative testing for module-info
Reviewed-by: jjg

! test/tools/javac/modules/ModuleInfoTest.java

Changeset: ce416299fd2d
Author:vromero
Date:  2017-02-14 15:45 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/ce416299fd2d

8174243: incorrect error message for nested service provider
Reviewed-by: jjg, jlahoda

! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/AttrContext.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Modules.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java
! 
src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties
+ test/tools/javac/diags/examples/ServiceImplNotPublic/ServiceImplNotPublic.java
+ test/tools/javac/diags/examples/ServiceImplNotPublic/example/ServiceImpl.java
+ test/tools/javac/diags/examples/ServiceImplNotPublic/example/SomeService.java
+ test/tools/javac/diags/examples/ServiceImplNotPublic/module-info.java
! test/tools/javac/modules/ProvidesTest.java
+ test/tools/javac/modules/WrongErrorMessageForNestedServiceProviderTest.java

Changeset: 2815405955ee
Author:jlahoda
Date:  2017-02-15 11:27 +0100
URL:   http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/2815405955ee

8175007: Incorrect error messages for inaccessible classes in visible packages
Summary: Recovery lookup may be triggered for inaccessible classes in visible 
packages - providing better 

hg: jigsaw/jake/corba: 2 new changesets

[alan . bateman]

Changeset: 6feea77d2083
Author:lana
Date:  2017-02-23 16:21 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/corba/rev/6feea77d2083

Added tag jdk-9+158 for changeset de6bdf38935f

! .hgtags

Changeset: 6045b4585af9
Author:alanb
Date:  2017-02-24 11:07 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/corba/rev/6045b4585af9

Merge

! .hgtags

hg: jigsaw/jake: 9 new changesets

[alan . bateman]

Changeset: 04a685edd41d
Author:darcy
Date:  2017-02-14 09:00 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/04a685edd41d

8174945: Turn on doclint reference checking in build of java.compiler module
Reviewed-by: erikj

! make/CompileJavaModules.gmk

Changeset: 15a37edc6717
Author:sherman
Date:  2017-02-15 12:11 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/15a37edc6717

8173207: Upgrade compression library
Reviewed-by: alanb, erikj

! common/bin/unshuffle_list.txt
! common/nb_native/nbproject/configurations.xml

Changeset: acad65b6e3ee
Author:darcy
Date:  2017-02-15 13:43 -0800
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/acad65b6e3ee

8175045: Turn on doclint reference checking in build of the java.management.rmi 
module
Reviewed-by: lancea, mchung

! make/CompileJavaModules.gmk

Changeset: 5a8a2aa4d40a
Author:erikj
Date:  2017-02-16 09:04 +0100
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/5a8a2aa4d40a

8174895: test/TestCommon.gmk: value of JTREG_TESTVM_MEMORY_OPTION is missing
Reviewed-by: dholmes, ihse

! test/TestCommon.gmk

Changeset: 42d0b3eec016
Author:chegar
Date:  2017-02-16 13:10 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/42d0b3eec016

8175071: Minor cleanup in Javadoc.gmk
Reviewed-by: erikj, ihse

! make/Javadoc.gmk

Changeset: ecb22730b4df
Author:lana
Date:  2017-02-16 18:27 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/ecb22730b4df

Merge


Changeset: a4087bc10a88
Author:redestad
Date:  2017-02-17 18:12 +0100
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/a4087bc10a88

8175026: Capture build-time parameters to --generate-jli-classes
Reviewed-by: mchung, ihse

! make/GenerateLinkOptData.gmk
! make/Images.gmk
! make/Main.gmk

Changeset: 281d20ee4076
Author:lana
Date:  2017-02-23 16:21 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/281d20ee4076

Added tag jdk-9+158 for changeset a4087bc10a88

! .hgtags

Changeset: 8c79dde5671e
Author:alanb
Date:  2017-02-24 11:07 +
URL:   http://hg.openjdk.java.net/jigsaw/jake/rev/8c79dde5671e

Merge

! .hgtags
! make/CompileJavaModules.gmk
! make/Images.gmk
! make/Javadoc.gmk
! make/Main.gmk

Re: RFR: 8175385: ServiceLoader$LazyClassPathLookupIterator scans boot and platform modules for services

[Alan Bateman]

On 23/02/2017 23:19, Claes Redestad wrote:

Hi,

related to the same regression as JDK-8175561, there are now some cases
where we unnecessarily scan platform modules for services.  This patch
allows us to skip the platform and boot classloaders in certain cases,
which speeds up loading and enumerating the built-in services.

Bug: https://bugs.openjdk.java.net/browse/JDK-8175561
Webrev: http://cr.openjdk.java.net/~redestad/8175385/jdk.01/
Looks good. A minor nit is to move BootLoader hasClassPath() so it's 
co-located with the other public static methods.


-Alan