Re: Excessive memory usage

[Michel Krämer]

Hi!

I can confirm that metaspace usage is much better with JDK 9 Build b85. 
My little sample program now only needs 1 GB of resident memory and not 
1.5 GB like before. That's a huge improvement! Thank you so much!


Still I'm wondering - if the heap usage is only at 150 MB, why does 
Nashorn need 850 MB of metaspace after all just to execute a JavaScript 
program. OK, the TypeScript compiler is not really small, but 850 MB 
seems to be a lot compared to what the V8 engine needs for example (just 
a few MB). Maybe it's worth looking deeper into this issue?


Is there any chance these fixes might be backported to Java 8 or is it 
too much work?


Cheers,
Michel


Am 05.10.2015 um 22:38 schrieb Michel Krämer:

(Ooops. I didn't press the reply-to-list button.)

Thanks for the link to the bug report. I will keep observing the issue
and let you know if I find something. If possible I'll also do a test
with a snapshot build of JDK9 and see if things have changed.

Thanks,
Michel


Am 05.10.2015 um 22:19 schrieb Hannes Wallnoefer:

Well maybe there is a problem if the process is killed because it uses
too much memory. I just think it's not heap space, at least not in that
particular code in your github repository. Is that the same code that is
running in the CI environments?

I must say I did see increased GC activity and a slowdown after half the
iterations.

We did fix some problems recently that may cause memory problems under
certain circumstances, such as
https://bugs.openjdk.java.net/browse/JDK-8137333

So I'm not saying there isn't a problem - there may well be one. I just
think that the observed memory numbers on Linux are not that
extraordinary. Please keep observing and let us know if you find
anything new and noteworthy.

Hannes


Am 2015-10-05 um 21:46 schrieb Michel Krämer:

Hannes,

Thanks for the quick response. I see...

The reason why I'm asking all this is because I tried to run the unit
tests for vertx-lang-typescript on Travis CI and Circle CI and they
both killed the process after it had reached 4GB (in fact very quickly
after the process had started). I had to play with environment
variables such as MALLOC_ARENA_MAX to get it down to 2.5 GB.

At least it runs now, but I figured it might be something of interest
for you as I've never experienced this behaviour with any other code
that I ran on Travis CI or Circle CI (and I have a couple of open
source projects running there). The memory usage increased quickly
when I ran JavaScript code so I thought it might have something to do
with Nashorn.

I was able to reproduce the issue on my own Ubuntu machine so it's not
related to Travis or Circle.

As far as I can tell heap memory is not the problem. It seems
metaspace is used a lot or maybe direct memory buffers or maybe even
something else!?

Anyway, if you say this much of memory is normal under Linux then I'm
completely fine with it. I just thought if it wasn't the case you
might want to know.

Cheers,
Michel


Am 05.10.2015 um 21:34 schrieb Hannes Wallnoefer:

Hi Michel,

Thanks for the quick and easy github repository to reproduce this.

Looking at the code running with jvisualvm I can't see any excessive
memory usage. Both with JDK 8u60 and 9 heap usage is around 150 MB.

The fact that Java reserves a lot of memory on Linux is a well-known
fact. It's not related to Nashorn, but rather to how Java interacts
with
the Linux memory system. It is a bit annoying, but usually it's not a
problem and does not reflect Java heap usage.

Hannes


Am 2015-10-05 um 20:02 schrieb Michel Krämer:

Hi!

I'm currently working on TypeScript support for Vert.x. I'm trying to
run the TypeScript compiler on Nashorn. It works well, but the process
uses a lot of memory. I'm wondering if there is a bug in Nashorn or if
I'm doing something wrong.

I uploaded a small example project demonstrating the issue:
https://github.com/michel-kraemer/nashorn-memory-test

I tested it under Ubuntu 14.04 with JDK 8u60. I run

javac Main.java && java -Xmx256M -cp . Main

and watch the process with top. The resident memory quickly goes up to
1 GB and after about 20 iterations grows even further to 1.2 GB. After
about 500 iterations it sometimes even goes up to 1.5 GB where it
stays until the end of the program. That's 1.3 GB more than I
specified with -Xmx. I know this is metaspace, but I wonder why
Nashorn needs so much of it. By the way, even if I do only 1 iteration
the process still goes up to 400-500 MB.

Any ideas?

Cheers,
Michel

Re: Review request for JDK-8139888: Improve Dynalink JavaDoc some more

[Sundararajan Athijegannathan]

* File: 
http://cr.openjdk.java.net/~attila/8139888/webrev.jdk9/src/jdk.scripting.nashorn/share/classes/jdk/internal/dynalink/DynamicLinkerFactory.java.udiff.html


"they are themselves" -> "these are" ?

+1

-Sundar


On 10/19/2015 8:04 PM, Attila Szegedi wrote:

Please review JDK-8139888 "Improve Dynalink JavaDoc some more" at 
 for 


This is the last anticipated JavaDoc focused changeset.

In addition to JavaDoc improvements, there are few minor changes:
- DynamicLinkerFactory rejects nulls in lists of linkers being passed
- LinkRequest.replaceArguments was made variable arity (taking Object… as its 
last arg instead of Object[]), making it consistent with SipleLinkRequest 
constructor
- SimpleLinkRequest rejects nulls in constructor parameters now, and clones 
incoming arguments lists
- Lookup and NameCodec classes were made final
  
Thanks,

   Attila.

Re: Review request for JDK-8139905: Add a convenience AccessControlContext factory

[Sundararajan Athijegannathan]

+1

PS. Should the nashorn's AccessControlContextFactory be under 
jdk.nashorn.internal.runtime? I think this is the first class under 
jdk.nashorn.internal...


-Sundar

On 10/20/2015 12:03 AM, Attila Szegedi wrote:

Please note that there’s two AccessControlContextFactory classes; it’s 
unfortunate, but we’ll be separating Dynalink from Nashorn, and while Nashorn 
can rely on Dynalink, we don’t want to expose the ACC factory from Dynalink, so 
lacking a better place, we’ll duplicate this small utility for now.

Attila.


On Oct 19, 2015, at 8:07 PM, Attila Szegedi  wrote:

Please review JDK-8139905 "Add a convenience AccessControlContext factory" at 
 for 


Thanks,
  Attila.

Re: Review request for JDK-8139919: Make CallSiteDescriptor a concrete class

[Sundararajan Athijegannathan]

+1

-Sundar

On 10/20/2015 12:17 AM, Attila Szegedi wrote:

Please review JDK-8139919 "Make CallSiteDescriptor a concrete class" at 
 for 


Thanks,
   Attila.

Re: Review request for JDK-8139895: Introduce GuardingDynamicLinkerExporter

[Sundararajan Athijegannathan]

* File: DynamicLinkerFactory:

+ String autoLoaderClassName = "unknown";
+ final GuardingDynamicLinkerExporter autoLoader = it.next();
+ try {
+ autoLoaderClassName = autoLoader.getClass().getName();
+ final String knownAutoLoaderClassName = autoLoaderClassName;

Do we need 2 variables to store auto loaded class name? 
autoLoaderClassName seems to be used only to initialize second (final) 
variable. Am I missing something?


* If a particular linker exporter jar does not have necessary permission 
to export, it appears that it can still do "deniel of service". i.e., 
SecurityException will force for loop in "discoverAutoLoadLinkers"method 
to exit early and prevent other legitimate 'linker exporters" from working.


-Sundar

On 10/19/2015 9:51 PM, Attila Szegedi wrote:

Please review JDK-8139895 "Introduce GuardingDynamicLinkerExporter" at 
 for 


Remarks:
- DynamicLinkerFactory now loads the linkers in a doPrivileged block so that 
the lack of “dynalink.exportLinkersAutomatically” permission in the caller 
doesn’t preclude it from loading trusted linkers. This is consistent with how 
e.g. ScriptEngineManager uses ServiceLoader.
- DynamicLinkerFactory now has a new method "List 
getAutoLoadingErrors()” so that the user can inspect if some automatically loaded linkers 
failed to load. This is especially significant as the GuardingDynamicLinkerExporter can 
have some failure modes: it can lack the dynalink.exportLinkersAutomatically permission, 
it can return null from get() or any element of the returned element might be null

Thanks,
   Attila.

Review request for JDK-8139919: Make CallSiteDescriptor a concrete class

[Attila Szegedi]

Please review JDK-8139919 "Make CallSiteDescriptor a concrete class" at 
 for 


Thanks,
  Attila.

Re: Review request for JDK-8139905: Add a convenience AccessControlContext factory

[Attila Szegedi]

Please note that there’s two AccessControlContextFactory classes; it’s 
unfortunate, but we’ll be separating Dynalink from Nashorn, and while Nashorn 
can rely on Dynalink, we don’t want to expose the ACC factory from Dynalink, so 
lacking a better place, we’ll duplicate this small utility for now.

Attila.

> On Oct 19, 2015, at 8:07 PM, Attila Szegedi  wrote:
> 
> Please review JDK-8139905 "Add a convenience AccessControlContext factory" at 
>  for 
> 
> 
> Thanks,
>  Attila.

Review request for JDK-8139905: Add a convenience AccessControlContext factory

[Attila Szegedi]

Please review JDK-8139905 "Add a convenience AccessControlContext factory" at 
 for 


Thanks,
  Attila.

Review request for JDK-8139895: Introduce GuardingDynamicLinkerExporter

[Attila Szegedi]

Please review JDK-8139895 "Introduce GuardingDynamicLinkerExporter" at 
 for 


Remarks:
- DynamicLinkerFactory now loads the linkers in a doPrivileged block so that 
the lack of “dynalink.exportLinkersAutomatically” permission in the caller 
doesn’t preclude it from loading trusted linkers. This is consistent with how 
e.g. ScriptEngineManager uses ServiceLoader.
- DynamicLinkerFactory now has a new method "List 
getAutoLoadingErrors()” so that the user can inspect if some automatically 
loaded linkers failed to load. This is especially significant as the 
GuardingDynamicLinkerExporter can have some failure modes: it can lack the 
dynalink.exportLinkersAutomatically permission, it can return null from get() 
or any element of the returned element might be null

Thanks,
  Attila.

Re: Review request for JDK-8139884: Use privileged blocks when working with class loaders

[Hannes Wallnoefer]

+1

Am 2015-10-19 um 15:53 schrieb Attila Szegedi:

Please review JDK-8139884 "Use privileged blocks when working with class loaders" at 
 for 


Thanks,
   Attila.

Re: Review request for JDK-8139435: Make sure CallSiteDescriptor.getLookup is subject to a security check

[Hannes Wallnoefer]

Am 2015-10-19 um 08:29 schrieb Attila Szegedi:

On Oct 16, 2015, at 5:11 PM, Hannes Wallnoefer  
wrote:

What's the rationale for providing the static lookupEquals and lookupHashCode  
methods in AbstractCallSiteDescriptor instead of just putting the code in 
non-abstract instance methods? This way it's a bit more flexible, but I'm not 
sure it warrants the additional API surface.

I agree that the API surface increase is not great. The idea was to externalize 
into subclasses anything that needs access to the lookup object instead of 
providing access to the lookup object in a superclass. But in the end, that’s 
actually a better idea (as Sundar pointed out). Fortunately, another changeset 
coming soon will fix this by making CallSiteDescriptor into a class instead of 
an interface, and then reduce the API surface back (and eventually eliminate 
all subclasses and just leave CallSiteDescriptor).


Thanks for the explanation, sounds good.

Hannes

Re: Review request for JDK-8139756: Eliminate GuardedTypeConversion, DynamicLinker.getCurrentLinkRequest and its associated permission

[Hannes Wallnoefer]

+1


Am 2015-10-16 um 15:39 schrieb Attila Szegedi:

Please review JDK-8139756 "Eliminate GuardedTypeConversion, DynamicLinker.getCurrentLinkRequest 
and its associated permission" at  
for 

Note: webrev also shows some previous issues in the history of some of the 
files, but the changes for those issue are not in the diffs.

Thanks,
   Attila.

Re: Review request for JDK-8139887: Reduce visibility of few methods in TypeUtilities and Guards API

[Sundararajan Athijegannathan]

+1

-Sundar

On 10/19/2015 7:32 PM, Attila Szegedi wrote:

Please review JDK-8139887 "Reduce visibility of few methods in TypeUtilities and Guards 
API" at  for 


Thanks,
   Attila.

Review request for JDK-8139888: Improve Dynalink JavaDoc some more

[Attila Szegedi]

Please review JDK-8139888 "Improve Dynalink JavaDoc some more" at 
 for 


This is the last anticipated JavaDoc focused changeset.

In addition to JavaDoc improvements, there are few minor changes:
- DynamicLinkerFactory rejects nulls in lists of linkers being passed 
- LinkRequest.replaceArguments was made variable arity (taking Object… as its 
last arg instead of Object[]), making it consistent with SipleLinkRequest 
constructor
- SimpleLinkRequest rejects nulls in constructor parameters now, and clones 
incoming arguments lists
- Lookup and NameCodec classes were made final
 
Thanks,
  Attila.

Re: Review request for JDK-8139884: Use privileged blocks when working with class loaders

[Michael Haupt]

Hi Attila,

lower-case thumbs up, with just the remark that lambdas could make the code 
more concise.

Best,

Michael

> Am 19.10.2015 um 15:53 schrieb Attila Szegedi :
> 
> Please review JDK-8139884 "Use privileged blocks when working with class 
> loaders" at  for 
> 
> 
> Thanks,
>  Attila.

Re: Review request for JDK-8139884: Use privileged blocks when working with class loaders

[Sundararajan Athijegannathan]

+1

On 10/19/2015 7:23 PM, Attila Szegedi wrote:

Please review JDK-8139884 "Use privileged blocks when working with class loaders" at 
 for 


Thanks,
   Attila.

Review request for JDK-8139887: Reduce visibility of few methods in TypeUtilities and Guards API

[Attila Szegedi]

Please review JDK-8139887 "Reduce visibility of few methods in TypeUtilities 
and Guards API" at  for 


Thanks,
  Attila.

Review request for JDK-8139884: Use privileged blocks when working with class loaders

[Attila Szegedi]

Please review JDK-8139884 "Use privileged blocks when working with class 
loaders" at  for 


Thanks,
  Attila.

Re: RFR 8139852: jjs interactive mode fails to work with security manager

[Hannes Wallnoefer]

+1

Am 2015-10-19 um 07:50 schrieb Sundararajan Athijegannathan:
Please review  http://cr.openjdk.java.net/~sundar/8139852/ for 
https://bugs.openjdk.java.net/browse/JDK-8139852


Thanks,
-Sundar

Re: Review request for JDK-8139761: Improve Dynalink class nomenclature and package organization

[Sundararajan Athijegannathan]

+1 on

http://mail.openjdk.java.net/pipermail/nashorn-dev/2015-October/005431.html

-Sundar