Re: Getting a live view of environment variables (Gradle and JDK 9)

Actual mutability of the Unix process environment is something we have to give up when moving to multi-threaded processes. The only time you get to change the environment is when running single-threaded, typically around fork/exec. Learn to live with it. The Java System.getenv() is initialized

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 12.10.2017 16:23, Kirk Pepperdine wrote: Hi, On Oct 12, 2017, at 2:54 PM, Mario Torre wrote: 2017-10-12 11:58 GMT+02:00 Cédric Champeau : 1. an API in 18.3 which would let us refresh the environment variables, even if

Re: Getting a live view of environment variables (Gradle and JDK 9)

Yes that's one option. Note that declaring environment variables is not a small change. It has consequences on the project layout. Typically, if they need to be in the settings.gradle file, then now the client would have to compile, parse and potentially partially execute code. Similarly if it's

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cedric, I think you should play like the JDK 9 i.e. help users to transition from th old world to the new world where env variables need to be declared explicitly. So warn user that the env variables should now be declared explicitly, use an agent to simulate the old getenv+grapefruit,

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi, > On Oct 12, 2017, at 2:54 PM, Mario Torre > wrote: > > 2017-10-12 11:58 GMT+02:00 Cédric Champeau : > >> 1. an API in 18.3 which would let us refresh the environment variables, >> even if inherently unsafe (we can take the risk,

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-12 11:58 GMT+02:00 Cédric Champeau : > 1. an API in 18.3 which would let us refresh the environment variables, > even if inherently unsafe (we can take the risk, if the Javadocs explains > that if you're really unlucky calling such a method could kill your VM).

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi folks, Let me resurrect this thread now that Java 9 is out (congrats to the team!). We still have the problem, and actually some users complained about the poor performance of Gradle under JDK 9, and it turned out to be exactly the problem I mentioned in this thread. Now with JDK 9, everytime

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 19.05.2017 08:08, Noctarius wrote: Hey Jochen, Couldn't the gradle call transfer his view of the environment? I know that's a workaround but it would solve the issue, wouldn't it? IMHO it could so by calling native code using posix functions. But what does it matter if getEnv is not

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hey Jochen, Couldn't the gradle call transfer his view of the environment? I know that's a workaround but it would solve the issue, wouldn't it? Chris On May 19, 2017 8:02:58 AM GMT+02:00, Jochen Theodorou wrote: >On 16.05.2017 19:46, Uwe Schindler wrote: >> Hi, >> >> I

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 16.05.2017 19:46, Uwe Schindler wrote: Hi, I still don't understand why you need to change the environment variables of the actual process. I was talking with Rémi about that last week, and it's not obvious to me why you need that. Sure, it is easier to change the environment of the actual

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 18/05/2017 6:19 PM, Cédric Champeau wrote: Can you elaborate as to why specifying the "big kill switch" --permit-illegal-access is not viable? Specifically if you use: -XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access it should work on 9 and be ignored on earlier releases. mmm

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 18/05/2017 09:35, Stephen Colebourne wrote: : I agree with Daivid that my expectation for the method System.getenv would be that it calls native code every time it is invoked assuming that the Javadoc didn't say otherwise (because environment variables are outside the Java universe). As

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 18 May 2017 at 09:07, David Holmes wrote: > I'm quite surprised by some of the reactions here. Given the read-once > nature of System.getenv was never documented I expected most people to be > surprised that it took a snapshot-on-first-use, and that reading the actual

Re: Getting a live view of environment variables (Gradle and JDK 9)

> On 18 May 2017, at 09:07, David Holmes wrote: >> ... >> The reason to use a daemon is to avoid multiple starts of a JVM in-between >> steps in a build. > > Yes, though I've been looking at this from the actual API proposal > perspective, not the "how can we avoid

Re: Getting a live view of environment variables (Gradle and JDK 9)

> > > Can you elaborate as to why specifying the "big kill switch" > --permit-illegal-access is not viable? Specifically if you use: > > -XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access > > it should work on 9 and be ignored on earlier releases. > > mmm that's interesting, I actually

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cedric, Starting a new sub-thread on this as I want to back up on something. Can you elaborate as to why specifying the "big kill switch" --permit-illegal-access is not viable? Specifically if you use: -XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access it should work on 9 and be

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 18/05/2017 5:46 PM, Kirk Pepperdine wrote: On May 18, 2017, at 6:15 AM, David Holmes wrote: On 18/05/2017 2:36 AM, Martin Buchholz wrote: Stop trying to add APIs to mutate environment variables. It's not going to happen! Not sure who or what you are

Re: Getting a live view of environment variables (Gradle and JDK 9)

> On May 18, 2017, at 6:15 AM, David Holmes wrote: > > On 18/05/2017 2:36 AM, Martin Buchholz wrote: >> Stop trying to add APIs to mutate environment variables. It's not going >> to happen! > > Not sure who or what you are directing this at Martin. Certainly I'm not

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cedric, I understand what you are trying to do, it is my opinion that you are trying to do this in a completely unexpected way. That the environment variables in the JVM are read only once is an issue in that a nohup should refresh them That all the code in the JVM may (or may not) react to

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cedric, On 05/16/2017 12:05 PM, Cédric Champeau wrote: Thanks Peter for your thoughts, but I don't think it's as simple as that. As I explained in my original email, there are multiple ways the environment variables can be mutated, and it can be done _externally_. Typically, during a task

Re: Getting a live view of environment variables (Gradle and JDK 9)

On Wed, May 17, 2017 at 2:15 PM, David Holmes wrote: > On 18/05/2017 2:36 AM, Martin Buchholz wrote: > >> >> Yes, Java still has a serious startup performance problem; that should >> be attacked more directly. >> > > ?? Not sure what startup has to do with the topic

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 18/05/2017 2:36 AM, Martin Buchholz wrote: Stop trying to add APIs to mutate environment variables. It's not going to happen! Not sure who or what you are directing this at Martin. Certainly I'm not suggesting any API to mutate environment variables. This is about an API to re-read an

Re: Getting a live view of environment variables (Gradle and JDK 9)

Stop trying to add APIs to mutate environment variables. It's not going to happen! There is no known way to mutate environment variables in a multithreaded Unix process without risk of native crashes (and I have worn the hat of poor support engineer trying to diagnose SEGV in getenv). IIRC we

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 17/05/2017 8:45 PM, Remi Forax wrote: What i really like in Java is the diversity of options. We are discussing here to add or not a way to get the live environment in a Java process, at the same time on hotspot-compiler-dev Yasumasa posts a message with a set of examples that shows how to

Re: Getting a live view of environment variables (Gradle and JDK 9)

What i really like in Java is the diversity of options. We are discussing here to add or not a way to get the live environment in a Java process, at the same time on hotspot-compiler-dev Yasumasa posts a message with a set of examples that shows how to do any C syscalls from Java using JVMCI

Re: Getting a live view of environment variables (Gradle and JDK 9)

> > >> I disagree, this would be totally expected behavior. The daemon and this >> process would run in different shells and I am unaware of any daemon >> process that auto-magically reconfigures it’s self to adapt to any other >> arbitrary shell’s changed environment variables. > > The thing is

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 17.05.2017 00:53, Kirk Pepperdine wrote: println System.getenv('MY_VAR') doesn't print "foo" after doing: MY_VAR=foo gradle printVar I disagree, this would be totally expected behavior. The daemon and this process would run in different shells and I am unaware of any daemon process that

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 17/05/2017 6:21 AM, Martin Buchholz wrote: On Tue, May 16, 2017 at 12:29 PM, Chris Hegarty wrote: Re-reading/refreshing the env variables has come up before, in other contexts. As David pointed out [1], re-reading introduces no more risk than is already there (

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi, > On May 17, 2017, at 3:11 AM, Cédric Champeau > wrote: > > Hi Uwe, > > I already explained multiple times why we need this. Short answer: because > we must. Slightly longer answer: because the build environment, the daemon, > has to reflect the environment

Re: Getting a live view of environment variables (Gradle and JDK 9)

On Tue, May 16, 2017 at 12:31 PM, Cédric Champeau wrote: > > So far the only suitable workaround I can think of is an agent that would > rewrite `System.getenv` to call our internal APIs. I'll suggest that in our > next meeting (which is happening 30 mins from now). >

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Jonathan, Bazel is a cathedral when Gradle is a bazaar, Bazel uses a controlled subset of Python as build language, and a strictly defined API that do not let you extend Bazel in an arbitrary way. Those two approaches, a strict DSL or a general purpose languages retargeted to be used as a

Re: Getting a live view of environment variables (Gradle and JDK 9)

On Tue, May 16, 2017 at 1:03 PM, Jonathan Bluett-Duncan < jbluettdun...@gmail.com> wrote: > Hi Cédric, > > I don't know if it's been considered, but has anyone from the Gradle team > asked the Bazel team about this problem? > I'm not on the Bazel team, but ... they are focused on being hermetic

Re: Getting a live view of environment variables (Gradle and JDK 9)

On Tue, May 16, 2017 at 12:29 PM, Chris Hegarty wrote: > > Re-reading/refreshing the env variables has come up before, in other > contexts. As David pointed out [1], re-reading introduces no more risk than > is already there ( if specified correctly ). > Adding a Java

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cédric, I don't know if it's been considered, but has anyone from the Gradle team asked the Bazel team about this problem? They may have useful insight about this problem with `System.getenv`, considering that Bazel is apparently very fast and (partially) written in Java. I hope this helps.

Re: Getting a live view of environment variables (Gradle and JDK 9)

I don't deny some people do not have the problem :) 2017-05-16 21:56 GMT+02:00 Uwe Schindler : > Hi, > > Another example is not having your problem is Jenkins CI: It maintains a > map of environment variables throughout the build. And you have to use that > a a state

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi, Another example is not having your problem is Jenkins CI: It maintains a map of environment variables throughout the build. And you have to use that a a state container throughout the build. I have never seen a plug-in not using that, because it's fundamental to the whole build server. If

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Mario, I'm not suggesting to change the JDK. My original email is about having a new API to get a live view. Alternatives, like new methods to refresh the view, are non-breaking (I think). I perfectly understand, and fully respect, the willingness not to break millions of developers, and

Re: Getting a live view of environment variables (Gradle and JDK 9)

> On 16 May 2017, at 20:19, Mario Torre wrote: > .. > Regarding your example: > > """ > println System.getenv('MY_VAR') > > doesn't print "foo" after doing: > > MY_VAR=foo gradle printVar > """ > > I agree that the environment variables may change during the

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 20:38 GMT+02:00 Cédric Champeau : > Let me rephrase: it's tiring to have to repeat why we need it, and why we > honor the contract of environment variables. Why is it so hard to admit the > JDK has a bug? Hello Cédric, I hope you don't take it wrong or

Re: Getting a live view of environment variables (Gradle and JDK 9)

Let me rephrase: it's tiring to have to repeat why we need it, and why we honor the contract of environment variables. Why is it so hard to admit the JDK has a bug? I proposed different solutions for this, including non breaking changes (a new method). All I have in return is suggestions to change

Re: Getting a live view of environment variables (Gradle and JDK 9)

I’m sorry you find it tiring, but I think you underestimate how high the bar is (and should be) for changing how Java works just because it’s inconvenient for your situation. In my view, you’re not remotely there on justifying this. By all means continue, but “this is too tiring” isn’t a

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Uwe, I already explained multiple times why we need this. Short answer: because we must. Slightly longer answer: because the build environment, the daemon, has to reflect the environment from the CLI (or the IDE, in case we call using the tooling API), at the moment the build is executed. Why?

RE: Getting a live view of environment variables (Gradle and JDK 9)

Hi, I still don't understand why you need to change the environment variables of the actual process. I was talking with Rémi about that last week, and it's not obvious to me why you need that. Sure, it is easier to change the environment of the actual process and then spawn a child process for

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 18:57 GMT+02:00 Sanne Grinovero : > Hi Cédric, > > we use Gradle a lot in our team (Hibernate) and all your efforts to > make it faster are highly appreciated. > > I can assure you that during a normal day of work while we might > rebuild a project a few hundred

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cédric, we use Gradle a lot in our team (Hibernate) and all your efforts to make it faster are highly appreciated. I can assure you that during a normal day of work while we might rebuild a project a few hundred times, I'll rarely have to change environment variables. Obviously I don't

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-15 7:14 GMT+02:00 David Holmes : >> There would have to be a caveat on System.getenv(true) if we went that >> path, that it is up to the user to ensure it is called in as safe a >> manner as possible having regard to any concurrent updates in their >> application

Re: Getting a live view of environment variables (Gradle and JDK 9)

Thanks Peter for your thoughts, but I don't think it's as simple as that. As I explained in my original email, there are multiple ways the environment variables can be mutated, and it can be done _externally_. Typically, during a task execution, a JNI call performed by a random native tool could

Re: Getting a live view of environment variables (Gradle and JDK 9)

Thanks for the explanation, David. That doesn't sound much more risky than what we already do today in getenv. Cheers, Dalibor -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 11.05.2017 18:29, Cédric Champeau wrote: Unfortunately, they are not safely mutable in multi-threaded programs on many operating system/libc combinations. But the problem is less about mutating, that it is about reading: the VM returns wrong values at some point, because it

Re: Getting a live view of environment variables (Gradle and JDK 9)

> > >> > Unfortunately, they are not safely mutable in multi-threaded programs on > many operating system/libc combinations. > > But the problem is less about mutating, that it is about reading: the VM returns wrong values at some point, because it _assumes_ that the environment variables are not

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-11 12:50 GMT+02:00 David Holmes : > Hi Cedric, > > On 11/05/2017 5:02 PM, Cédric Champeau wrote: > >> Thanks for the answers, folks, but I think they are kind of missing the >> point. The fact is that environment variables *are* mutable. Java >> > > Yes they are,

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 11.05.2017 09:02, Cédric Champeau wrote: Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Unfortunately, they are not safely mutable in multi-threaded programs on many operating system/libc combinations.

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cedric, On 11/05/2017 5:02 PM, Cédric Champeau wrote: Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Java Yes they are, provided you do it carefully and don't have to deal with the fact that its

Re: Getting a live view of environment variables (Gradle and JDK 9)

On 11/05/2017 08:02, Cédric Champeau wrote: Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Java happens to return an immutable view of the environment variables when the VM was started, which is not the

Re: Getting a live view of environment variables (Gradle and JDK 9)

Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Java happens to return an immutable view of the environment variables when the VM was started, which is not the reality. We cannot trust what `System.geteenv`

Re: Getting a live view of environment variables (Gradle and JDK 9)

Since you're OK with doing brain surgery on the JDK and you control the entire process, consider controlling your daemon with a bytecode rewriting agent (e.g. byteman) that changes the definition of System.getenv etc. (Whose side are you on Martin?! ... I confess I also wish for a faster gradle

Re: Getting a live view of environment variables (Gradle and JDK 9)

Hi Cedric, Like Martin I am somewhat surprised by the recent surge in interest in the JVM environment :) From a pragmatic perspective it is far too late to do anything about this in 9. I am curious what kind of environment variables are being set by the client, how they are shared with

Re: Getting a live view of environment variables (Gradle and JDK 9)

You're already acquiring superpowers and voiding any warranty by calling putenv from JNI. Why not go further and do your reflective shenanigans via JNI as well? In practice, if hotspot has started up and only one java thread is doing anything, putenv is unlikely to cause a crash (and probably

Re: Getting a live view of environment variables (Gradle and JDK 9)

It seems to be the day for environment variable mutation. System.getenv was always designed as an immutable snapshot, and more recently there's emerging consensus that one must never ever mutate environment variables in a multi-threaded Unix program. One tends to acquire a biased view on this