Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-13 Thread Martin Buchholz
Actual mutability of the Unix process environment is something we have to give up when moving to multi-threaded processes. The only time you get to change the environment is when running single-threaded, typically around fork/exec. Learn to live with it. The Java System.getenv() is initialized f

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-12 Thread dalibor topic
On 12.10.2017 16:23, Kirk Pepperdine wrote: Hi, On Oct 12, 2017, at 2:54 PM, Mario Torre wrote: 2017-10-12 11:58 GMT+02:00 Cédric Champeau : 1. an API in 18.3 which would let us refresh the environment variables, even if inherently unsafe (we can take the risk, if the Javadocs explains th

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-12 Thread Cédric Champeau
À: "Mario Torre" > > Cc: "core-libs-dev" > > Envoyé: Jeudi 12 Octobre 2017 16:23:22 > > Objet: Re: Getting a live view of environment variables (Gradle and JDK > 9) > > > Hi, > > > >> On Oct 12, 2017, at 2:54 PM, Mario Torre com> w

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-12 Thread Remi Forax
runtime if the agent is present, if it's not fork. step 2. retire the agent and profit cheers, Rémi - Mail original - > De: "Kirk Pepperdine" > À: "Mario Torre" > Cc: "core-libs-dev" > Envoyé: Jeudi 12 Octobre 2017 16:23:22 > Objet

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-12 Thread Kirk Pepperdine
Hi, > On Oct 12, 2017, at 2:54 PM, Mario Torre > wrote: > > 2017-10-12 11:58 GMT+02:00 Cédric Champeau : > >> 1. an API in 18.3 which would let us refresh the environment variables, >> even if inherently unsafe (we can take the risk, if the Javadocs explains >> that if you're really unlucky ca

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-12 Thread Mario Torre
2017-10-12 11:58 GMT+02:00 Cédric Champeau : > 1. an API in 18.3 which would let us refresh the environment variables, > even if inherently unsafe (we can take the risk, if the Javadocs explains > that if you're really unlucky calling such a method could kill your VM). Being a public API we would

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-10-12 Thread Cédric Champeau
Hi folks, Let me resurrect this thread now that Java 9 is out (congrats to the team!). We still have the problem, and actually some users complained about the poor performance of Gradle under JDK 9, and it turned out to be exactly the problem I mentioned in this thread. Now with JDK 9, everytime a

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Jochen Theodorou
On 19.05.2017 08:08, Noctarius wrote: Hey Jochen, Couldn't the gradle call transfer his view of the environment? I know that's a workaround but it would solve the issue, wouldn't it? IMHO it could so by calling native code using posix functions. But what does it matter if getEnv is not update

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Noctarius
Hey Jochen, Couldn't the gradle call transfer his view of the environment? I know that's a workaround but it would solve the issue, wouldn't it? Chris On May 19, 2017 8:02:58 AM GMT+02:00, Jochen Theodorou wrote: >On 16.05.2017 19:46, Uwe Schindler wrote: >> Hi, >> >> I still don't underst

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Jochen Theodorou
On 16.05.2017 19:46, Uwe Schindler wrote: Hi, I still don't understand why you need to change the environment variables of the actual process. I was talking with Rémi about that last week, and it's not obvious to me why you need that. Sure, it is easier to change the environment of the actual

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread David Holmes
On 18/05/2017 6:19 PM, Cédric Champeau wrote: Can you elaborate as to why specifying the "big kill switch" --permit-illegal-access is not viable? Specifically if you use: -XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access it should work on 9 and be ignored on earlier releases. mmm th

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Alan Bateman
On 18/05/2017 09:35, Stephen Colebourne wrote: : I agree with Daivid that my expectation for the method System.getenv would be that it calls native code every time it is invoked assuming that the Javadoc didn't say otherwise (because environment variables are outside the Java universe). As such,

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Stephen Colebourne
On 18 May 2017 at 09:07, David Holmes wrote: > I'm quite surprised by some of the reactions here. Given the read-once > nature of System.getenv was never documented I expected most people to be > surprised that it took a snapshot-on-first-use, and that reading the actual > process environment at t

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Chris Hegarty
> On 18 May 2017, at 09:07, David Holmes wrote: >> ... >> The reason to use a daemon is to avoid multiple starts of a JVM in-between >> steps in a build. > > Yes, though I've been looking at this from the actual API proposal > perspective, not the "how can we avoid the need for this API" persp

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Cédric Champeau
> > > Can you elaborate as to why specifying the "big kill switch" > --permit-illegal-access is not viable? Specifically if you use: > > -XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access > > it should work on 9 and be ignored on earlier releases. > > mmm that's interesting, I actually forgot

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread David Holmes
Hi Cedric, Starting a new sub-thread on this as I want to back up on something. Can you elaborate as to why specifying the "big kill switch" --permit-illegal-access is not viable? Specifically if you use: -XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access it should work on 9 and be ign

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread David Holmes
On 18/05/2017 5:46 PM, Kirk Pepperdine wrote: On May 18, 2017, at 6:15 AM, David Holmes wrote: On 18/05/2017 2:36 AM, Martin Buchholz wrote: Stop trying to add APIs to mutate environment variables. It's not going to happen! Not sure who or what you are directing this at Martin. Certainl

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Kirk Pepperdine
> On May 18, 2017, at 6:15 AM, David Holmes wrote: > > On 18/05/2017 2:36 AM, Martin Buchholz wrote: >> Stop trying to add APIs to mutate environment variables. It's not going >> to happen! > > Not sure who or what you are directing this at Martin. Certainly I'm not > suggesting any API to mu

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Kirk Pepperdine
Hi Cedric, I understand what you are trying to do, it is my opinion that you are trying to do this in a completely unexpected way. That the environment variables in the JVM are read only once is an issue in that a nohup should refresh them That all the code in the JVM may (or may not) react to

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-18 Thread Peter Levart
Hi Cedric, On 05/16/2017 12:05 PM, Cédric Champeau wrote: Thanks Peter for your thoughts, but I don't think it's as simple as that. As I explained in my original email, there are multiple ways the environment variables can be mutated, and it can be done _externally_. Typically, during a task e

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-17 Thread Martin Buchholz
On Wed, May 17, 2017 at 2:15 PM, David Holmes wrote: > On 18/05/2017 2:36 AM, Martin Buchholz wrote: > >> >> Yes, Java still has a serious startup performance problem; that should >> be attacked more directly. >> > > ?? Not sure what startup has to do with the topic here. There would be less ne

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-17 Thread David Holmes
On 18/05/2017 2:36 AM, Martin Buchholz wrote: Stop trying to add APIs to mutate environment variables. It's not going to happen! Not sure who or what you are directing this at Martin. Certainly I'm not suggesting any API to mutate environment variables. This is about an API to re-read an alr

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-17 Thread Martin Buchholz
Stop trying to add APIs to mutate environment variables. It's not going to happen! There is no known way to mutate environment variables in a multithreaded Unix process without risk of native crashes (and I have worn the hat of poor support engineer trying to diagnose SEGV in getenv). IIRC we ma

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-17 Thread David Holmes
/main/java/jnr/posix/POSIX.java - Mail original - De: "Cédric Champeau" À: "dalibor topic" Cc: "core-libs-dev" Envoyé: Mercredi 17 Mai 2017 09:21:33 Objet: Re: Getting a live view of environment variables (Gradle and JDK 9) I disagree, this would be

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-17 Thread Remi Forax
://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/POSIX.java - Mail original - > De: "Cédric Champeau" > À: "dalibor topic" > Cc: "core-libs-dev" > Envoyé: Mercredi 17 Mai 2017 09:21:33 > Objet: Re: Getting a live view of environm

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-17 Thread Cédric Champeau
> > >> I disagree, this would be totally expected behavior. The daemon and this >> process would run in different shells and I am unaware of any daemon >> process that auto-magically reconfigures it’s self to adapt to any other >> arbitrary shell’s changed environment variables. > > The thing is th

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread dalibor topic
On 17.05.2017 00:53, Kirk Pepperdine wrote: println System.getenv('MY_VAR') doesn't print "foo" after doing: MY_VAR=foo gradle printVar I disagree, this would be totally expected behavior. The daemon and this process would run in different shells and I am unaware of any daemon process that

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread David Holmes
On 17/05/2017 6:21 AM, Martin Buchholz wrote: On Tue, May 16, 2017 at 12:29 PM, Chris Hegarty wrote: Re-reading/refreshing the env variables has come up before, in other contexts. As David pointed out [1], re-reading introduces no more risk than is already there ( if specified correctly ).

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Kirk Pepperdine
Hi, > On May 17, 2017, at 3:11 AM, Cédric Champeau > wrote: > > Hi Uwe, > > I already explained multiple times why we need this. Short answer: because > we must. Slightly longer answer: because the build environment, the daemon, > has to reflect the environment from the CLI (or the IDE, in ca

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Martin Buchholz
On Tue, May 16, 2017 at 12:31 PM, Cédric Champeau wrote: > > So far the only suitable workaround I can think of is an agent that would > rewrite `System.getenv` to call our internal APIs. I'll suggest that in our > next meeting (which is happening 30 mins from now). > Yes, given that you have co

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Remi Forax
Champeau" > Cc: "core-libs-dev" > Envoyé: Mardi 16 Mai 2017 22:03:49 > Objet: Re: Getting a live view of environment variables (Gradle and JDK 9) > Hi Cédric, > > I don't know if it's been considered, but has anyone from the Gradle team > asked th

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Martin Buchholz
On Tue, May 16, 2017 at 1:03 PM, Jonathan Bluett-Duncan < jbluettdun...@gmail.com> wrote: > Hi Cédric, > > I don't know if it's been considered, but has anyone from the Gradle team > asked the Bazel team about this problem? > I'm not on the Bazel team, but ... they are focused on being hermetic -

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Martin Buchholz
On Tue, May 16, 2017 at 12:29 PM, Chris Hegarty wrote: > > Re-reading/refreshing the env variables has come up before, in other > contexts. As David pointed out [1], re-reading introduces no more risk than > is already there ( if specified correctly ). > Adding a Java API for re-reading would be

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Jonathan Bluett-Duncan
Hi Cédric, I don't know if it's been considered, but has anyone from the Gradle team asked the Bazel team about this problem? They may have useful insight about this problem with `System.getenv`, considering that Bazel is apparently very fast and (partially) written in Java. I hope this helps.

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Cédric Champeau
rely on the fact that another plugin or maybe another build running >>> at a later time is changing the Gradle process environment. >>> >>> At some point you have to break backwards compatibility and tell users: >>> Please update your code, otherwise plugin won&#x

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Uwe Schindler
's compatible to Java 9). >> >> Uwe >> >> - >> Uwe Schindler >> uschind...@apache.org >> ASF Member, Apache Lucene PMC / Committer >> Bremen, Germany >> http://lucene.apache.org/ >> >> > -Original Message- >> > F

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Cédric Champeau
Hi Mario, I'm not suggesting to change the JDK. My original email is about having a new API to get a live view. Alternatives, like new methods to refresh the view, are non-breaking (I think). I perfectly understand, and fully respect, the willingness not to break millions of developers, and that's

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Chris Hegarty
> On 16 May 2017, at 20:19, Mario Torre wrote: > .. > Regarding your example: > > """ > println System.getenv('MY_VAR') > > doesn't print "foo" after doing: > > MY_VAR=foo gradle printVar > """ > > I agree that the environment variables may change during the program > execution and that perha

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Mario Torre
2017-05-16 20:38 GMT+02:00 Cédric Champeau : > Let me rephrase: it's tiring to have to repeat why we need it, and why we > honor the contract of environment variables. Why is it so hard to admit the > JDK has a bug? Hello Cédric, I hope you don't take it wrong or personal, but we really are here

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Cédric Champeau
Let me rephrase: it's tiring to have to repeat why we need it, and why we honor the contract of environment variables. Why is it so hard to admit the JDK has a bug? I proposed different solutions for this, including non breaking changes (a new method). All I have in return is suggestions to change

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Brian Goetz
I’m sorry you find it tiring, but I think you underestimate how high the bar is (and should be) for changing how Java works just because it’s inconvenient for your situation. In my view, you’re not remotely there on justifying this. By all means continue, but “this is too tiring” isn’t a valid

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Cédric Champeau
From: core-libs-dev [mailto:core-libs-dev-boun...@openjdk.java.net] On > > Behalf Of Cédric Champeau > > Sent: Thursday, May 11, 2017 9:02 AM > > To: Martin Buchholz > > Cc: core-libs-dev > > Subject: Re: Getting a live view of environment variables (Gradle

RE: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Uwe Schindler
y 11, 2017 9:02 AM > To: Martin Buchholz > Cc: core-libs-dev > Subject: Re: Getting a live view of environment variables (Gradle and JDK 9) > > Thanks for the answers, folks, but I think they are kind of missing the > point. The fact is that environment variables *are* mutab

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Cédric Champeau
2017-05-16 18:57 GMT+02:00 Sanne Grinovero : > Hi Cédric, > > we use Gradle a lot in our team (Hibernate) and all your efforts to > make it faster are highly appreciated. > > I can assure you that during a normal day of work while we might > rebuild a project a few hundred times, I'll rarely have

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Sanne Grinovero
Hi Cédric, we use Gradle a lot in our team (Hibernate) and all your efforts to make it faster are highly appreciated. I can assure you that during a normal day of work while we might rebuild a project a few hundred times, I'll rarely have to change environment variables. Obviously I don't repres

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Mario Torre
2017-05-15 7:14 GMT+02:00 David Holmes : >> There would have to be a caveat on System.getenv(true) if we went that >> path, that it is up to the user to ensure it is called in as safe a >> manner as possible having regard to any concurrent updates in their >> application code and how the environmen

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Cédric Champeau
Thanks Peter for your thoughts, but I don't think it's as simple as that. As I explained in my original email, there are multiple ways the environment variables can be mutated, and it can be done _externally_. Typically, during a task execution, a JNI call performed by a random native tool could mu

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-16 Thread Dalibor Topic
Thanks for the explanation, David. That doesn't sound much more risky than what we already do today in getenv. Cheers, Dalibor -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 227

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-15 Thread David Holmes
Re-sending to core-libs-dev On 13/05/2017 5:56 PM, David Holmes wrote: Hi Dalibor, On 12/05/2017 11:28 PM, dalibor topic wrote: On 11.05.2017 18:29, Cédric Champeau wrote: Unfortunately, they are not safely mutable in multi-threaded programs on many operating system/libc combination

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-12 Thread dalibor topic
On 11.05.2017 18:29, Cédric Champeau wrote: Unfortunately, they are not safely mutable in multi-threaded programs on many operating system/libc combinations. But the problem is less about mutating, that it is about reading: the VM returns wrong values at some point, because it _assumes

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-11 Thread Cédric Champeau
> > >> > Unfortunately, they are not safely mutable in multi-threaded programs on > many operating system/libc combinations. > > But the problem is less about mutating, that it is about reading: the VM returns wrong values at some point, because it _assumes_ that the environment variables are not m

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-11 Thread Cédric Champeau
2017-05-11 12:50 GMT+02:00 David Holmes : > Hi Cedric, > > On 11/05/2017 5:02 PM, Cédric Champeau wrote: > >> Thanks for the answers, folks, but I think they are kind of missing the >> point. The fact is that environment variables *are* mutable. Java >> > > Yes they are, provided you do it careful

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-11 Thread dalibor topic
On 11.05.2017 09:02, Cédric Champeau wrote: Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Unfortunately, they are not safely mutable in multi-threaded programs on many operating system/libc combinations.

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-11 Thread David Holmes
Hi Cedric, On 11/05/2017 5:02 PM, Cédric Champeau wrote: Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Java Yes they are, provided you do it carefully and don't have to deal with the fact that its basically

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-11 Thread Alan Bateman
On 11/05/2017 08:02, Cédric Champeau wrote: Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Java happens to return an immutable view of the environment variables when the VM was started, which is not the reality

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-11 Thread Cédric Champeau
Thanks for the answers, folks, but I think they are kind of missing the point. The fact is that environment variables *are* mutable. Java happens to return an immutable view of the environment variables when the VM was started, which is not the reality. We cannot trust what `System.geteenv` returns

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-10 Thread Martin Buchholz
Since you're OK with doing brain surgery on the JDK and you control the entire process, consider controlling your daemon with a bytecode rewriting agent (e.g. byteman) that changes the definition of System.getenv etc. (Whose side are you on Martin?! ... I confess I also wish for a faster gradle .

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-10 Thread David Holmes
Hi Cedric, Like Martin I am somewhat surprised by the recent surge in interest in the JVM environment :) From a pragmatic perspective it is far too late to do anything about this in 9. I am curious what kind of environment variables are being set by the client, how they are shared with the

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-10 Thread Martin Buchholz
You're already acquiring superpowers and voiding any warranty by calling putenv from JNI. Why not go further and do your reflective shenanigans via JNI as well? In practice, if hotspot has started up and only one java thread is doing anything, putenv is unlikely to cause a crash (and probably non

Re: Getting a live view of environment variables (Gradle and JDK 9)

2017-05-10 Thread Martin Buchholz
It seems to be the day for environment variable mutation. System.getenv was always designed as an immutable snapshot, and more recently there's emerging consensus that one must never ever mutate environment variables in a multi-threaded Unix program. One tends to acquire a biased view on this aft

Getting a live view of environment variables (Gradle and JDK 9)

2017-05-10 Thread Cédric Champeau
Hi all, I'm writing this on behalf of the Gradle team. This email is closely related to the other thread just posted today, but just a timeline coincidence (just like the email exchange I had today about this with Alan Bateman ;)) and not exactly the same issue. We are in the process of making su