Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Andrew Haley]

On 3/20/19 8:02 AM, Langer, Christoph wrote:
> Hi Andrew,
> 
>> Thanks. I get that, I'm just questioning the need to backport it to 11.
>> No matter, I've approved it now.
> 
> What do you mean by that? The bug JDK-8220362 [0] still has jdk11u-fix-no.

No it doesn't. JDK-8212828 still had a jdk11u-fix-no, so I fixed that.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Thomas Stüfe]

On Tue, Mar 19, 2019 at 5:10 PM Martin Buchholz  wrote:

> I'm the one who introduced vfork for process spawning, and I support
> Thomas' plan to switch to posix_spawn.
> (although I have not seen a crash attributed to using vfork on Linux)
>

Thanks Martin. I think for a long time vfork was the best alternative.

I remember seeing one or two cases in the last five years which could have
been caused by vfork. In both cases, in a fork-heavy environment with many
short-lived processes, a broken script killed the wrong pid - a just
spawned child - and pid's father disappeared at the same time without a
trace.

Cheers, Thomas

RE: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Langer, Christoph]

Hi Andrew,

> Thanks. I get that, I'm just questioning the need to backport it to 11.
> No matter, I've approved it now.

What do you mean by that? The bug JDK-8220362 [0] still has jdk11u-fix-no.

In case you approve JDK-8220362, I'll set the CSR (JDK-8220362 [1]) to 
finalized, referring to your approval.

Thanks
Christoph

[0] https://bugs.openjdk.java.net/browse/JDK-8212828
[1] https://bugs.openjdk.java.net/browse/JDK-8220362

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Thomas Stüfe]

Thank you, Andrew

.. Thomas

On Tue, Mar 19, 2019, 5:42 PM Andrew Haley  wrote:

> On 3/19/19 4:10 PM, Martin Buchholz wrote:
> > I'm the one who introduced vfork for process spawning, and I support
> Thomas' plan to switch to posix_spawn.
> > (although I have not seen a crash attributed to using vfork on Linux)
>
> Thanks. I get that, I'm just questioning the need to backport it to 11.
> No matter, I've approved it now.
>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. 
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Andrew Haley]

On 3/19/19 4:10 PM, Martin Buchholz wrote:
> I'm the one who introduced vfork for process spawning, and I support Thomas' 
> plan to switch to posix_spawn.
> (although I have not seen a crash attributed to using vfork on Linux)

Thanks. I get that, I'm just questioning the need to backport it to 11.
No matter, I've approved it now.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Martin Buchholz]

I'm the one who introduced vfork for process spawning, and I support
Thomas' plan to switch to posix_spawn.
(although I have not seen a crash attributed to using vfork on Linux)

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Thomas Stüfe]

On Tue, Mar 19, 2019 at 10:39 AM Andrew Haley  wrote:

> On 3/19/19 7:33 AM, Thomas Stüfe wrote:
>
> > The established (since decades) method of forking off on Linux JDKs
> > has been to use vfork(3). Using vfork is risky. There are chances of
> > crashing the parent process. The risk of this happening is very low,
> > but not zero.
>
> Do you have a reference to this? If we're at risk, I'd like to know
> exactly what the risk is.
>

Sure, see here an old mail from me to core libs last year:

https://mail.openjdk.java.net/pipermail/core-libs-dev/2018-September/055333.html

(Please ignore my proposed fix, turned out David Loyd's posix_spawn
proposal was a way simpler solution.)

very short version:

After child process is spawned via vfork() but before it calls exec() both
parent and child are vulnerable in the following way:

A signal received by the child may damage or kill the parent. True for both
synchronous (error) signals in the child and outside signals sent to the
child.
- since the child process code is old and stable, error signals are not
likely, with one possible exception: hitting a stack overflow in the child
process (e.g. when calling fork in a low stack situation)
- asynchronous signals could be sent from the outside. Unlikely to be a
targeted kill, since the parent cannot yet have communicated the child pid
to the outside; but could be a group kill.

Note that the risk depends on the the length of the time span between fork
and exec, which depends on the number of open file descriptors in the
parent.

As I wrote, the risk is low - a freak accident - but not zero and has a
question mark around it since these crashes are hard to attribute to vfork.
Typical symptom is that the parent process just dies without a trace.

---

As to the question: if it is so important why do we not switch the default?

This is the result of a discussion with the core-libs maintainers and a
trade off between the vfork() risk and the risk of adding this patch. Roger
preferred to provide the non-default alternative for 12 and switch the
default to posix_spawn() for 13.

I think long term it definitely will make sense to switch the default to
posix_spawn on 11 and 12 too, maybe after a grace period.

Cheers, Thomas




> > Since analyzing those crashes and attributing them to vfork(3) is
> > very difficult, there may be a number of reported cases not
> > associated with vfork(3) but in fact caused by it.
>
> > The patch adds a second, safer way to fork off childs, using
> > posix_spawn(3). This one has been the default on other platforms for
> > quite some while. The patch does not change the default fork mode -
> > which remains vfork(3). So installations would not be affected
> > unless they explicitely choose to use vfork(3).
> >
> > Having this possibility would allow us to use posix_spawn(3) in
> > situations where we are analysing crashes and want to exclude
> > vfork(3) as a cause. It also would enable us to use posix_spawn(3)
> > as a safe alternative by default if we choose to do so.
> >
> > In addition to that, as David Loyd mentioned, vfork(3) is marked as
> > obsolete and may be vanish at some point in the life span of JDK
> > 11. I admit the chance of this happening is low.
>
> Sure, but as I replied to him, this patch doesn't solve that problem.
>

Fair enough.



>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. 
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Andrew Haley]

On 3/8/19 2:39 PM, Langer, Christoph wrote:
> please review the CSR backport 
> https://bugs.openjdk.java.net/browse/JDK-8220362.
> 
> It is the backport of https://bugs.openjdk.java.net/browse/JDK-8214511 to 
> JDK11.
> 
> We want to backport 
> JDK-8212828 to jdk11u and 
> hence backporting the CSR is a prerequisite. The CSR is, however, more or 
> less identical for 11u.

OK.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Andrew Haley]

On 3/19/19 7:33 AM, Thomas Stüfe wrote:

> The established (since decades) method of forking off on Linux JDKs
> has been to use vfork(3). Using vfork is risky. There are chances of
> crashing the parent process. The risk of this happening is very low,
> but not zero.

Do you have a reference to this? If we're at risk, I'd like to know
exactly what the risk is.

> Since analyzing those crashes and attributing them to vfork(3) is
> very difficult, there may be a number of reported cases not
> associated with vfork(3) but in fact caused by it.

> The patch adds a second, safer way to fork off childs, using
> posix_spawn(3). This one has been the default on other platforms for
> quite some while. The patch does not change the default fork mode -
> which remains vfork(3). So installations would not be affected
> unless they explicitely choose to use vfork(3).
> 
> Having this possibility would allow us to use posix_spawn(3) in
> situations where we are analysing crashes and want to exclude
> vfork(3) as a cause. It also would enable us to use posix_spawn(3)
> as a safe alternative by default if we choose to do so.
> 
> In addition to that, as David Loyd mentioned, vfork(3) is marked as
> obsolete and may be vanish at some point in the life span of JDK
> 11. I admit the chance of this happening is low.

Sure, but as I replied to him, this patch doesn't solve that problem.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Andrew Haley]

On 3/11/19 1:09 PM, David Lloyd wrote:
> Is vfork still guaranteed to be functional by the end of the Java 11
> support frame, from the perspective of any organization which supports
> JDKs of this version?

It's extremely unlikely to be made nonfunctional. Linux backwards
compatibility is excellent.

Besides, if vfork() is nonfunctional we'll need to change the Linux
default -- not just provide an alternative -- so this patch will be
inadequate. All this patch does is provide an alternative to
experiment with.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

RE: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Langer, Christoph]

Hm, I’m not sure about the “reconsideration” process… I restored jdk11u-fix-no, 
because it’s probably not in my powers to remove it. But still the request for 
reconsideration remains 

/Christoph

From: Langer, Christoph
Sent: Dienstag, 19. März 2019 09:22
To: Thomas Stüfe ; Andrew Haley 
Cc: Java Core Libs ; Alan Bateman 
; Roger Riggs ; 
jdk-updates-...@openjdk.java.net; David Lloyd 
Subject: RE: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid 
value to jdk.lang.Process.launchMechanism on Linux

I have removed the jdk11u-fix-no label from the bug and added a fix request 
comment to trigger reconsideration.

/Christoph

From: Thomas Stüfe mailto:thomas.stu...@gmail.com>>
Sent: Dienstag, 19. März 2019 08:33
To: Andrew Haley mailto:a...@redhat.com>>
Cc: Langer, Christoph 
mailto:christoph.lan...@sap.com>>; Java Core Libs 
mailto:core-libs-dev@openjdk.java.net>>; Alan 
Bateman mailto:alan.bate...@oracle.com>>; Roger Riggs 
mailto:roger.ri...@oracle.com>>; 
jdk-updates-...@openjdk.java.net; 
David Lloyd mailto:david.ll...@redhat.com>>
Subject: Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid 
value to jdk.lang.Process.launchMechanism on Linux

Hi Andrew,

This would be good to have in jdk11 for the following reason:

The established (since decades) method of forking off on Linux JDKs has been to 
use vfork(3). Using vfork is risky. There are chances of crashing the parent 
process. The risk of this happening is very low, but not zero. Since analyzing 
those crashes and attributing them to vfork(3) is very difficult, there may be 
a number of reported cases not associated with vfork(3) but in fact caused by 
it.

The patch adds a second, safer way to fork off childs, using posix_spawn(3). 
This one has been the default on other platforms for quite some while. The 
patch does not change the default fork mode - which remains vfork(3). So 
installations would not be affected unless they explicitely choose to use 
vfork(3).

Having this possibility would allow us to use posix_spawn(3) in situations 
where we are analysing crashes and want to exclude vfork(3) as a cause. It also 
would enable us to use posix_spawn(3) as a safe alternative by default if we 
choose to do so.

In addition to that, as David Loyd mentioned, vfork(3) is marked as obsolete 
and may be vanish at some point in the life span of JDK 11. I admit the chance 
of this happening is low.

Kind Regards, Thomas

On Fri, Mar 8, 2019 at 4:20 PM Andrew Haley 
mailto:a...@redhat.com>> wrote:
On 3/8/19 2:39 PM, Langer, Christoph wrote:
> please review the CSR backport 
> https://bugs.openjdk.java.net/browse/JDK-8220362.
>
> It is the backport of https://bugs.openjdk.java.net/browse/JDK-8214511 to 
> JDK11.
>
> We want to backport 
> JDK-8212828 to jdk11u and 
> hence backporting the CSR is a prerequisite. The CSR is, however, more or 
> less identical for 11u.

I don't see any explanation of why this should go into jdk11.

--
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

RE: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Langer, Christoph]

I have removed the jdk11u-fix-no label from the bug and added a fix request 
comment to trigger reconsideration.

/Christoph

From: Thomas Stüfe 
Sent: Dienstag, 19. März 2019 08:33
To: Andrew Haley 
Cc: Langer, Christoph ; Java Core Libs 
; Alan Bateman ; Roger 
Riggs ; jdk-updates-...@openjdk.java.net; David Lloyd 

Subject: Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid 
value to jdk.lang.Process.launchMechanism on Linux

Hi Andrew,

This would be good to have in jdk11 for the following reason:

The established (since decades) method of forking off on Linux JDKs has been to 
use vfork(3). Using vfork is risky. There are chances of crashing the parent 
process. The risk of this happening is very low, but not zero. Since analyzing 
those crashes and attributing them to vfork(3) is very difficult, there may be 
a number of reported cases not associated with vfork(3) but in fact caused by 
it.

The patch adds a second, safer way to fork off childs, using posix_spawn(3). 
This one has been the default on other platforms for quite some while. The 
patch does not change the default fork mode - which remains vfork(3). So 
installations would not be affected unless they explicitely choose to use 
vfork(3).

Having this possibility would allow us to use posix_spawn(3) in situations 
where we are analysing crashes and want to exclude vfork(3) as a cause. It also 
would enable us to use posix_spawn(3) as a safe alternative by default if we 
choose to do so.

In addition to that, as David Loyd mentioned, vfork(3) is marked as obsolete 
and may be vanish at some point in the life span of JDK 11. I admit the chance 
of this happening is low.

Kind Regards, Thomas

On Fri, Mar 8, 2019 at 4:20 PM Andrew Haley 
mailto:a...@redhat.com>> wrote:
On 3/8/19 2:39 PM, Langer, Christoph wrote:
> please review the CSR backport 
> https://bugs.openjdk.java.net/browse/JDK-8220362.
>
> It is the backport of https://bugs.openjdk.java.net/browse/JDK-8214511 to 
> JDK11.
>
> We want to backport 
> JDK-8212828 to jdk11u and 
> hence backporting the CSR is a prerequisite. The CSR is, however, more or 
> less identical for 11u.

I don't see any explanation of why this should go into jdk11.

--
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Thomas Stüfe]

Hi Andrew,

This would be good to have in jdk11 for the following reason:

The established (since decades) method of forking off on Linux JDKs has
been to use vfork(3). Using vfork is risky. There are chances of crashing
the parent process. The risk of this happening is very low, but not zero.
Since analyzing those crashes and attributing them to vfork(3) is very
difficult, there may be a number of reported cases not associated with
vfork(3) but in fact caused by it.

The patch adds a second, safer way to fork off childs, using
posix_spawn(3). This one has been the default on other platforms for quite
some while. The patch does not change the default fork mode - which remains
vfork(3). So installations would not be affected unless they explicitely
choose to use vfork(3).

Having this possibility would allow us to use posix_spawn(3) in situations
where we are analysing crashes and want to exclude vfork(3) as a cause. It
also would enable us to use posix_spawn(3) as a safe alternative by default
if we choose to do so.

In addition to that, as David Loyd mentioned, vfork(3) is marked as
obsolete and may be vanish at some point in the life span of JDK 11. I
admit the chance of this happening is low.

Kind Regards, Thomas

On Fri, Mar 8, 2019 at 4:20 PM Andrew Haley  wrote:

> On 3/8/19 2:39 PM, Langer, Christoph wrote:
> > please review the CSR backport
> https://bugs.openjdk.java.net/browse/JDK-8220362.
> >
> > It is the backport of https://bugs.openjdk.java.net/browse/JDK-8214511
> to JDK11.
> >
> > We want to backport JDK-8212828<
> https://bugs.openjdk.java.net/browse/JDK-8212828> to jdk11u and hence
> backporting the CSR is a prerequisite. The CSR is, however, more or less
> identical for 11u.
>
> I don't see any explanation of why this should go into jdk11.
>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. 
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[David Lloyd]

Is vfork still guaranteed to be functional by the end of the Java 11
support frame, from the perspective of any organization which supports
JDKs of this version?

On Fri, Mar 8, 2019 at 9:22 AM Andrew Haley  wrote:
>
> On 3/8/19 2:39 PM, Langer, Christoph wrote:
> > please review the CSR backport 
> > https://bugs.openjdk.java.net/browse/JDK-8220362.
> >
> > It is the backport of https://bugs.openjdk.java.net/browse/JDK-8214511 to 
> > JDK11.
> >
> > We want to backport 
> > JDK-8212828 to jdk11u and 
> > hence backporting the CSR is a prerequisite. The CSR is, however, more or 
> > less identical for 11u.
>
> I don't see any explanation of why this should go into jdk11.
>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. 
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



-- 
- DML

Re: [11u]: RFR CSR backport: JDK-8220362: Add "POSIX_SPAWN" as valid value to jdk.lang.Process.launchMechanism on Linux

[Andrew Haley]

On 3/8/19 2:39 PM, Langer, Christoph wrote:
> please review the CSR backport 
> https://bugs.openjdk.java.net/browse/JDK-8220362.
> 
> It is the backport of https://bugs.openjdk.java.net/browse/JDK-8214511 to 
> JDK11.
> 
> We want to backport 
> JDK-8212828 to jdk11u and 
> hence backporting the CSR is a prerequisite. The CSR is, however, more or 
> less identical for 11u.

I don't see any explanation of why this should go into jdk11.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671