Re: Question about fatal JavaFX crashes

[Daniel Peintner]

All,

Thank you very much for all the comments/suggestions.
I have been learning a lot about JVM C2 compilation the last couple of days.

BTW, in the meanwhile a similar bug has been reported with Debian Linux and
has been added to https://bugs.openjdk.java.net/browse/JDK-828338
.

I would like to briefly summarize what my solution is for the time being.
Since in my case stability is *much* more important than speed I used a
rather radical approach to prohibit C2 to do any *aggressive* optimization.

-XX:+TieredCompilation   (enable C1)
-XX:TieredStopAtLevel=1 (disable C2)

It seems somewhat similar to the out-dated "-client" option.
The application didn't really feel any slower. However, that certainly
depends heavily on the use case.

Note: I very much look forward to the day the GraalVM compiler gets the
default.

Hope this might be useful for others.

Thanks,

-- Daniel




On Fri, Mar 18, 2022 at 11:10 PM Scott Palmer  wrote:

> I wonder, could you do the opposite and force compilation to trigger the
> bug more consistently?
>
> Scott
>
> > On Mar 18, 2022, at 2:03 PM, Philip Race  wrote:
> >
> > I have at least seen  JIT compiler crashes like this in other unrelated
> cases ..
> >
> > in theory you can use an option like
> >
> > -XX:CompileCommand=exclude,javafx/scene/control/TableView$5::onChanged
> >
> > although I am very unsure about the syntax for the last part of it
> especially with what looks like
> > some anonymous method or lambda expression. I can almost guarantee that
> my example is wrong.
> >
> > Excluding the whole of TableView might be easier to specify but will
> slow you down a lot.
> >
> > -phil.
> >
> >
> >> On 3/18/22 10:37 AM, Kevin Rushforth wrote:
> >> Thanks, Phil. That was my take as well. I don't see how this can be a
> JavaFX bug given where it is crashing. FWIW, I haven't ever seen anything
> like this.
> >>
> >> -- Kevin
> >>
> >>
> >>> On 3/18/2022 10:25 AM, Philip Race wrote:
> >>> I think it is probable that this is a hotspot VM problem in the C2 JIT
> compiler code.
> >>>
> >>> I've moved your bug report to hotspot :
> https://bugs.openjdk.java.net/browse/JDK-8283386
> >>> The interesting question isn't about which version of FX used to work.
> >>> It is what was the last working version of the JDK.
> >>> It looks a bit like a JDK 17 bug from the evidence so far so if
> >>> you were running FX 17 on JDK 17 GA you maybe have picked up a later
> update release  of JDK 17 along with FX 18 ??
> >>>
> >>> Regardless this doesn't look like an FX bug.
> >>> But unless you can actually provide a test case, or, by luck the
> hotspot folks recognise the issue,
> >>> I don't know what can/will happen.
> >>>
> >>> -phil.
> >>>
> >>> On 3/18/22 7:08 AM, Daniel Peintner wrote:
>  Hi John, all,
> 
>  Thanks for your detailed reply.
>  I submitted a bug report with detailed information.
> 
>  - Run a different Java version
>  I tried different versions and vendors with the same result.
>  * OpenJDK 17.0.1
>  * Zulu 17.0.2
>  * I wanted to check also JDK18-ea but gradle does not yet support it
> 
> 
> > - Try switching to a different GC
> > - Use different VM options (are you using anything special?)
> > - Anything else that is not often used, non-standard or experimental,
> > try going to a more common setup
> >
>  I use no specific setting, all is default.
>  I have been reading that the -client flag might help in some cases but
>  unfortunately this flag is no longer taken into account for 64bit
> JVMS.
> 
>  Thanks,
> 
>  -- Daniel
> 
> 
> 
> 
> > --John
> >
> > On 18/03/2022 09:43, Daniel Peintner wrote:
> >> Hello,
> >>
> >> I take the liberty to ask on the email reflector if there are other
> > people
> >> with similar problems.
> >>
> >> Since updating my application to JavaFX18 I get random fatal
> crashes.
> >> Unfortunately it is not predictable but after some time the app
> crashes
> >> with "EXCEPTION_ACCESS_VIOLATION".
> >>
> >> The hs_error_pidX reports the following.
> >>
> >>
> >> ---  T H R E A D  ---
> >>
> >> Current thread (0x016be9c9b410):  JavaThread "C2
> CompilerThread0"
> >> daemon [_thread_in_native, id=3068,
> >> stack(0x00393e80,0x00393e90)]
> >>
> >> Current CompileTask:
> >> C2:10515286 18185   4
> >   javafx.scene.control.TableView$5::onChanged
> >> (1049 bytes)
> >>
> >>
> >> Am I the only one seeing this?
> >> I am not sure if this relates to changing JavaFX 17 to 18 or
> whether it
> >> makes it just more likely.
> >>
> >> It seems to be related to
> "javafx.scene.control.TableView$5::onChanged"
> >> since all crashes show this line.
> >>
> >> Having said that, I cannot provide a test-case 

Re: Question about fatal JavaFX crashes

[Scott Palmer]

I wonder, could you do the opposite and force compilation to trigger the bug 
more consistently?

Scott

> On Mar 18, 2022, at 2:03 PM, Philip Race  wrote:
> 
> I have at least seen  JIT compiler crashes like this in other unrelated 
> cases ..
> 
> in theory you can use an option like
> 
> -XX:CompileCommand=exclude,javafx/scene/control/TableView$5::onChanged
> 
> although I am very unsure about the syntax for the last part of it especially 
> with what looks like
> some anonymous method or lambda expression. I can almost guarantee that my 
> example is wrong.
> 
> Excluding the whole of TableView might be easier to specify but will slow you 
> down a lot.
> 
> -phil.
> 
> 
>> On 3/18/22 10:37 AM, Kevin Rushforth wrote:
>> Thanks, Phil. That was my take as well. I don't see how this can be a JavaFX 
>> bug given where it is crashing. FWIW, I haven't ever seen anything like this.
>> 
>> -- Kevin
>> 
>> 
>>> On 3/18/2022 10:25 AM, Philip Race wrote:
>>> I think it is probable that this is a hotspot VM problem in the C2 JIT 
>>> compiler code.
>>> 
>>> I've moved your bug report to hotspot : 
>>> https://bugs.openjdk.java.net/browse/JDK-8283386
>>> The interesting question isn't about which version of FX used to work.
>>> It is what was the last working version of the JDK.
>>> It looks a bit like a JDK 17 bug from the evidence so far so if
>>> you were running FX 17 on JDK 17 GA you maybe have picked up a later update 
>>> release  of JDK 17 along with FX 18 ??
>>> 
>>> Regardless this doesn't look like an FX bug.
>>> But unless you can actually provide a test case, or, by luck the hotspot 
>>> folks recognise the issue,
>>> I don't know what can/will happen.
>>> 
>>> -phil.
>>> 
>>> On 3/18/22 7:08 AM, Daniel Peintner wrote:
 Hi John, all,
 
 Thanks for your detailed reply.
 I submitted a bug report with detailed information.
 
 - Run a different Java version
 I tried different versions and vendors with the same result.
 * OpenJDK 17.0.1
 * Zulu 17.0.2
 * I wanted to check also JDK18-ea but gradle does not yet support it
 
 
> - Try switching to a different GC
> - Use different VM options (are you using anything special?)
> - Anything else that is not often used, non-standard or experimental,
> try going to a more common setup
> 
 I use no specific setting, all is default.
 I have been reading that the -client flag might help in some cases but
 unfortunately this flag is no longer taken into account for 64bit JVMS.
 
 Thanks,
 
 -- Daniel
 
 
 
 
> --John
> 
> On 18/03/2022 09:43, Daniel Peintner wrote:
>> Hello,
>> 
>> I take the liberty to ask on the email reflector if there are other
> people
>> with similar problems.
>> 
>> Since updating my application to JavaFX18 I get random fatal crashes.
>> Unfortunately it is not predictable but after some time the app crashes
>> with "EXCEPTION_ACCESS_VIOLATION".
>> 
>> The hs_error_pidX reports the following.
>> 
>> 
>> ---  T H R E A D  ---
>> 
>> Current thread (0x016be9c9b410):  JavaThread "C2 CompilerThread0"
>> daemon [_thread_in_native, id=3068,
>> stack(0x00393e80,0x00393e90)]
>> 
>> Current CompileTask:
>> C2:10515286 18185   4
>   javafx.scene.control.TableView$5::onChanged
>> (1049 bytes)
>> 
>> 
>> Am I the only one seeing this?
>> I am not sure if this relates to changing JavaFX 17 to 18 or whether it
>> makes it just more likely.
>> 
>> It seems to be related to "javafx.scene.control.TableView$5::onChanged"
>> since all crashes show this line.
>> 
>> Having said that, I cannot provide a test-case since it happens all of a
>> sudden and sometimes after hours using the application.
>> 
>> I am open to any feedback or input on how to proceed best.
>> 
>> Thanks!
>> 
>> -- Daniel
>>> 
>> 
> 

Re: Question about fatal JavaFX crashes

[Philip Race]

I have at least seen  JIT compiler crashes like this in other unrelated 
cases ..


in theory you can use an option like

-XX:CompileCommand=exclude,javafx/scene/control/TableView$5::onChanged

although I am very unsure about the syntax for the last part of it 
especially with what looks like
some anonymous method or lambda expression. I can almost guarantee that 
my example is wrong.


Excluding the whole of TableView might be easier to specify but will 
slow you down a lot.


-phil.


On 3/18/22 10:37 AM, Kevin Rushforth wrote:
Thanks, Phil. That was my take as well. I don't see how this can be a 
JavaFX bug given where it is crashing. FWIW, I haven't ever seen 
anything like this.


-- Kevin


On 3/18/2022 10:25 AM, Philip Race wrote:
I think it is probable that this is a hotspot VM problem in the C2 
JIT compiler code.


I've moved your bug report to hotspot : 
https://bugs.openjdk.java.net/browse/JDK-8283386

The interesting question isn't about which version of FX used to work.
It is what was the last working version of the JDK.
It looks a bit like a JDK 17 bug from the evidence so far so if
you were running FX 17 on JDK 17 GA you maybe have picked up a later 
update release  of JDK 17 along with FX 18 ??


Regardless this doesn't look like an FX bug.
But unless you can actually provide a test case, or, by luck the 
hotspot folks recognise the issue,

I don't know what can/will happen.

-phil.

On 3/18/22 7:08 AM, Daniel Peintner wrote:

Hi John, all,

Thanks for your detailed reply.
I submitted a bug report with detailed information.

- Run a different Java version
I tried different versions and vendors with the same result.
* OpenJDK 17.0.1
* Zulu 17.0.2
* I wanted to check also JDK18-ea but gradle does not yet support it



- Try switching to a different GC
- Use different VM options (are you using anything special?)
- Anything else that is not often used, non-standard or experimental,
try going to a more common setup


I use no specific setting, all is default.
I have been reading that the -client flag might help in some cases but
unfortunately this flag is no longer taken into account for 64bit JVMS.

Thanks,

-- Daniel





--John

On 18/03/2022 09:43, Daniel Peintner wrote:

Hello,

I take the liberty to ask on the email reflector if there are other

people

with similar problems.

Since updating my application to JavaFX18 I get random fatal crashes.
Unfortunately it is not predictable but after some time the app 
crashes

with "EXCEPTION_ACCESS_VIOLATION".

The hs_error_pidX reports the following.


---  T H R E A D  ---

Current thread (0x016be9c9b410):  JavaThread "C2 CompilerThread0"
daemon [_thread_in_native, id=3068,
stack(0x00393e80,0x00393e90)]

Current CompileTask:
C2:10515286 18185   4

  javafx.scene.control.TableView$5::onChanged

(1049 bytes)


Am I the only one seeing this?
I am not sure if this relates to changing JavaFX 17 to 18 or 
whether it

makes it just more likely.

It seems to be related to 
"javafx.scene.control.TableView$5::onChanged"

since all crashes show this line.

Having said that, I cannot provide a test-case since it happens 
all of a

sudden and sometimes after hours using the application.

I am open to any feedback or input on how to proceed best.

Thanks!

-- Daniel

Re: Question about fatal JavaFX crashes

[Kevin Rushforth]

Thanks, Phil. That was my take as well. I don't see how this can be a 
JavaFX bug given where it is crashing. FWIW, I haven't ever seen 
anything like this.


-- Kevin


On 3/18/2022 10:25 AM, Philip Race wrote:
I think it is probable that this is a hotspot VM problem in the C2 JIT 
compiler code.


I've moved your bug report to hotspot : 
https://bugs.openjdk.java.net/browse/JDK-8283386

The interesting question isn't about which version of FX used to work.
It is what was the last working version of the JDK.
It looks a bit like a JDK 17 bug from the evidence so far so if
you were running FX 17 on JDK 17 GA you maybe have picked up a later 
update release  of JDK 17 along with FX 18 ??


Regardless this doesn't look like an FX bug.
But unless you can actually provide a test case, or, by luck the 
hotspot folks recognise the issue,

I don't know what can/will happen.

-phil.

On 3/18/22 7:08 AM, Daniel Peintner wrote:

Hi John, all,

Thanks for your detailed reply.
I submitted a bug report with detailed information.

- Run a different Java version
I tried different versions and vendors with the same result.
* OpenJDK 17.0.1
* Zulu 17.0.2
* I wanted to check also JDK18-ea but gradle does not yet support it



- Try switching to a different GC
- Use different VM options (are you using anything special?)
- Anything else that is not often used, non-standard or experimental,
try going to a more common setup


I use no specific setting, all is default.
I have been reading that the -client flag might help in some cases but
unfortunately this flag is no longer taken into account for 64bit JVMS.

Thanks,

-- Daniel





--John

On 18/03/2022 09:43, Daniel Peintner wrote:

Hello,

I take the liberty to ask on the email reflector if there are other

people

with similar problems.

Since updating my application to JavaFX18 I get random fatal crashes.
Unfortunately it is not predictable but after some time the app 
crashes

with "EXCEPTION_ACCESS_VIOLATION".

The hs_error_pidX reports the following.


---  T H R E A D  ---

Current thread (0x016be9c9b410):  JavaThread "C2 CompilerThread0"
daemon [_thread_in_native, id=3068,
stack(0x00393e80,0x00393e90)]

Current CompileTask:
C2:10515286 18185   4

  javafx.scene.control.TableView$5::onChanged

(1049 bytes)


Am I the only one seeing this?
I am not sure if this relates to changing JavaFX 17 to 18 or 
whether it

makes it just more likely.

It seems to be related to 
"javafx.scene.control.TableView$5::onChanged"

since all crashes show this line.

Having said that, I cannot provide a test-case since it happens all 
of a

sudden and sometimes after hours using the application.

I am open to any feedback or input on how to proceed best.

Thanks!

-- Daniel

Re: Question about fatal JavaFX crashes

[Philip Race]

I think it is probable that this is a hotspot VM problem in the C2 JIT 
compiler code.


I've moved your bug report to hotspot : 
https://bugs.openjdk.java.net/browse/JDK-8283386

The interesting question isn't about which version of FX used to work.
It is what was the last working version of the JDK.
It looks a bit like a JDK 17 bug from the evidence so far so if
you were running FX 17 on JDK 17 GA you maybe have picked up a later 
update release  of JDK 17 along with FX 18 ??


Regardless this doesn't look like an FX bug.
But unless you can actually provide a test case, or, by luck the hotspot 
folks recognise the issue,

I don't know what can/will happen.

-phil.

On 3/18/22 7:08 AM, Daniel Peintner wrote:

Hi John, all,

Thanks for your detailed reply.
I submitted a bug report with detailed information.

- Run a different Java version
I tried different versions and vendors with the same result.
* OpenJDK 17.0.1
* Zulu 17.0.2
* I wanted to check also JDK18-ea but gradle does not yet support it



- Try switching to a different GC
- Use different VM options (are you using anything special?)
- Anything else that is not often used, non-standard or experimental,
try going to a more common setup


I use no specific setting, all is default.
I have been reading that the -client flag might help in some cases but
unfortunately this flag is no longer taken into account for 64bit JVMS.

Thanks,

-- Daniel





--John

On 18/03/2022 09:43, Daniel Peintner wrote:

Hello,

I take the liberty to ask on the email reflector if there are other

people

with similar problems.

Since updating my application to JavaFX18 I get random fatal crashes.
Unfortunately it is not predictable but after some time the app crashes
with "EXCEPTION_ACCESS_VIOLATION".

The hs_error_pidX reports the following.


---  T H R E A D  ---

Current thread (0x016be9c9b410):  JavaThread "C2 CompilerThread0"
daemon [_thread_in_native, id=3068,
stack(0x00393e80,0x00393e90)]

Current CompileTask:
C2:10515286 18185   4

  javafx.scene.control.TableView$5::onChanged

(1049 bytes)


Am I the only one seeing this?
I am not sure if this relates to changing JavaFX 17 to 18 or whether it
makes it just more likely.

It seems to be related to "javafx.scene.control.TableView$5::onChanged"
since all crashes show this line.

Having said that, I cannot provide a test-case since it happens all of a
sudden and sometimes after hours using the application.

I am open to any feedback or input on how to proceed best.

Thanks!

-- Daniel

Re: Question about fatal JavaFX crashes

[Daniel Peintner]

Hi John, all,

Thanks for your detailed reply.
I submitted a bug report with detailed information.

- Run a different Java version
>

I tried different versions and vendors with the same result.
* OpenJDK 17.0.1
* Zulu 17.0.2
* I wanted to check also JDK18-ea but gradle does not yet support it


> - Try switching to a different GC
> - Use different VM options (are you using anything special?)
> - Anything else that is not often used, non-standard or experimental,
> try going to a more common setup
>

I use no specific setting, all is default.
I have been reading that the -client flag might help in some cases but
unfortunately this flag is no longer taken into account for 64bit JVMS.

Thanks,

-- Daniel




>
> --John
>
> On 18/03/2022 09:43, Daniel Peintner wrote:
> > Hello,
> >
> > I take the liberty to ask on the email reflector if there are other
> people
> > with similar problems.
> >
> > Since updating my application to JavaFX18 I get random fatal crashes.
> > Unfortunately it is not predictable but after some time the app crashes
> > with "EXCEPTION_ACCESS_VIOLATION".
> >
> > The hs_error_pidX reports the following.
> >
> >
> > ---  T H R E A D  ---
> >
> > Current thread (0x016be9c9b410):  JavaThread "C2 CompilerThread0"
> > daemon [_thread_in_native, id=3068,
> > stack(0x00393e80,0x00393e90)]
> >
> > Current CompileTask:
> > C2:10515286 18185   4
>  javafx.scene.control.TableView$5::onChanged
> > (1049 bytes)
> >
> >
> > Am I the only one seeing this?
> > I am not sure if this relates to changing JavaFX 17 to 18 or whether it
> > makes it just more likely.
> >
> > It seems to be related to "javafx.scene.control.TableView$5::onChanged"
> > since all crashes show this line.
> >
> > Having said that, I cannot provide a test-case since it happens all of a
> > sudden and sometimes after hours using the application.
> >
> > I am open to any feedback or input on how to proceed best.
> >
> > Thanks!
> >
> > -- Daniel
>

Re: Question about fatal JavaFX crashes

[John Hendrikx]

I haven't seen this one; the code in TableView in onChanged hasn't had 
any updates 6 years.


I have my doubts this is a JavaFX problem as it seems the Compiler is 
crashing here in a piece of native code.  It's possible this problem is 
only occuring on systems where the compiler decides it needs to compile 
(or recompile) this method, so at a minimum you'd need to make use of 
TableView in a way to hits that code path often enough to trigger 
compilation.


Oracle provides some documentation on how to deal with what looks like a 
compiler bug, please read here:


https://docs.oracle.com/en/java/javase/17/troubleshoot/troubleshoot-system-crashes.html

Further, I think you should report this problem with as much details as 
possible on bugs.openjdk.java.net -- you can look for similar bugs to 
see what kind of detail and settings would be helpful to debug this.


Other things you could try to solve it yourself:

- Run a different Java version
- Try switching to a different GC
- Use different VM options (are you using anything special?)
- Anything else that is not often used, non-standard or experimental, 
try going to a more common setup


--John

On 18/03/2022 09:43, Daniel Peintner wrote:

Hello,

I take the liberty to ask on the email reflector if there are other people
with similar problems.

Since updating my application to JavaFX18 I get random fatal crashes.
Unfortunately it is not predictable but after some time the app crashes
with "EXCEPTION_ACCESS_VIOLATION".

The hs_error_pidX reports the following.


---  T H R E A D  ---

Current thread (0x016be9c9b410):  JavaThread "C2 CompilerThread0"
daemon [_thread_in_native, id=3068,
stack(0x00393e80,0x00393e90)]

Current CompileTask:
C2:10515286 18185   4   javafx.scene.control.TableView$5::onChanged
(1049 bytes)


Am I the only one seeing this?
I am not sure if this relates to changing JavaFX 17 to 18 or whether it
makes it just more likely.

It seems to be related to "javafx.scene.control.TableView$5::onChanged"
since all crashes show this line.

Having said that, I cannot provide a test-case since it happens all of a
sudden and sometimes after hours using the application.

I am open to any feedback or input on how to proceed best.

Thanks!

-- Daniel

Question about fatal JavaFX crashes

[Daniel Peintner]

Hello,

I take the liberty to ask on the email reflector if there are other people
with similar problems.

Since updating my application to JavaFX18 I get random fatal crashes.
Unfortunately it is not predictable but after some time the app crashes
with "EXCEPTION_ACCESS_VIOLATION".

The hs_error_pidX reports the following.


---  T H R E A D  ---

Current thread (0x016be9c9b410):  JavaThread "C2 CompilerThread0"
daemon [_thread_in_native, id=3068,
stack(0x00393e80,0x00393e90)]

Current CompileTask:
C2:10515286 18185   4   javafx.scene.control.TableView$5::onChanged
(1049 bytes)


Am I the only one seeing this?
I am not sure if this relates to changing JavaFX 17 to 18 or whether it
makes it just more likely.

It seems to be related to "javafx.scene.control.TableView$5::onChanged"
since all crashes show this line.

Having said that, I cannot provide a test-case since it happens all of a
sudden and sometimes after hours using the application.

I am open to any feedback or input on how to proceed best.

Thanks!

-- Daniel