Re: CVE-2023-32200

[Brandon Sara]

Awesome! Thanks for the quick response

> On Jul 20, 2023, at 11:13 AM, Andy Seaborne  wrote:
>
> "EXTERNAL EMAIL" – Always use caution when reviewing mail from outside of the 
> organization.
>
>
>
> On 20/07/2023 17:18, Brandon Sara wrote:
>> I just came across CVE-2023-32200 and was wondering, is it different than 
>> CVE-2023-22665 and, if so, how is it different?
>
>
> Jena 4.8.0 addresses CVE-2023-22665 by requiring the Java system property 
> "jena:scripting" to enable scripting.
>
> Jena 4.9.0 addresses CVE-2023-32200 which happens if scripting is enabled 
> (4.8.0). The change goes further than only addressing the security issue by 
> requiring script functions to be in an "allowed" list; that is, there is an 
> API contract for callable scripts. Other functions in the script file are not 
> callable which should help development.
>
> Running Java17 means there is no scripting engine unless the deployment
> has added one. Java11 has a scriting engine in the JDK.
>
>Andy
>

No PHI in Email: PointClickCare and Collective Medical, A PointClickCare 
Company, policies prohibit sending protected health information (PHI) by email, 
which may violate regulatory requirements. If sending PHI is necessary, please 
contact the sender for secure delivery instructions.

Confidentiality Notice: This email message, including any attachments, is for 
the sole use of the intended recipient(s) and may contain confidential and 
privileged information. Any unauthorized review, use, disclosure or 
distribution is prohibited. If you are not the intended recipient, please 
contact the sender by reply email and destroy all copies of the original 
message.

Required Java version

[Andy Seaborne]

A reminder that Jena will move from requiring Java11 to requiring Java17.

The project aims to support for 2 LTS versions of Java.
Java21 is scheduled for September 19 this year and is LTS.

Some time after that date that Jena will switch, with a major version 
bump to Jena 5.x.y


For Fuseki, Jena5 will use jakarta.* packages.

Jena is already tested for Java17 and Java21_latest_EA on a regular 
basis. You can switch JVMs now.


Andy

Re: CVE-2023-32200

[Andy Seaborne]

On 20/07/2023 17:18, Brandon Sara wrote:


I just came across CVE-2023-32200 and was wondering, is it different than 
CVE-2023-22665 and, if so, how is it different?



Jena 4.8.0 addresses CVE-2023-22665 by requiring the Java system 
property "jena:scripting" to enable scripting.


Jena 4.9.0 addresses CVE-2023-32200 which happens if scripting is 
enabled (4.8.0). The change goes further than only addressing the 
security issue by requiring script functions to be in an "allowed" list; 
that is, there is an API contract for callable scripts. Other functions 
in the script file are not callable which should help development.


Running Java17 means there is no scripting engine unless the deployment
has added one. Java11 has a scriting engine in the JDK.

Andy

CVE-2023-32200

[Brandon Sara]

I just came across CVE-2023-32200 and was wondering, is it different than 
CVE-2023-22665 and, if so, how is it different?
No PHI in Email: PointClickCare and Collective Medical, A PointClickCare 
Company, policies prohibit sending protected health information (PHI) by email, 
which may violate regulatory requirements. If sending PHI is necessary, please 
contact the sender for secure delivery instructions.

Confidentiality Notice: This email message, including any attachments, is for 
the sole use of the intended recipient(s) and may contain confidential and 
privileged information. Any unauthorized review, use, disclosure or 
distribution is prohibited. If you are not the intended recipient, please 
contact the sender by reply email and destroy all copies of the original 
message.

RE: Re: Mystery memory leak in fuseki

[Conal McLaughlin]

Hey Andy, 

Metaspace seems to be stable!

We’re running this on Java 11 currently.
I can check it out with Java17 though.

We’ve currently set Xms/Xmx to 2560MB & MaxMetaspaceSize to 256MB.

The ECS task is set with a ceiling of 4GB Memory & 1vcpu.

Could it be more of a race condition than size of used objects, due to logging?
I do see some time sensitive eviction code in Jetty - 
https://github.com/eclipse/jetty.project/blob/9e16d81cf8922c75e3d2d96c66442b896a9c69e1/jetty-io/src/main/java/org/eclipse/jetty/io/ArrayRetainableByteBufferPool.java#L374
Not sure if the same type of thing exists in the Jena codebase.

I will try to check with `—empty` also.


Thanks,
Conal

On 2023/07/19 21:10:24 Andy Seaborne wrote:
> Conal,
> 
> Thanks for the information.
> Can you see if metaspace is growing as well?
> 
> All,
> 
> Could someone please try running Fuseki main, with no datasets (--empty) 
> with some healthcheck ping traffic.
> 
>  Andy
> 
> On 19/07/2023 14:42, Conal McLaughlin wrote:
> > Hey Dave,
> > 
> > Thank you for providing an in depth analysis of your issues.
> > We appear to be witnessing the same type of problems with our current 
> > Fuseki deployment.
> > We are deploying a containerised Fuseki into a AWS ECS task alongside 
> > other containers - this may not be ideal but that’s a different story.
> > 
> > I just wanted to add another data point to everything you have described.
> > Firstly, it does seem like “idle” (or very low traffic) instances are 
> > the problem, for us (coupled with a larger heap than necessary).
> > We witness the same increase in the ECS task memory consumption up until 
> > the whole thing is killed off. Which includes the Fuseki container.
> > 
> > In an attempt to see what was going on beneath the hood, we turned up 
> > the logging to TRACE in the log4j2.xml file provided to Fuseki.
> > This appeared to stabilise the increasing memory consumption.
> > Even just switching the `logger.jetty.level` to TRACE alleviates the issue.
> 
> Colour me confused!
> 
> A Log4j logger that is active will use a few objects - may that's enough 
> to trigger a minor GC which in turn is enough to flush some non-heap 
> resources.
> 
> How big is the heap?
> This is Java17?
> 
> > We are testing this on Fuseki 4.8.0/TDB2 with close to 0 triples and 
> > extremely low query traffic / health checks via /ping.
> > KPk7uhH2F9Lp.png
> > ecs-task-memory - Image on Pasteboard 
> > 
> > pasteboard.co 
> > 
> > 
> > 
> > 
> > Cheers,
> > Conal
> > 
> > On 2023/07/11 09:31:25 Dave Reynolds wrote:
> >  > Hi Rob,
> >  >
> >  > Good point. Will try to find time to experiment with that but given the
> >  > testing cycle time that will take a while and can't start immediately.
> >  >
> >  > I'm a little sceptical though. As mentioned before, all the metrics we
> >  > see show the direct memory pool that Jetty uses cycling up the max heap
> >  > size and then being collected but with no long term growth to match the
> >  > process size growth. This really feels more like a bug (though not sure
> >  > where) than tuning. The fact that actual behaviour doesn't match the
> >  > documentation isn't encouraging.
> >  >
> >  > It's also pretty hard to figure what the right pool configuration would
> >  > be. This thing is just being asked to deliver a few metrics (12KB per
> >  > request) several times a minute but manages to eat 500MB of direct
> >  > buffer space every 5mins. So what the right pool parameters are to
> >  > support real usage peaks is not going to be easy to figure out.
> >  >
> >  > None the less you are right. That's something that should be explored.
> >  >
> >  > Dave
> >  >
> >  >
> >  > On 11/07/2023 09:45, Rob @ DNR wrote:
> >  > > Dave
> >  > >
> >  > > Thanks for the further information.
> >  > >
> >  > > Have you experimented with using Jetty 10 but providing more 
> > detailed configuration?Fuseki supports providing detailed Jetty 
> > configuration if needed via the --jetty-config option
> >  > >
> >  > > The following section look relevant:
> >  > >
> >  > > 
> > https://eclipse.dev/jetty/documentation/jetty-10/operations-guide/index.html#og-module-bytebufferpool
> >  > >
> >  > > It looks like the default is that Jetty uses a heuristic to 
> > determine these values, sadly the heuristic in question is not detailed 
> > in that documentation.
> >  > >
> >  > > Best guess from digging through their code is that the “heuristic” 
> > is this:
> >  > >
> >  > > 
> > https://github.com/eclipse/jetty.project/blob/jetty-10.0.x/jetty-io/src/main/java/org/eclipse/jetty/io/AbstractByteBufferPool.java#L78-L84
> >  > >
> >  > > i.e., ¼ of the configured max heap size.This doesn’t necessarily 
> > align with the exact sizes of process growth you see but I note the 
> > documentation does explicitly say that buffers used can go beyond these 
> > limits but that