On 26/10/17 22:42, Bruno P. Kinoshita wrote:
Hi Andy,
Good to know that @Rule/tmpFolder isolation address the problem. It >might
well be the best long term, but the effect of disk space that has
been a problem for TDB1 might arise.
But perhaps it is safe as the folders are normally deleted after the test
execution?
Yes - after 3.5.0 time to go back and make sure the test coverage does
accidentally have tests that make use of being serial.
The catch is on Windows, with mmap files, the space goes away after the
JVM exits.
With a lot of test databases, the peak space is quite large.
(And for some crash conditions a couple of times, we overflow the
Jenkins windows slaves. I recall it hit you a couple of years ago)
"The TemporaryFolder Rule allows creation of files and folders that should be
deleted when the test method finishes (whether it passes or fails). Whether the deletion
is successful or not is not checked by this rule. No exception will be thrown in case the
deletion fails."
http://junit.org/junit4/javadoc/4.12/org/junit/rules/TemporaryFolder.html
Could people try this please? For OSX and Windows.
Sure thing, thanks for preparing the fix. Tested on the same Win machine as
yesterday. Same problem. Reported in the pull request in GitHub.
Yes - ProcessFileLock (although 5 failures is now 6 ... which oddly is
good as they are all the identical)
Did your fix make that change for you?
TestDatabaseOps with the @Rule (or ConfigTest.getCleanDir() which is
Windows/non-windows sensitive for the space issues)
I'm concerned this is the sign of wider issue, which is why I tried to
change the POM for several modules.
Let's continue on GH.
And what exactly are the default settings for Surefire?
reuseForks is true so there is also a chance that it is reusing the
forked VM is causing contamination via statics and open file descriptors.
I believe by default surefire uses multiple threads. And the number of threads
is defined by the number of cores. For all the settings plus their default
values in the project, we can run
mvn surefire:help -Ddetail=true
mvn surefire:help -Ddetail=true -Dgoal=test
Not sure if it helps much.
It's the setting of 'parallel' I don't see the default for.
On PR#297, Rob is setting that to 'classes', and 'threadCount' to 4 and
it makes a difference
perCoreThreadCount defaults to true which helps cope with different
machines.
Andy
Cheers
Bruno
________________________________
From: Andy Seaborne <[email protected]>
To: [email protected]
Sent: Friday, 27 October 2017 1:34 AM
Subject: Re: @Test TestDatabaseOps.compact_prefixes_3 [Was Re: [] Release
Apache Jena 3.5.0]
Good to know that @Rule/tmpFolder isolation address the problem. It
might well be the best long term, but the effect of disk space that has
been a problem for TDB1 might arise.
On PR#296, I've put an alternative that applies to the other modules
that touch disk (maybe it sorts out out TestProcessFileLock - it also
adds @Before testing for TestProcessFileLock.
Could people try this please? For OSX and Windows.
And what exactly are the default settings for Surefire?
reuseForks is true so there is also a chance that it is reusing the
forked VM is causing contamination via statics and open file descriptors.
Andy
On 26/10/17 09:27, Bruno P. Kinoshita wrote:
Using a temporary folder instead of a pre-defined one, and changing to a
instance field, fixed the issue for me.
https://github.com/apache/jena/pull/295
I found no other errors. So in case the pull request solution is not
appropriate, I'd still be happy to vote +1 in the other thread. Though I'm not
sure what to do about the Windows test failures. Ignore them all?
CheersBruno
From: Andy Seaborne <[email protected]>
To: [email protected]
Sent: Thursday, 26 October 2017 10:49 AM
Subject: Re: @Test TestDatabaseOps.compact_prefixes_3 [Was Re: [] Release
Apache Jena 3.5.0]
On 25/10/17 22:39, Bruno P. Kinoshita wrote:
ps : mvn clean test install -Pdev would stop in the Jena base I think,
complaining it couldn't find the jena-guava-shaded module... but mvn clean
install worked. Just for what's worth.
Use -Pbootstrap if it is a clean build, or if it tries to pick up
snapshots from the development deploy snapshots.
I'm sure that latter is a new "feature" of maven.
I think we ought to collapse -Pdev and -Pbootstrap. It's not a much of
a time difference.
Andy
