Changing the tests to make them run isn't testing any more.
We have been through this issues many times now. It's a nuisance, but
we have Jenkins, though the Windows slaves are less stable and we
sometimes have to disable the job.
Helping out with the Windows slaves is a *much* better solution.
I reviewed what had been run recently enough, together with noting
changes in TDB (none relevant) when I was doing the build to be happy
that testing had been done at some point.
If someone has a better solution - great. It has to be a proper
solution though and, as reading the Java bug reports shows, "works for
me at the moment" does not reliably mean "works", for everyone or in the
future.
There are several different JVMs.
Andy
On 10/03/15 11:14, Stian Soiland-Reyes wrote:
The System.gc() is not guaranteed to work - but it did work for me.
That is what I meant by a 'best effort' within the test. The test does
not require all those old test folders to hang around.
Could we do a workaround for the test to set say a tinier block-size
instead of 8 MB?
As I have been unable to build all of Jena on Windows - what is the
actual disk space requirement? It should at least be documented in the
README.
On 10 March 2015 at 11:07, Rob Vesse <[email protected]> wrote:
Using an alternative approach would not make any difference
It is a fundamental bug in Windows memory mapped files that means that a
JVM can never guarantee to completely release memory mapped files while
the JVM is alive.
Andy has posted this many times on threads about TDB on Windows in the
past. No workaround we could attempt could ever solve the issue on
Windows so there is really no point in expending effort changing something
low level that otherwise works fine across multiple platforms.
Rob
On 10/03/2015 10:25, "Stian Soiland-Reyes" <[email protected]> wrote:
Thanks, Rob!
I tried looking yesterday at ways to reduce the disk space
requirements when building on Windows - including truncating the files
after closing. This seems to require deep changes into TDBs
ChannelManager which keeps the corresponding FileChannels - perhaps a
new method for that purpose?
https://github.com/apache/jena/blob/master/jena-tdb/src/main/java/com/hp/h
pl/jena/tdb/base/file/ChannelManager.java
It seems on Windows with Oracle/OpenJDK you can call System.gc() to
(hopefully) release the ByteBuffers that lock the memory regions (and
then making the files deletable) - but this adds a significant
overhead. The dispose methods on ByteBufferImpls are not easily
accessible - you would need some introspection hackery to get hold of
that cleaner() and that would of course only work on Oracle/OpenJDK.
as fc.map() still does the same thing.
Close your eyes - GPL3!
https://github.com/stain/jdk8u/blob/master/src/share/classes/java/nio/Dire
ct-X-Buffer.java.template#L72
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b
132/java/nio/DirectByteBuffer.java/#72
I tried using the FileChannels from JDK7 NIO2 (e.g.
FileChannel.open(Path)) instead of through RandomAccessFile - but it
did not make any difference
Perhaps System.gc() is not worth it in general (* on Windows) when
closing a dataset - I tried to modify the ChannelManager to always do
this on release, it meant each test in jena-jdbc-tdb took 1.5s instead
of 0.2s, but it did allow me to delete the used folders from target/
while the JVM/test was running.
For the tests we could do something like for every 10 tests do
System.gc() and wipe the old data.
Perhaps Fuseki 2 could do System.gc() on [Remove] && SystemTDB.isWindows.
On 10 March 2015 at 10:00, ASF GitHub Bot (JIRA) <[email protected]> wrote:
[
https://issues.apache.org/jira/browse/JENA-897?page=com.atlassian.jira.pl
ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354617#com
ment-14354617 ]
ASF GitHub Bot commented on JENA-897:
-------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/jena/pull/41
jena-jdbc-tdb tests use %TEMP% instead of target/
-------------------------------------------------
Key: JENA-897
URL: https://issues.apache.org/jira/browse/JENA-897
Project: Apache Jena
Issue Type: Bug
Components: JDBC
Affects Versions: Jena 2.12.1, Jena 2.13.0
Environment: Windowx 8.0 x64, C: with 34 GB free
Reporter: Stian Soiland-Reyes
Priority: Critical
Fix For: Jena 2.13.1
.. and thus mvn clean install on Windows will easily consume 37 GB on
C: and run out of disk space - even if Jena is built on a larger
partition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
