On 29/10/13 08:02, Nikolaos Konstantinou wrote:
Dear all,
I am trying to write an RDF file to disk, using the following code:
Which version of Jena is this?
resultModel.write(new BufferedWriter(new FileWriter("file.rdf")), "N3");
Aside: Better top flush the BufferedWriter explicitly:
BufferedWriter x = new BufferedWriter(....)
resultModel.write(x, "N3") ;
x.flush() ;
or close it explicitly.
All works ok when the file is less than 100mb.
When the file is larger, this has a strange behaviour: the program does not
throw an exception, java runtime does not throw an out-of-memory error, and yet
the file does not get written. I have increased java memory to -Xmx3072m
The same model can be successfully stored using SDB (>1.5 million triples), so
I guess the problem is not in the model. I guess it is a general Java issue?
(http://www.java-forums.org/new-java/42031-writing-huge-sized-file-data-more-than-100mb-output-stream-converting-byte.html)
Did anyone else have this issue? Has anyone succeeded in writing rdf files
larger than 100mb to disk (and how)?
Are you sure it is no just being very, very slow?
http://jena.apache.org/documentation/io/rdf-output.html
"N3" (better, "TTL") is trying to pretty print the data which requires
analysing the data first. That can involve of small probes in the data
which for a database is terrible for performance especially if done
outside a transaction.
You need to use a format that does less analysis e.g.
RDFFormat.TURTLE_BLOCKS with RDFDataMgr.
Thank you in advance for your support.
Best regards,Nikolaos Konstantinou
Is resultModel stored in SDB?
Try the following:
OutputStream out = new FileOutputStream
RDFDataMgr.write(out, resultModel, RDFFormat.NTRIPLES) ;
// Or lang.NTRIPLES
which is the best scaling output. RDFFormat.TURTLE_BLOCKS isn't bad though.
Andy
