Hi, emri.
In my experience with Jena I have observed that Graphs are more efficient
than Models when there is too much data being iterated. Also, at every
createUnion() call, your code is creating a new Union graph which in the
end will yield a tree of 700 models that will potentially be traversed when
doing searches (which the RDF/XML serializer will eventually have to do in
order to layout resources properly inside the XML).
Maybe this will be faster:
public static void main(String[] args) throws Exception{
File folder = new File("C:\\Users\\Admin\\Desktop\\KG");
Graph acc = GraphFactory.createDefaultGraph();
RDFDataMgr.read(acc,
"file:C:\\Users\\Admin\\Desktop\\KG\\AccessComponent.bo.rdf");
for (File file : folder.listFiles()) {
if (file.isFile())
RDFDataMgr.read(acc, file.toURI().toString());
}
try (FileOutputStream out = new
FileOutputStream("C:\\Users\\Admin\\Desktop\\merged.rdf")) {
RDFDataMgr.write(out, acc, RDFFormat.RDFXML);
}
}
Note that I have not tested this, you may want to run some iterations under
the debugger.
On Fri, Feb 12, 2021 at 9:29 AM emri mbiemri <emrimbiemri8...@gmail.com>
wrote:
> Hi, the code is:
> ----------------------------------
>
>
> public class RDFMerge {
>
> private static File folder;
> private static Model kg;
> // private static OutputStream out;
>
> public static void iterate() {
>
> folder = new File("C:\\Users\\Admin\\Desktop\\KG");
> kg =
>
> ModelFactory.createDefaultModel().read("C:\\Users\\Admin\\Desktop\\KG\\AccessComponent.bo.rdf");
>
> File[] listOfFiles = folder.listFiles(); // Interating through the
> directory
> for (File file : listOfFiles) {
>
> if (file.isFile()) {
>
> kg = merge(kg, file);
>
> }
>
> }
>
>
> OutputStream out1;
> try { out1 = new FileOutputStream( new
> File("C:\\Users\\Admin\\Desktop\\merged.rdf"));
> kg.write( out1, "RDF/XML", null );
> System.out.println("RDFs merged successfully!"); }
> catch(FileNotFoundException e) { e.printStackTrace(); }
>
>
> }
>
>
> public static Model merge(Model k, File i) {
>
> // Model kg_in =
> ModelFactory.createDefaultModel().read(k.getAbsolutePath());
> Model other_in =
> ModelFactory.createDefaultModel().read(i.getAbsolutePath());
>
> Model union = ModelFactory.createUnion(k, other_in);
> return union;
> }}
>
> On Fri, Feb 12, 2021 at 2:12 PM Andy Seaborne <a...@apache.org> wrote:
>
> > Hi there - the attachment didn't make it through.
> >
> > Could you include the code in the body of the message please? Or put in
> > somewhere like a gist on github / pastebin /... and send a link.
> >
> > Andy
> >
> > On 12/02/2021 11:45, emri mbiemri wrote:
> > > Dear all,
> > >
> > > Do you know how I can merge some thousands RDF models into a single
> one?
> > > I have tried it by iteration through all files within a folder and then
> > > using Jena's union function to merge them one by one! The problem is
> > > that the program is running for more than 13 hours and is still not
> > > stopping (with only 50 models as test).
> > >
> > > So far I have close to 700 models, in total 68MB.
> > >
> > > Attached you seen the code I am using for.
> > >
> > > Do you have any idea what I can do to merge all these files into a
> > > single knowledge-graph?
> > >
> > > Thanks for your help.
> >
>
--
Alexis Armin Huf <alexis...@gmail.com>
