Hi Mr.Huf,
I think your advice worked fine for me. I have in the end on RDF/XML file
which I think contains all the data from the other files.
Thanks a lot.
On Fri, Feb 12, 2021 at 3:43 PM Alexis Armin Huf <alexis...@gmail.com>
wrote:
> Hi, emri.
>
> In my experience with Jena I have observed that Graphs are more efficient
> than Models when there is too much data being iterated. Also, at every
> createUnion() call, your code is creating a new Union graph which in the
> end will yield a tree of 700 models that will potentially be traversed when
> doing searches (which the RDF/XML serializer will eventually have to do in
> order to layout resources properly inside the XML).
>
> Maybe this will be faster:
>
> public static void main(String[] args) throws Exception{
> File folder = new File("C:\\Users\\Admin\\Desktop\\KG");
> Graph acc = GraphFactory.createDefaultGraph();
> RDFDataMgr.read(acc,
> "file:C:\\Users\\Admin\\Desktop\\KG\\AccessComponent.bo.rdf");
> for (File file : folder.listFiles()) {
> if (file.isFile())
> RDFDataMgr.read(acc, file.toURI().toString());
> }
> try (FileOutputStream out = new
> FileOutputStream("C:\\Users\\Admin\\Desktop\\merged.rdf")) {
> RDFDataMgr.write(out, acc, RDFFormat.RDFXML);
> }
> }
>
>
> Note that I have not tested this, you may want to run some iterations under
> the debugger.
>
> On Fri, Feb 12, 2021 at 9:29 AM emri mbiemri <emrimbiemri8...@gmail.com>
> wrote:
>
> > Hi, the code is:
> > ----------------------------------
> >
> >
> > public class RDFMerge {
> >
> > private static File folder;
> > private static Model kg;
> > // private static OutputStream out;
> >
> > public static void iterate() {
> >
> > folder = new File("C:\\Users\\Admin\\Desktop\\KG");
> > kg =
> >
> >
> ModelFactory.createDefaultModel().read("C:\\Users\\Admin\\Desktop\\KG\\AccessComponent.bo.rdf");
> >
> > File[] listOfFiles = folder.listFiles(); // Interating through the
> > directory
> > for (File file : listOfFiles) {
> >
> > if (file.isFile()) {
> >
> > kg = merge(kg, file);
> >
> > }
> >
> > }
> >
> >
> > OutputStream out1;
> > try { out1 = new FileOutputStream( new
> > File("C:\\Users\\Admin\\Desktop\\merged.rdf"));
> > kg.write( out1, "RDF/XML", null );
> > System.out.println("RDFs merged successfully!"); }
> > catch(FileNotFoundException e) { e.printStackTrace(); }
> >
> >
> > }
> >
> >
> > public static Model merge(Model k, File i) {
> >
> > // Model kg_in =
> > ModelFactory.createDefaultModel().read(k.getAbsolutePath());
> > Model other_in =
> > ModelFactory.createDefaultModel().read(i.getAbsolutePath());
> >
> > Model union = ModelFactory.createUnion(k, other_in);
> > return union;
> > }}
> >
> > On Fri, Feb 12, 2021 at 2:12 PM Andy Seaborne <a...@apache.org> wrote:
> >
> > > Hi there - the attachment didn't make it through.
> > >
> > > Could you include the code in the body of the message please? Or put in
> > > somewhere like a gist on github / pastebin /... and send a link.
> > >
> > > Andy
> > >
> > > On 12/02/2021 11:45, emri mbiemri wrote:
> > > > Dear all,
> > > >
> > > > Do you know how I can merge some thousands RDF models into a single
> > one?
> > > > I have tried it by iteration through all files within a folder and
> then
> > > > using Jena's union function to merge them one by one! The problem is
> > > > that the program is running for more than 13 hours and is still not
> > > > stopping (with only 50 models as test).
> > > >
> > > > So far I have close to 700 models, in total 68MB.
> > > >
> > > > Attached you seen the code I am using for.
> > > >
> > > > Do you have any idea what I can do to merge all these files into a
> > > > single knowledge-graph?
> > > >
> > > > Thanks for your help.
> > >
> >
>
>
> --
> Alexis Armin Huf <alexis...@gmail.com>
>
