I'm using Jena 2.10.0 and i'm not on a virtual machine. I will create a small example of our implementation and send it to you.
Thanks for answering to me so quicky !! 2013/5/14 Frederic Toublanc <[email protected]> > Ok this works thanks a lot. > > I'm still testing the behaviour of transactions. > > > 2013/5/13 "Dr. André Lanka" <[email protected]> > >> Hi, >> >> you could give >> >> TDB.getContext().set(SystemTDB.symFileMode, FileMode.direct); >> >> a shot. This forces the old (direct) file mode even on Java64 systems. >> We use this too (because, the files in the other mode are too large for >> us). We use it heavily with transaction and don't encounter abnormal >> file sizes. Perhaps you have to rebuild the TDB store from scratch (in >> direct mode of course) to get small tdb files. >> >> Greetings >> André >> >> >> On 13.05.2013 17:36, Frederic Toublanc wrote: >> > Ok but is there a way to reconstruct the indexes to save some space. >> > This size problem is just not acceptable. >> > After adding 200 elements (2ko each in rdf format) the size of the TDB >> is >> > 344mo ... Thats too much >> > >> > >> > 2013/5/13 David Jordan <[email protected]> >> > >> >> TDB manages data on a block basis, I forget the specific size of the >> >> block, but as I recall, the block size seemed relatively large to me. >> Many >> >> systems allow you to configure the block size, but it does not seem TDB >> >> supports this. The other aspect to triple stores is that despite the >> fact >> >> that the schema is relatively simple, the data is extensively indexed, >> the >> >> indexes in some/all cases include all the data from the triple. So >> there >> >> ends up being lots of data duplication. Since things are managed on a >> block >> >> basis, there could also be a possible issue with the amount of the >> blocks >> >> that are unused. >> >> >> >> If you intend to have multiple applications doing updates on the data, >> you >> >> must use transactions. >> >> >> >> -----Original Message----- >> >> From: Frederic Toublanc [mailto:[email protected]] >> >> Sent: Monday, May 13, 2013 10:44 AM >> >> To: [email protected] >> >> Subject: Re: Size of Jena TDB >> >> >> >> Ok i gave up with transaction. >> >> >> >> I still dont understand why the TDB is inscreasing to dramatically >> when i >> >> dont use transactions ... :( >> >> >> >> >> >> 2013/5/13 Andy Seaborne <[email protected]> >> >> >> >>>> Why thoses changes solve the size problem ? >> >>> >> >>> Frederic, >> >>> >> >>> Which version are you running? >> >>> >> >>> We can't run your example because it is is tied to your application >> >>> code so >> >>> >> >>> With the current 2.10.0 and previous 2.7.4 releases of Jena, nested >> >>> write transactions on the same thread aren't allowed and it's checked >> >> for: >> >>> >> >>> >> >>> public static void main(String[] args) //throws Exception >> >>> { >> >>> Dataset ds = TDBFactory.createDataset() ; >> >>> ds.begin(ReadWrite.WRITE) ; >> >>> ds.begin(ReadWrite.WRITE) ; >> >>> >> >>> ds.commit() ; >> >>> ds.commit() ; >> >>> } >> >>> >> >>> throws an exception on the second .begin. >> >>> >> >>> Andy >> >>> >> >>> >> >>> On 13/05/13 07:30, Frederic Toublanc wrote: >> >>> >> >>>> Anyway i still have a question. >> >>>> >> >>>> Here the changes : >> >>>> >> >>>> I addes the begin write transaction before calling getDefaultModel() >> >>>> // Begin write transaction >> >>>> ds.begin(ReadWrite.WRITE); >> >>>> model = ds.getDefaultModel(); >> >>>> >> >>>> >> >>>> At the end i close the connection as follow : >> >>>> >> >>>> // End transaction >> >>>> ds.end(); >> >>>> // Close the model >> >>>> model.close(); >> >>>> >> >>>> // Close the dataset. >> >>>> ds.close(); >> >>>> >> >>>> >> >>>> Same thing for retrieving data (before i didnt start and end any >> >>>> transactions) so now : >> >>>> >> >>>> // Begin readtransaction >> >>>> ds.begin(ReadWrite.READ); >> >>>> model = ds.getDefaultModel(); >> >>>> >> >>>> >> >>>> and i close it as follow at the end : >> >>>> >> >>>> // End transaction >> >>>> ds.end(); >> >>>> // Close the model >> >>>> model.close(); >> >>>> >> >>>> // Close the dataset. >> >>>> ds.close(); >> >>>> >> >>>> >> >>>> Why thoses changes solve the size problem ? >> >>>> >> >>>> >> >>>> 2013/5/10 Brian McBride <[email protected]> >> >>>> >> >>>> On 10/05/2013 15:29, Bill Roberts wrote: >> >>>>> >> >>>>> >> >>>>>> I'm pretty confident that an empty TDB database does not occupy >> >> 192MB. >> >>>>>>> >> >>>>>>> It does on a Mac - as explained in Andy's mail a little while >> ago. >> >>>>>>> >> >>>>>> >> >>>>>> Gosh! I just reread Andy's message. Frederic says he is using >> >>>>> Windows 7. >> >>>>> >> >>>>> Brian >> >>>>> >> >>>>> >> >>>>> >> >>>>> Not sure what OS Frederic is using. >> >>>>>> >> >>>>>> total 393216 >> >>>>>> drwxr-xr-x 29 bill staff 986 10 May 15:27 ./ >> >>>>>> drwxr-xr-x 19 bill staff 646 10 May 15:26 ../ >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GOSP.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GOSP.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GPOS.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GPOS.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GSPO.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GSPO.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSP.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSP.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSPG.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSPG.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POS.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POS.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POSG.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POSG.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPO.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPO.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPOG.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPOG.idn >> >>>>>> -rw-r--r-- 1 bill staff 0 10 May 15:27 journal.jrnl >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 node2id.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 node2id.idn >> >>>>>> -rw-r--r-- 1 bill staff 0 10 May 15:27 nodes.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefix2id.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefix2id.idn >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefixIdx.dat >> >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefixIdx.idn >> >>>>>> -rw-r--r-- 1 bill staff 0 10 May 15:27 prefixes.dat >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>> -- >> >>>>> Epimorphics Ltd (http://www.epimorphics.com) >> >>>>> >> >>>>> Epimorphics Ltd. is a limited company registered in England (number >> >>>>> 7016688) >> >>>>> Registered address: Court Lodge, 105 High Street, Portishead, >> >>>>> Bristol >> >>>>> BS20 >> >>>>> 6PT, UK >> >>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> >> >> > >> >> -- >> Dr. André Lanka * 0178 / 134 44 47 * http://dr-lanka.de >> > >
