Hi, you could give
TDB.getContext().set(SystemTDB.symFileMode, FileMode.direct); a shot. This forces the old (direct) file mode even on Java64 systems. We use this too (because, the files in the other mode are too large for us). We use it heavily with transaction and don't encounter abnormal file sizes. Perhaps you have to rebuild the TDB store from scratch (in direct mode of course) to get small tdb files. Greetings André On 13.05.2013 17:36, Frederic Toublanc wrote: > Ok but is there a way to reconstruct the indexes to save some space. > This size problem is just not acceptable. > After adding 200 elements (2ko each in rdf format) the size of the TDB is > 344mo ... Thats too much > > > 2013/5/13 David Jordan <[email protected]> > >> TDB manages data on a block basis, I forget the specific size of the >> block, but as I recall, the block size seemed relatively large to me. Many >> systems allow you to configure the block size, but it does not seem TDB >> supports this. The other aspect to triple stores is that despite the fact >> that the schema is relatively simple, the data is extensively indexed, the >> indexes in some/all cases include all the data from the triple. So there >> ends up being lots of data duplication. Since things are managed on a block >> basis, there could also be a possible issue with the amount of the blocks >> that are unused. >> >> If you intend to have multiple applications doing updates on the data, you >> must use transactions. >> >> -----Original Message----- >> From: Frederic Toublanc [mailto:[email protected]] >> Sent: Monday, May 13, 2013 10:44 AM >> To: [email protected] >> Subject: Re: Size of Jena TDB >> >> Ok i gave up with transaction. >> >> I still dont understand why the TDB is inscreasing to dramatically when i >> dont use transactions ... :( >> >> >> 2013/5/13 Andy Seaborne <[email protected]> >> >>>> Why thoses changes solve the size problem ? >>> >>> Frederic, >>> >>> Which version are you running? >>> >>> We can't run your example because it is is tied to your application >>> code so >>> >>> With the current 2.10.0 and previous 2.7.4 releases of Jena, nested >>> write transactions on the same thread aren't allowed and it's checked >> for: >>> >>> >>> public static void main(String[] args) //throws Exception >>> { >>> Dataset ds = TDBFactory.createDataset() ; >>> ds.begin(ReadWrite.WRITE) ; >>> ds.begin(ReadWrite.WRITE) ; >>> >>> ds.commit() ; >>> ds.commit() ; >>> } >>> >>> throws an exception on the second .begin. >>> >>> Andy >>> >>> >>> On 13/05/13 07:30, Frederic Toublanc wrote: >>> >>>> Anyway i still have a question. >>>> >>>> Here the changes : >>>> >>>> I addes the begin write transaction before calling getDefaultModel() >>>> // Begin write transaction >>>> ds.begin(ReadWrite.WRITE); >>>> model = ds.getDefaultModel(); >>>> >>>> >>>> At the end i close the connection as follow : >>>> >>>> // End transaction >>>> ds.end(); >>>> // Close the model >>>> model.close(); >>>> >>>> // Close the dataset. >>>> ds.close(); >>>> >>>> >>>> Same thing for retrieving data (before i didnt start and end any >>>> transactions) so now : >>>> >>>> // Begin readtransaction >>>> ds.begin(ReadWrite.READ); >>>> model = ds.getDefaultModel(); >>>> >>>> >>>> and i close it as follow at the end : >>>> >>>> // End transaction >>>> ds.end(); >>>> // Close the model >>>> model.close(); >>>> >>>> // Close the dataset. >>>> ds.close(); >>>> >>>> >>>> Why thoses changes solve the size problem ? >>>> >>>> >>>> 2013/5/10 Brian McBride <[email protected]> >>>> >>>> On 10/05/2013 15:29, Bill Roberts wrote: >>>>> >>>>> >>>>>> I'm pretty confident that an empty TDB database does not occupy >> 192MB. >>>>>>> >>>>>>> It does on a Mac - as explained in Andy's mail a little while ago. >>>>>>> >>>>>> >>>>>> Gosh! I just reread Andy's message. Frederic says he is using >>>>> Windows 7. >>>>> >>>>> Brian >>>>> >>>>> >>>>> >>>>> Not sure what OS Frederic is using. >>>>>> >>>>>> total 393216 >>>>>> drwxr-xr-x 29 bill staff 986 10 May 15:27 ./ >>>>>> drwxr-xr-x 19 bill staff 646 10 May 15:26 ../ >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GOSP.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GOSP.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GPOS.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GPOS.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GSPO.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 GSPO.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSP.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSP.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSPG.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSPG.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POS.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POS.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POSG.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 POSG.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPO.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPO.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPOG.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPOG.idn >>>>>> -rw-r--r-- 1 bill staff 0 10 May 15:27 journal.jrnl >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 node2id.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 node2id.idn >>>>>> -rw-r--r-- 1 bill staff 0 10 May 15:27 nodes.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefix2id.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefix2id.idn >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefixIdx.dat >>>>>> -rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefixIdx.idn >>>>>> -rw-r--r-- 1 bill staff 0 10 May 15:27 prefixes.dat >>>>>> >>>>>> >>>>>> >>>>> -- >>>>> Epimorphics Ltd (http://www.epimorphics.com) >>>>> >>>>> Epimorphics Ltd. is a limited company registered in England (number >>>>> 7016688) >>>>> Registered address: Court Lodge, 105 High Street, Portishead, >>>>> Bristol >>>>> BS20 >>>>> 6PT, UK >>>>> >>>>> >>>>> >>>> >>> >> >> > -- Dr. André Lanka * 0178 / 134 44 47 * http://dr-lanka.de
