On 13.05.2013 17:36, Frederic Toublanc wrote:
Ok but is there a way to reconstruct the indexes to save some space.
This size problem is just not acceptable.
After adding 200 elements (2ko each in rdf format) the size of the TDB is
344mo ... Thats too much
Which version are you running?
What sort of hardware are you using? Or is it a virtual machine?
Please can you provide a complete, minimal example that we can try?
Andy
2013/5/13 David Jordan <[email protected]>
TDB manages data on a block basis, I forget the specific size of the
block, but as I recall, the block size seemed relatively large to me. Many
systems allow you to configure the block size, but it does not seem TDB
supports this. The other aspect to triple stores is that despite the fact
that the schema is relatively simple, the data is extensively indexed, the
indexes in some/all cases include all the data from the triple. So there
ends up being lots of data duplication. Since things are managed on a block
basis, there could also be a possible issue with the amount of the blocks
that are unused.
If you intend to have multiple applications doing updates on the data, you
must use transactions.
-----Original Message-----
From: Frederic Toublanc [mailto:[email protected]]
Sent: Monday, May 13, 2013 10:44 AM
To: [email protected]
Subject: Re: Size of Jena TDB
Ok i gave up with transaction.
I still dont understand why the TDB is inscreasing to dramatically when i
dont use transactions ... :(
2013/5/13 Andy Seaborne <[email protected]>
Why thoses changes solve the size problem ?
Frederic,
Which version are you running?
We can't run your example because it is is tied to your application
code so
With the current 2.10.0 and previous 2.7.4 releases of Jena, nested
write transactions on the same thread aren't allowed and it's checked
for:
public static void main(String[] args) //throws Exception
{
Dataset ds = TDBFactory.createDataset() ;
ds.begin(ReadWrite.WRITE) ;
ds.begin(ReadWrite.WRITE) ;
ds.commit() ;
ds.commit() ;
}
throws an exception on the second .begin.
Andy
On 13/05/13 07:30, Frederic Toublanc wrote:
Anyway i still have a question.
Here the changes :
I addes the begin write transaction before calling getDefaultModel()
// Begin write transaction
ds.begin(ReadWrite.WRITE);
model = ds.getDefaultModel();
At the end i close the connection as follow :
// End transaction
ds.end();
// Close the model
model.close();
// Close the dataset.
ds.close();
Same thing for retrieving data (before i didnt start and end any
transactions) so now :
// Begin readtransaction
ds.begin(ReadWrite.READ);
model = ds.getDefaultModel();
and i close it as follow at the end :
// End transaction
ds.end();
// Close the model
model.close();
// Close the dataset.
ds.close();
Why thoses changes solve the size problem ?
2013/5/10 Brian McBride <[email protected]>
On 10/05/2013 15:29, Bill Roberts wrote:
I'm pretty confident that an empty TDB database does not occupy
192MB.
It does on a Mac - as explained in Andy's mail a little while ago.
Gosh! I just reread Andy's message. Frederic says he is using
Windows 7.
Brian
Not sure what OS Frederic is using.
total 393216
drwxr-xr-x 29 bill staff 986 10 May 15:27 ./
drwxr-xr-x 19 bill staff 646 10 May 15:26 ../
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 GOSP.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 GOSP.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 GPOS.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 GPOS.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 GSPO.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 GSPO.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSP.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSP.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSPG.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 OSPG.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 POS.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 POS.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 POSG.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 POSG.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPO.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPO.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPOG.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 SPOG.idn
-rw-r--r-- 1 bill staff 0 10 May 15:27 journal.jrnl
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 node2id.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 node2id.idn
-rw-r--r-- 1 bill staff 0 10 May 15:27 nodes.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefix2id.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefix2id.idn
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefixIdx.dat
-rw-r--r-- 1 bill staff 8388608 10 May 15:27 prefixIdx.idn
-rw-r--r-- 1 bill staff 0 10 May 15:27 prefixes.dat
--
Epimorphics Ltd (http://www.epimorphics.com)
Epimorphics Ltd. is a limited company registered in England (number
7016688)
Registered address: Court Lodge, 105 High Street, Portishead,
Bristol
BS20
6PT, UK
