Hi Steve,
Thanks for the update.
On 25/02/14 15:15, Steve Groeger wrote:
Hi Andy,
I picked up a snapshot of 2.11.2 and tried running our test using these
libraries.
Adding the changes you suggested seems to have stopped the exception "
BlockException: No such block", the but we are now seeing other symptoms,
which we saw previously but not as frequently as we are now.
The change in behaviour seems to be result of the SystemTDB.NullOut = true
; as I also tried various version of the listStatements and this gave the
same results, What does the SystemTDB.NullOut=true do?
NullOut makes the B+Tree code overwrite areas of blocks that are now
unused because of a delete with some value (FillByte = 0xFF). In normal
operation, it just leaves it as junk, but of NullOut is set it goes and
sets the bytes to a specific value, and one which does not make sense.
I would not expect it to change the frequency of errors, but it would
change the symptom.
What we are now seeing is data not being returned correctly.
We initially populate the database with a Resource with 1000 Properties,
then in the ReaderThread we query the database, using the following:
StmtIterator found = model.listStatements((Resource)null,
(Property)null, (RDFNode)null);
we are expecting to get back 1000 statements when we perform the
count = found.toList().size();
however we occasionally get a batch of results that show 0 or non 1000
statements being returned, then after several more reads this goes back to
getting 1000 statements and everything continues as normal.
I'm not clear whether that is with or without NullOut set true.
We need to determine why we occasionally dont get the 1000 statements
returned that we are expecting. Any advice you caould provide would be
great.
It looks as if under some unusual conditions, either the B+Tree branch
node or leaf node is not being updated correctly for deletions.
("unusual" as this code is not new and for anyone doing deletes, heavily
used).
That gives us something to go on.
Could you please add some code that not only counts the StmtIterator but
also touches every part of the statement. Execute with
SystemTDB.NullOut=true
There is some delayed evaluation under the covers and this forces that
evaluation.
StmtIterator found = ....
long count = 0 ;
while(found.hasNext()) {
count++ ;
Statement s = found.next();
try {
Node subj = s.getSubject().asNode() ;
Node pred = s.getPredicate().asNode() ;
Node obj = s.getObject().asNode() ;
} catch (Exception ex) {
ex.printStackTrace(System.err) ;
System.exit(99) ;
}
}
Warning - this may slow the execution down, or not, depending on cache
efficiencies.
Andy
Thanks
Steve Groeger
IBM WebSphere Service Registry Development
MP 211, Hursley
Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129
Fax (44) 1962 816800
Lotus Notes: Steve Groeger/UK/IBM
Internet: [email protected]
