Hi Andy, What is the best way to get the code to you?
Thanks Steve Groeger From: Andy Seaborne <[email protected]> To: [email protected], Date: 27/02/2014 15:09 Subject: Re: BlockException: No such block On 27/02/14 13:50, Steve Groeger wrote: > Hi Andy, > > All the latest tests I was running were setting SystemTDB.NullOut = true; > > I will add the code you specified to our tests and re-run them and will > send you our results shortly. Thanks - if send the code as well, I'll try to run it at my end. Even though I wasn't managing to catch this nearly as frequently as you, this new situation might be better. Andy > > Thanks > Steve Groeger > IBM WebSphere Service Registry Development > MP 211, Hursley > Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 > Fax (44) 1962 816800 > Lotus Notes: Steve Groeger/UK/IBM > Internet: [email protected] > > > > From: Andy Seaborne <[email protected]> > To: [email protected], > Date: 27/02/2014 13:36 > Subject: Re: BlockException: No such block > > > > Hi Steve, > > Thanks for the update. > > On 25/02/14 15:15, Steve Groeger wrote: >> Hi Andy, >> >> I picked up a snapshot of 2.11.2 and tried running our test using these >> libraries. >> >> Adding the changes you suggested seems to have stopped the exception " >> BlockException: No such block", the but we are now seeing other > symptoms, >> which we saw previously but not as frequently as we are now. >> The change in behaviour seems to be result of the SystemTDB.NullOut = > true >> ; as I also tried various version of the listStatements and this gave > the >> same results, What does the SystemTDB.NullOut=true do? > > NullOut makes the B+Tree code overwrite areas of blocks that are now > unused because of a delete with some value (FillByte = 0xFF). In normal > operation, it just leaves it as junk, but of NullOut is set it goes and > sets the bytes to a specific value, and one which does not make sense. > > I would not expect it to change the frequency of errors, but it would > change the symptom. > >> What we are now seeing is data not being returned correctly. >> We initially populate the database with a Resource with 1000 Properties, >> then in the ReaderThread we query the database, using the following: >> >> StmtIterator found = model.listStatements((Resource)null, >> (Property)null, (RDFNode)null); >> >> we are expecting to get back 1000 statements when we perform the >> >> count = found.toList().size(); >> >> however we occasionally get a batch of results that show 0 or non 1000 >> statements being returned, then after several more reads this goes back > to >> getting 1000 statements and everything continues as normal. > > I'm not clear whether that is with or without NullOut set true. > >> >> We need to determine why we occasionally dont get the 1000 statements >> returned that we are expecting. Any advice you caould provide would be >> great. > > It looks as if under some unusual conditions, either the B+Tree branch > node or leaf node is not being updated correctly for deletions. > ("unusual" as this code is not new and for anyone doing deletes, heavily > used). > > That gives us something to go on. > > Could you please add some code that not only counts the StmtIterator but > also touches every part of the statement. Execute with > > SystemTDB.NullOut=true > > There is some delayed evaluation under the covers and this forces that > evaluation. > > StmtIterator found = .... > long count = 0 ; > while(found.hasNext()) { > count++ ; > Statement s = found.next(); > try { > Node subj = s.getSubject().asNode() ; > Node pred = s.getPredicate().asNode() ; > Node obj = s.getObject().asNode() ; > } catch (Exception ex) { > ex.printStackTrace(System.err) ; > System.exit(99) ; > } > } > > Warning - this may slow the execution down, or not, depending on cache > efficiencies. > > Andy > >> >> Thanks >> Steve Groeger >> IBM WebSphere Service Registry Development >> MP 211, Hursley >> Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 >> Fax (44) 1962 816800 >> Lotus Notes: Steve Groeger/UK/IBM >> Internet: [email protected] > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
