local I am about to start writing a simple test program to illustrate the point. FYI, we first reproduced the problem on a large installation (because I didn't suspect a problem here and we didn't have a junit to cover this). What was interesting was that after we had released the store, the index appeared empty (and I checked this by running tdbDump on it), so we reindexed everything again. The size on disk had doubled after this operation suggesting that the data was all there and persisted, just not "reachable"
Simon From: Andy Seaborne <[email protected]> To: Simon Helsen/Toronto/IBM@IBMCA Cc: [email protected] Date: 06/05/2012 04:33 PM Subject: Re: StoreConnection.release() does not allow to connect again Simon - do you use remote disk, or local disk? Andy On 04/06/12 17:30, Andy Seaborne wrote: > On 04/06/12 14:53, Simon Helsen wrote: >> Andy, >> >> Although I know someone tied it to Windows, I don't find it intuitive >> here that this is a windows problem. I will try to verify the behavior >> on linux, but it may take some time. As for the build, it is quite >> recent. This was the tdb snapshot: jena-tdb-0.9.1-20120514.040618-2 >> >> Just to clarify: are you saying that if you run >> StoreConnection.release()on linux, that you can still connect afterwards >> and query the same store? Do you have junits which do this? > > I was asking to find out before I investigate (and I was away from my > dev machines at the time). > > Your report did not include a clear example, only that release and > reconnect, not, say, how you had used the dataset or whether there may > be in-progress actions. As the mantra goes: "complete, minimal example" > please. > > I sense you have one as your description hints at a sequence of actions. > If you know the issue is windows specific, it's going to save me a lot > of time investigating the wrong thing. > > So if you could provide the sequence of actions that cause the problem > it would be helpful - ideally a test case and is it Windows specific? > > (Windows is different to Linux - you can't delete open files on Windows) > > Andy > >> >> thanks >> >> Simon >> >> >> >> >> From: Andy Seaborne <[email protected]> >> To: [email protected] >> Date: 06/03/2012 06:01 AM >> Subject: Re: StoreConnection.release() does not allow to connect again >> >> >> ------------------------------------------------------------------------ >> >> >> >> Simon, >> >> Which build of the snapshot are you running and are you saying it does >> not happen on Linux? >> >> Andy >> (not near a Windows machine for a few days) >> >> On 02/06/12 00:58, Simon Helsen wrote: >> > thanks for the quick response Rob, >> > >> > yes, this is on windows, but I use direct I/O (deliberately) because of >> > the known problems with mapped I/O. I have not yet had a chance to >> > follow-up on the suggestions earlier this month to work around the >> problem >> > with mapped I/O on windows. >> > >> > I think this is an entirely different matter though. It seems to me >> that >> > the Journal is open when I don't call StoreConnection.release(), but >> when >> > I do this, as I mentioned below, the persisted store is lost, i.e >> even a >> > new Java process cannot connect to it again (or rather, it can do this, >> > but thinks the store is empty). It seems to me that that is the >> problem, >> > not that the journal object holds on to its channel without an explicit >> > close. I actually tried to just close the journal, but then it >> cannot be >> > re-opened again within the same Java process. >> > >> > Simon >> > >> > >> > >> > >> > >> > From: >> > Rob Vesse<[email protected]> >> > To: >> > "[email protected]"<[email protected]> >> > Date: >> > 06/01/2012 07:06 PM >> > Subject: >> > Re: StoreConnection.release() does not allow to connect again >> > >> > >> > >> > Hi Simon >> > >> > Are you running on Windows per chance? >> > >> > TDB uses memory mapped files for its data and there are some issues >> around >> > Windows releasing memory mapped files. I thought this was mostly >> confined >> > to the case where you were trying to delete such files but maybe it >> also >> > applies in this case? >> > >> > Someone like Andy/Paolo who is more knowledgeable about the >> internals of >> > TDB would be better placed to answer your question. >> > >> > The details of your environment (like OS) will likely be helpful in >> > tracking down whether this is a bug or not >> > >> > Rob >> > >> > >> > On 6/1/12 3:35 PM, "Simon Helsen"<[email protected]> wrote: >> > >> >> Hi everyone, >> >> >> >> before filing a bug, I want to make sure I am not doing something >> wrong. >> >> I >> >> am running in the following problem. >> >> >> >> Until I call StoreConnection.release(location) the TDB code holds a >> lock >> >> on the file system of the index. I think this is the journal file >> mostly >> >> because I can separately close a dataset. >> >> >> >> However, when I call this and then later try to "reconnect", e.g. by >> >> calling StoreConnection.make(location) or just creating a dataset, it >> >> seems that the persisted data is not accessible anymore. I.e. it seems >> >> that StoreConnection.release is doing more than just releasing the >> >> location. It seems to persist something that makes the data go away. >> >> >> >> Note that I am seeing this independent on whether I use >> transactions or >> >> not (I have 2 versions around to compare the transactional and >> >> non-transactional behavior) >> >> >> >> This is troublesome. I need some way to "disconnect" from a TDB >> store (so >> >> all file handles are released), but such that I can reconnect at a >> later >> >> time. Am I doing something wrong? Or is this a bug? >> >> >> >> thanks >> >> >> >> Simon >> >> >> >> PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption >> > >> > >> > >> > >> >> >> >
