[SAIL-Dev] Re: Problem with session backup persistence

[Aaron Unger]

I'm CC:ing this to sail-dev as well, since most of my response is oriented around the SAIL side of things, rather than the otrunk side. I implemented a solution on my local computer. Before I check anything in, I wanted to run it past everyone to see what they say. I used java.util.Timer to enable periodic automatic saving, like Scott suggested, in both OTrunkStep in pas-otrunk, and SailOTViewer in sail-otrunk. I set it to save every 1 minute. The old behavior of saving on exit/step close still exists. If the userDb isn't dirty (ie nothing has been changed by the user since the last save), then it skips saving anything. All of this I feel fine about. Then, in the SAIL EMF stuff, I had to make a couple changes which I'm a little more concerned about.   1) In the ISock interface, I add a new method: boolean add(T o, boolean replace);   2) I updated EMFSock and MemorySock to implement the new method by replacing add(T o) with add(T o, boolean replace), and then implementing add(T o) to call the new add method with a default of false which will use the existing behavior. (And MemorySock ignores the replace parameter altogether).   3) In EMFSock, if replace is true, it gets the most recent sockEntry (eSock.get(eSock.size()-1)) and then modifies it with the new value that was passed in. If replace is false, then the old behavior applies -- it creates a new sockEntry and adds it to the Sock. Originally, I had just modified EMFSock to do the "replace" behavior automatically when working with a rim of name "ot.learner.data". This worked fine when starting with a clean slate, but when I tried it in a situation where there was exisiting learner data, it would grab the most recent sockEntry and modify it. Unfortunately, that sockEntry was in the data passed down from the SDS, so it was never stored into the current session bundle (and therefore never saved back to the SDS -- bad!). By adding in the boolean parameter to add(), I could therefore ensure that the first time data was saved it would force the creation of a sockEntry, and then later saves could update that sockEntry. So my questions are:   - Is there a way to detect whether a sock entry comes from the current sessionBundle (thus possibly avoiding the need to modify ISock)?   - Are there any repercussions to modifying the ISock interface that I didn't think about/cover?   - Are EMFSock and MemorySock the only classes implementing ISock right now? If nobody has any objections for now, I'll go ahead and check in my code. The current behavior should be retained for all but the OTrunk steps. Thanks! -- Aaron Scott Cytacki wrote: I think the timer thread is best idea. This is similar to a word processor saving things on a regular basis. You can use the java.util.Timer class for this. That way if anyone else is using a timer it _might_ reuse the thread. This type of approach would be helpful to udl authors as well so they would have a backup when things crash. But in that case they aren't saving into a rim, they are just saving a file. One issue you'll have to deal with is the sock entry model. In order to save the data it will need to be put into a sock, and that means a new sock entry. Which means during a session there will be multiple copies of the same ot.learner.data. A few solutions: - bypass the agent service and generate a session bundle directly if the data has changed and then save that to disk. - add some new "hint" to the rim/sock/sock entry design. This hint would say that this rim is for storing a large chunk of data so each change to it should replace the previous value instead of adding a new one. Then persistence implementation (emf in this case), could do its best to deal with this. How things should be handled across multiple sessions is a bit tricky here. - add a special case to the emf persistence implementation so it handles ot.learner.data labeled rims like the hint above. It is probably best to do something quick like the last solution, and then discuss this at the tels or sail meeting to see if anyone has opinions about it. Even in the medium term I would vote for the last solution. It is hacky, but I think once we get moving on the jackrabbit/CMS stuff, our method of using rims/socks/sock entries will get replaced by something different. So it isn't worth the effort to do it more cleanly. Scott Aaron Unger wrote: I had a suspicion crystallize yesterday in respect to how a pure otrunk sail session persists it's data to disk during the session in order to have a backup in case Bad Things(tm) happen. Basically, it doesn't. And yesterday I realized how that could be, and this morning I've confirmed it by looking at the code for SailOTViewer and actually testing -- run an ITSI activity, do stuff, force quit the application. So, some background for people unfamiliar with it: When a "pure" otrunk session is run in Sail, it's really running in a curnit with 1 pod, and 1 rim. The Pod sets up a SailOTViewer, which saves its data into a sock on that 1 rim via a SockEntry. SailOTViewer uses 1 sockentry throughout the session. Sail will automatically persist its collection of rims and socks and sockentries to the local disk whenever anything gets changed. That way, if something bad happens, you'll still have all of the data up until the last change. And that's where the problem comes up. SailOTViewer (and OTrunkStep in wise) only writes to its sockentry at exit. Which means that your entire session is being stored in OT learner data in memory, but never persisted to disk until you close. In the WISE world, this isn't so bad -- steps aren't usually designed to be long-running (although a few models are). In the UDL world, this is terrible! It means you can work on a project for a whole period, and if your computer crashes before you close, you lose *everything* (which we've now had happen in Alaska). So, I'm soliciting solutions or opinions on some solutions I've thought of (or both). A couple options: - listen for changes to the ot learner data, and save whenever anything changes. I worry about the potential performance hit from doing this, though. - launch a timer thread on startup which persists the ot learner data every N time period. If nothing is changed, this call is cheap. If things have changed, we'll still get a performance hit, but only every N, not every time the learner data changes (which for OTText would probably mean every time a key is typed.) Any other ideas? Comments on the ideas I put forth? Thanks- -- Aaron --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "SAIL-Dev" group. To post to this group, send email to SAIL-Dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/SAIL-Dev?hl=en -~----------~----~----~----~------~----~------~--~---