[Zope] sessions in the presence of conflicts
Zope 2.8.4, ZODB 3.4.2 I am still trying to resolve a session variables in the presence of conflicts problem. The observed symptom is that suddenly either some session variables disappear or all session variables disappear. In both cases, Zope has raised a KeyError exception for the a particular session variable because it is not present and an attempt was being made to access it from the session. The cause appears to be related to the presence of a conflict error and a botched recovery, but the exact mechanism remains a mystery. Zope equates a session object with a TransientObject. For our instance, that's fine, since our only TransientObject is a session. For the general case, I would have thought a SessionObject inheriting from TransiendObject would have been used so that conflict resolution for sessions could be easily specialized. The conflict resolution for a TransientObject (that is, a Session Object) is clearly suboptimal. In the next paragraphs I'll review what is currently done and then propose a different approach which I am about to implement for our systems. Comments, corrections, and suggestions much appreciated. _p_resolveConflict(self, oldState, savedState, newState) returns the state of the object after resolving different changes. The arguments are: oldState -- state of the object at the beginning of the current transaction (mutable) savedState -- state currently stored in the database. This state was written after oldState and reflect changes made by a transaction that committed before the current transaction (immutable) newState -- state after changes made by the current transaction (immutable) The standard conflict resolution for a TransientObject resolves according to the following rules: 1. if any of the states are invalid (that is, has a key '_invalid') return the invalid state. 2. if any any of the attributes ['token','id','_created'] differ then there is a conflict, raise the conflict exception. 3. choose the state most recently modified, if possible. 4. otherwise, choose the state most recently accessed. It seems to me that we can do much better for sessions because we know a bit about the semantics of sessions. A session object is a dictionary-like object mapping key-value pairs. Adding or deleting keys or changing the value associated with a key are independent operations and do not conflict unless the keys are duplicated in both the transactions. Any conflict resolution mechanism needs to be able to manage multiple keys independently since the session object is modified as a unit. In addition, new keys may be added and old keys deleted; any conflict resolution mechanism at the key level needs to be comprehend those operations. A more session-friendly conflict resolution might use: 1. if any of the states are invalid (that is, has a key '_invalid') return the invalid state. 2. if any any of the states attributes ['token','id','_created'] differ then there is a conflict, raise the conflict exception. 3. order the newState and savedState by modification time (or if that cannot be computed, by access time). 4. any key appearing in oldState's dictionary but not appearing in both savedState and newState should be removed from all. This corresponds to a key-value pair being deleted in one of the transactions. Insertions will be managed automatically by the updates. 5. beginning with the oldest, update oldState dictionary of key-value pairs using the dictionary part of newState and savedState. Return oldState. This does several things. First, it captures independent key-value changes made in both potentially conflicting transactions. Second, it provides a reasonable ordering for multiple (potentially conflicting) key-value pair updates. Third, it manages insertions and deletions to the session variable set in the presence of conflicts. Does this make sense? I have yet to figure out how to map a TransientObject state back to the object it represents, but it clearly is possible. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] sessions in the presence of conflicts
Dennis Allison wrote: Zope 2.8.4, ZODB 3.4.2 I am still trying to resolve a session variables in the presence of conflicts problem. The observed symptom is that suddenly either some session variables disappear or all session variables disappear. In both cases, Zope has raised a KeyError exception for the a particular session variable because it is not present and an attempt was being made to access it from the session. The cause appears to be related to the presence of a conflict error and a botched recovery, but the exact mechanism remains a mystery. Zope equates a session object with a TransientObject. For our instance, that's fine, since our only TransientObject is a session. For the general case, I would have thought a SessionObject inheriting from TransiendObject would have been used so that conflict resolution for sessions could be easily specialized. The conflict resolution for a TransientObject (that is, a Session Object) is clearly suboptimal. In the next paragraphs I'll review what is currently done and then propose a different approach which I am about to implement for our systems. Comments, corrections, and suggestions much appreciated. _p_resolveConflict(self, oldState, savedState, newState) returns the state of the object after resolving different changes. The arguments are: oldState -- state of the object at the beginning of the current transaction (mutable) savedState -- state currently stored in the database. This state was written after oldState and reflect changes made by a transaction that committed before the current transaction (immutable) newState -- state after changes made by the current transaction (immutable) The standard conflict resolution for a TransientObject resolves according to the following rules: 1. if any of the states are invalid (that is, has a key '_invalid') return the invalid state. 2. if any any of the attributes ['token','id','_created'] differ then there is a conflict, raise the conflict exception. 3. choose the state most recently modified, if possible. 4. otherwise, choose the state most recently accessed. It seems to me that we can do much better for sessions because we know a bit about the semantics of sessions. A session object is a dictionary-like object mapping key-value pairs. Adding or deleting keys or changing the value associated with a key are independent operations and do not conflict unless the keys are duplicated in both the transactions. Any conflict resolution mechanism needs to be able to manage multiple keys independently since the session object is modified as a unit. In addition, new keys may be added and old keys deleted; any conflict resolution mechanism at the key level needs to be comprehend those operations. A more session-friendly conflict resolution might use: 1. if any of the states are invalid (that is, has a key '_invalid') return the invalid state. 2. if any any of the states attributes ['token','id','_created'] differ then there is a conflict, raise the conflict exception. 3. order the newState and savedState by modification time (or if that cannot be computed, by access time). 4. any key appearing in oldState's dictionary but not appearing in both savedState and newState should be removed from all. This corresponds to a key-value pair being deleted in one of the transactions. Insertions will be managed automatically by the updates. 5. beginning with the oldest, update oldState dictionary of key-value pairs using the dictionary part of newState and savedState. Return oldState. This does several things. First, it captures independent key-value changes made in both potentially conflicting transactions. Second, it provides a reasonable ordering for multiple (potentially conflicting) key-value pair updates. Third, it manages insertions and deletions to the session variable set in the presence of conflicts. Does this make sense? I have yet to figure out how to map a TransientObject state back to the object it represents, but it clearly is possible. Dennis, Im only curious and this is no recommendation ... and I asked this on the list maybe a year ago (and got zero reponses) Have you given HappySession a try? Has anyone? I've never had a problem with it, but then its never been under much load and is no longer actively supported (least last time I checked). I converted back and forth between Zope SESSION and HappySession (for reasons of legacy code). Their interfaces have just few differences, e.g., HappySession['this'] returns None (if 'this' is not a key) and SESSION raises an exception. That type of thing. HappySession does not read/write to zodb (if I recall it stores session data in RAM (session) cookies). I just do not know if its otherwise up to your
Re: [Zope] sessions in the presence of conflicts
Hi Davids. Thanks for your note. I remember your post. No, I have not used HappySession although we looked at it when we started. When we started development, sessions had just become part of the standard release Zope and we decided that in the long run we'd be better off using the release product. As I've said in other posts, we use the session mechanism fairly heavily. We have O(200) simultaneous interactive users each of whom has 40 or so session variables which maintain the dynamic state of their instance. Nearly every substantive operation touches one or more session variables. Everything works most of the time, but the error frequency is high enough for it to be of concern. I'm a bit cautious about moving to another sessioning system on a sizable production system running 24x7. Dennis, Im only curious and this is no recommendation ... and I asked this on the list maybe a year ago (and got zero reponses) Have you given HappySession a try? Has anyone? I've never had a problem with it, but then its never been under much load and is no longer actively supported (least last time I checked). I converted back and forth between Zope SESSION and HappySession (for reasons of legacy code). Their interfaces have just few differences, e.g., HappySession['this'] returns None (if 'this' is not a key) and SESSION raises an exception. That type of thing. HappySession does not read/write to zodb (if I recall it stores session data in RAM (session) cookies). I just do not know if its otherwise up to your requirements. All best, David -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
