Re: Addition of 'dirty' field to Session interface
--- Carlos Gaston Alvarez [EMAIL PROTECTED] We have an instance of the database to store the sessions. What does the database mean? If we're persisting the sessions to a real RDBMS, then that's a serious performance hit above and beyond just having to serialize the session whenever it's accessed. Not to mention that the user would have to add DB config info into the TC config. An alternate implementation would be to maintain this data in memory using a similar table-like object. When sessions are created/modified, you broadcast the changes on a multicast address. All other interested servlet engines can pick up this change and replicate it locally. This could conceivably be done using the Servlet 2.3 HttpSessionAttributeListener interface, and could also provide transparent failover for distributed sessions. Persistence to a database really only makes sense if you're dealing with MANY sessions (or perhaps many long-lived sessions) and you need an activate/passivate behavior to conserve memory. Session creation is a litle tricky because be should get sure that no other virtual machine is trying to create the same session (for the same user) and if so syncronize them. Well, there's really not much you can do about this. This situation would occur if the same browser accessed two different servlets simultaneously. For cookie-based sessions, the browser would get the session id for whichever servlet sent the cookie last. For url-rewriting based sessions, you'd have two different sessions started. I think this situation is sufficiently rare to not worry about. On every request: - get the lock sessionLocker (check if the locker is 0, if it is we take it; if it is not we wait until the locker is 0 try again). [sessionLocker = me] - check if it is the same sessionVersion. - if yes just use the session at memory. (why unserilize it if we can have them in memory?). - if not get the session of the database (sessionStream) and unserialize it. * now the user request runs as normal, until it is finished when .. - sessionVersion++ - sessionStream gets updated. - sessionLocker = 0 There are other optimizations but I will not discuss them now not to make a mess. A simple optimization: you mention above On every request... but really it is only On every request that calls getSession because if the servlet never accesses the session, we don't need to worry about it changing. So the pre-request stuff you mention above should really be done when getSession is called, and the post-request stuff should be done after the request is completed, but only if getSession was called during the request. All of this points to a solution presented much earlier in this thread - that any time the session is accessed we have to consider it dirty. The more I think about this the more it makes sense - you *really* don't want the developer to have to keep track of whether or not the session is dirty. Finally, this is a lot of extra processing to do if the web app is only run on a single machine. Perhaps a directive in the server.xml could indicate whether sessions should be shared/synchronized across multiple servlet engines or not. - osama. __ Do You Yahoo!? Make international calls for as low as $.04/minute with Yahoo! Messenger http://phonecard.yahoo.com/
Re: Addition of 'dirty' field to Session interface
It seams that we all think in a similar way. We can have a solution like this: We have an instance of the database to store the sessions. The table for the session can be something like this: sessionId, sessionVersion, sessionLocker, extraSessionData, sessionStream sessionId : a unique id for session [yes the id ;-)] sessionVersion : an identifier which say when it was modified for the last time (not a timestamp but a number) sessionLocker : who is locking the session (an id for the virtual machine) extraSessionData : some data like when it should experire (on dbtime) or well, I dont know. sessionStream : the serialized session (so all the object stored in the session should implement serializable). Session creation is a litle tricky because be should get sure that no other virtual machine is trying to create the same session (for the same user) and if so syncronize them. On every request: - get the lock sessionLocker (check if the locker is 0, if it is we take it; if it is not we wait until the locker is 0 try again). [sessionLocker = me] - check if it is the same sessionVersion. - if yes just use the session at memory. (why unserilize it if we can have them in memory?). - if not get the session of the database (sessionStream) and unserialize it. * now the user request runs as normal, until it is finished when .. - sessionVersion++ - sessionStream gets updated. - sessionLocker = 0 There are other optimizations but I will not discuss them now not to make a mess. The problems I see are most of all user problems like not making the objects serializable or making a bad use of the singleton pattern (we are running on multiple machines), or some dirty finalize methods. Ok, there will we some overhead but it will be after the http stream was flushed. An there is nothing like a free meal. (Yes, it should only get activated if the user dessires so) ((user=apache admin). Does apache has a mean of doing load balancing? If so, can we make it sticky so we optimize resources? Chau, Gaston - Original Message - From: Reilly, John [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, August 24, 2001 8:51 PM Subject: RE: Addition of 'dirty' field to Session interface This is just an idea from the top of my head, would it be possible having a second vector that contains a footprint(not a full clone) of the object for a session and have a reaper thread checking the footprints against the real objects and determine if they changed or not and based on that replicate of whatever we want to do. My thoughts exactly. If you want to be able to support transparent fail-over for sessions within a cluster, you are going to have to take the performance hit of persisting the session data on at least 1 other machine in the cluster after every request. If you're already taking that step, you might as well maintain an in-memory image of the serialized session object. You could compare an MD5 on the bytes comprising the session before the request was handled with the MD5 for after the request completed. Could this work? The overhead could be fairly signifigant. - osama.
RE: Addition of 'dirty' field to Session interface
For permanent business objects, that is probably true ... but the use cases we'd like to be able to deal with include: * Load-balanced distributed container that can move sessions around as various servers get overloaded. * Fail-safe distributed container that automatically recovers from server failures and reconnects you to a different one with your session intact. without the application developer having to worry about this for his/her session beans. The first case isn't so hard -- the only time you have to persist is when you are going to migrate, so you would just do it unconditinally. The second case is harder, unless you can afford the performance hit of persisting after *every* request. I don't think there's a single policy that will cover all reasonable use cases, so configurable selection of different policies is likely to be useful. I've been interested in getting my teeth into this problem for a while but I didn't have the time I may get a chance the near future though. Cheers, jr
RE: Addition of 'dirty' field to Session interface
-Original Message- From: Craig R. McClanahan [mailto:[EMAIL PROTECTED]] [...] * Load-balanced distributed container that can move sessions around as various servers get overloaded. * Fail-safe distributed container that automatically recovers from server failures and reconnects you to a different one with your session intact. without the application developer having to worry about this for his/her session beans. The first case isn't so hard -- the only time you have to persist is when you are going to migrate, so you would just do it unconditinally. The second case is harder, unless you can afford the performance hit of persisting after *every* request. This is just an idea from the top of my head, would it be possible having a second vector that contains a footprint(not a full clone) of the object for a session and have a reaper thread checking the footprints against the real objects and determine if they changed or not and based on that replicate of whatever we want to do. Similar to how PersistentManager check sessions to determine if they should be swapped to disk or backed up. Or is this just plain dumb? -bip thelin
RE: Addition of 'dirty' field to Session interface
--- Bip Thelin [EMAIL PROTECTED] wrote: This is just an idea from the top of my head, would it be possible having a second vector that contains a footprint(not a full clone) of the object for a session and have a reaper thread checking the footprints against the real objects and determine if they changed or not and based on that replicate of whatever we want to do. My thoughts exactly. If you want to be able to support transparent fail-over for sessions within a cluster, you are going to have to take the performance hit of persisting the session data on at least 1 other machine in the cluster after every request. If you're already taking that step, you might as well maintain an in-memory image of the serialized session object. You could compare an MD5 on the bytes comprising the session before the request was handled with the MD5 for after the request completed. Could this work? - osama. __ Do You Yahoo!? Make international calls for as low as $.04/minute with Yahoo! Messenger http://phonecard.yahoo.com/
RE: Addition of 'dirty' field to Session interface
This is just an idea from the top of my head, would it be possible having a second vector that contains a footprint(not a full clone) of the object for a session and have a reaper thread checking the footprints against the real objects and determine if they changed or not and based on that replicate of whatever we want to do. My thoughts exactly. If you want to be able to support transparent fail-over for sessions within a cluster, you are going to have to take the performance hit of persisting the session data on at least 1 other machine in the cluster after every request. If you're already taking that step, you might as well maintain an in-memory image of the serialized session object. You could compare an MD5 on the bytes comprising the session before the request was handled with the MD5 for after the request completed. Could this work? The overhead could be fairly signifigant. - osama.
Re: Addition of 'dirty' field to Session interface
Craig R. McClanahan typed the following on 12:40 PM 8/9/2001 -0700 Now that I think about it though, any time a session is used in a request, its lastAccessedTime will be updated, so the session must be considered dirty. So worrying about tracking attributes isn't necessary: the session only needs to be flagged dirty when it is retrieved. Tracking the dirty status is still a good optimization, since it ensures sessions aren't saved multiple times between requests, or after requests which never access the session. If I knew that the access time had been updated but not any attributes, I could probably distribute that information pretty cheaply (without having to serialize and deserialize the attributes as well). Thus, it's probably worth distinguishing between the two cases. But we're still stuck with trusting the user to signal that they've modified an attribute, which I'm not comfortable with. Asking them to call setAttribute() is fairly clean, portability wise, but we would be guaranteed to get a perpetual stream of developers missing that bit of the docs and asking why Catalina sometimes loses their session data across restarts. Plus people might use 3rd party code which doesn't conform to this Catalina-specific requirement. I think my suggestion of flagging any attribute retrieved with getAttribute() as dirty should guarantee modified attributes are always saved, although these would be unnecessarily saved if the attributes are only read. My opinion is that guaranteeing correctness without relying on developers following a non-standard technique is worth the trade-off. Kief
Re: Addition of 'dirty' field to Session interface
On Fri, 10 Aug 2001, Kief Morris wrote: Craig R. McClanahan typed the following on 12:40 PM 8/9/2001 -0700 Now that I think about it though, any time a session is used in a request, its lastAccessedTime will be updated, so the session must be considered dirty. So worrying about tracking attributes isn't necessary: the session only needs to be flagged dirty when it is retrieved. Tracking the dirty status is still a good optimization, since it ensures sessions aren't saved multiple times between requests, or after requests which never access the session. If I knew that the access time had been updated but not any attributes, I could probably distribute that information pretty cheaply (without having to serialize and deserialize the attributes as well). Thus, it's probably worth distinguishing between the two cases. But we're still stuck with trusting the user to signal that they've modified an attribute, which I'm not comfortable with. Asking them to call setAttribute() is fairly clean, portability wise, but we would be guaranteed to get a perpetual stream of developers missing that bit of the docs and asking why Catalina sometimes loses their session data across restarts. Plus people might use 3rd party code which doesn't conform to this Catalina-specific requirement. I think my suggestion of flagging any attribute retrieved with getAttribute() as dirty should guarantee modified attributes are always saved, although these would be unnecessarily saved if the attributes are only read. My opinion is that guaranteeing correctness without relying on developers following a non-standard technique is worth the trade-off. Kief Sounds like a policy decision we should leave in the hands of whoever deploys the application (i.e. a configuration switch :-). Craig
Re: Addition of 'dirty' field to Session interface
Craig R. McClanahan typed the following on 12:57 PM 8/8/2001 -0700 Another possibility would be to flag the session is dirty when getAttribute() is called - it would result in unnecessary saves since it assumes the attribute was modified even when it wasn't, but it would be safer. Someone has told me (haven't confirmed personally) that some app servers treat a setAttribute() -- as opposed to getAttribute() -- as the signal to set the internal dirty flag. The idea is that, if your application modifies the internal state of an existing session attribute, then you should call session.setAttribute() again to notify the container. Yes, flagging the session is dirty on setAttribute() makes sense. I was thinking that by also flagging it on getAttribute(), you're depending less on developers to take an extra step (calling setDirty() or making another call to setAttribute()). If an attribute is retrieved from the session, it may have been modified, so make the assumption that it was just to be safe. But this could erase a lot of the benefit of the dirty flag optimization, since writes are typically more common than reads. Now that I think about it though, any time a session is used in a request, its lastAccessedTime will be updated, so the session must be considered dirty. So worrying about tracking attributes isn't necessary: the session only needs to be flagged dirty when it is retrieved. Tracking the dirty status is still a good optimization, since it ensures sessions aren't saved multiple times between requests, or after requests which never access the session. Vishy, what do you think? Kief
Re: Addition of 'dirty' field to Session interface
On Thu, 9 Aug 2001, Kief Morris wrote: Craig R. McClanahan typed the following on 12:57 PM 8/8/2001 -0700 Another possibility would be to flag the session is dirty when getAttribute() is called - it would result in unnecessary saves since it assumes the attribute was modified even when it wasn't, but it would be safer. Someone has told me (haven't confirmed personally) that some app servers treat a setAttribute() -- as opposed to getAttribute() -- as the signal to set the internal dirty flag. The idea is that, if your application modifies the internal state of an existing session attribute, then you should call session.setAttribute() again to notify the container. Yes, flagging the session is dirty on setAttribute() makes sense. I was thinking that by also flagging it on getAttribute(), you're depending less on developers to take an extra step (calling setDirty() or making another call to setAttribute()). If an attribute is retrieved from the session, it may have been modified, so make the assumption that it was just to be safe. But this could erase a lot of the benefit of the dirty flag optimization, since writes are typically more common than reads. Now that I think about it though, any time a session is used in a request, its lastAccessedTime will be updated, so the session must be considered dirty. So worrying about tracking attributes isn't necessary: the session only needs to be flagged dirty when it is retrieved. Tracking the dirty status is still a good optimization, since it ensures sessions aren't saved multiple times between requests, or after requests which never access the session. If I knew that the access time had been updated but not any attributes, I could probably distribute that information pretty cheaply (without having to serialize and deserialize the attributes as well). Thus, it's probably worth distinguishing between the two cases. Vishy, what do you think? Kief Craig
Re: Addition of 'dirty' field to Session interface
Jim Seach typed the following on 04:29 PM 8/7/2001 -0700 Selected setXXX methods in StandardSession will set the dirty bit to true indicating that Session data has changed and it needs to be saved in the next save cycle by PersistentManager. But what happens if in one servlet you put an object in the session, then later, after the session has been saved, another request is handled by a different servlet that get's the object from the session and changes its state. In this case, you have to have the cooperation of the application developer to call setDirty(true) so you know something has changed. This doesn't seem like a good idea - not only is it prone to developer error as you said, it also makes any application which uses it non-portable to other servlet containers. Another possibility would be to flag the session is dirty when getAttribute() is called - it would result in unnecessary saves since it assumes the attribute was modified even when it wasn't, but it would be safer. Maybe it's possible to use reflection to detect if an object has been modified? I've seen a DB persistence package which appears to do this, although I haven't examined that part of the code (ObjectBridge, aka OJB, on sourceforge). Kief
Re: Addition of 'dirty' field to Session interface
On Wed, 8 Aug 2001, Kief Morris wrote: Jim Seach typed the following on 04:29 PM 8/7/2001 -0700 Selected setXXX methods in StandardSession will set the dirty bit to true indicating that Session data has changed and it needs to be saved in the next save cycle by PersistentManager. But what happens if in one servlet you put an object in the session, then later, after the session has been saved, another request is handled by a different servlet that get's the object from the session and changes its state. In this case, you have to have the cooperation of the application developer to call setDirty(true) so you know something has changed. This doesn't seem like a good idea - not only is it prone to developer error as you said, it also makes any application which uses it non-portable to other servlet containers. Another possibility would be to flag the session is dirty when getAttribute() is called - it would result in unnecessary saves since it assumes the attribute was modified even when it wasn't, but it would be safer. Maybe it's possible to use reflection to detect if an object has been modified? I've seen a DB persistence package which appears to do this, although I haven't examined that part of the code (ObjectBridge, aka OJB, on sourceforge). Someone has told me (haven't confirmed personally) that some app servers treat a setAttribute() -- as opposed to getAttribute() -- as the signal to set the internal dirty flag. The idea is that, if your application modifies the internal state of an existing session attribute, then you should call session.setAttribute() again to notify the container. I don't believe there is any general mechanism that a container can use to tell whether the internal state of an application object has changed in a way that is significant to the application. The setAttribute() rule would seem to be a reasonable alternative, because it lets the application tell the container about changes - in a way that doesn't require servlet API changes. Kief Craig
RE: Addition of 'dirty' field to Session interface
Best to do this by layering your own abstraction on top of Sessions. That's what we do [although we don't use it to solve this particular problem]. -tom -Original Message- From: Kief Morris [mailto:[EMAIL PROTECTED]] Sent: Wednesday, August 08, 2001 12:36 PM To: [EMAIL PROTECTED] Subject: Re: Addition of 'dirty' field to Session interface Jim Seach typed the following on 04:29 PM 8/7/2001 -0700 Selected setXXX methods in StandardSession will set the dirty bit to true indicating that Session data has changed and it needs to be saved in the next save cycle by PersistentManager. But what happens if in one servlet you put an object in the session, then later, after the session has been saved, another request is handled by a different servlet that get's the object from the session and changes its state. In this case, you have to have the cooperation of the application developer to call setDirty(true) so you know something has changed. This doesn't seem like a good idea - not only is it prone to developer error as you said, it also makes any application which uses it non-portable to other servlet containers. Another possibility would be to flag the session is dirty when getAttribute() is called - it would result in unnecessary saves since it assumes the attribute was modified even when it wasn't, but it would be safer. Maybe it's possible to use reflection to detect if an object has been modified? I've seen a DB persistence package which appears to do this, although I haven't examined that part of the code (ObjectBridge, aka OJB, on sourceforge). Kief
Re: Addition of 'dirty' field to Session interface
--- Vishy Kasar [EMAIL PROTECTED] wrote: Hi, In order to support persistent failover, we have written our own Store class that writes session data to DB of our choice. We decided to use maxIdleBackups as that will save the data periodically to disk without getting rid of it in memory. PersistentManagerBase.processMaxIdleBackups() writes the session data to DB at periodical intervals irrespective of session has changed or not. I think it will be very valuable addition to add getDirty() and setDirty(boolean) methods to the Session interface. When Wouldn't this require a change to the Servlet API? This is probably not the right list for that :) PersistentManagerBase saves the session contents, it can set dirty bit to false and from then onwards save the session only if the bit is set to true. Selected setXXX methods in StandardSession will set the dirty bit to true indicating that Session data has changed and it needs to be saved in the next save cycle by PersistentManager. But what happens if in one servlet you put an object in the session, then later, after the session has been saved, another request is handled by a different servlet that get's the object from the session and changes its state. In this case, you have to have the cooperation of the application developer to call setDirty(true) so you know something has changed. If you are going to rely on developers following some kind of convention, it could probably be done without changes to the spec. What about asking them to call setAttribute(Dirty,True) whenever they make a change? Your PersistentManager could then remove the attribute after a save. (not that I believe they would do this consistently any more than they will call setDirty(true) ) Is there something I am missing? Jim Let me know what you guys think of this useful optimization. I can contribute the code if necessary. -- Cheers! __ Do You Yahoo!? Make international calls for as low as $.04/minute with Yahoo! Messenger http://phonecard.yahoo.com/
