Yeah, it appears we're looking at two very different beasts, and a solution for one won't work on the other. I've pretty much only ever used infrequently updates caches, which puts a very different spin on things (including two layers, memory and filesystem, in some cases), versus frequently updated cacheing where the lifespan of a cached item is measured in minutes rather than days or weeks.
barneyb --- Barney Boisvert, Senior Development Engineer AudienceCentral (formerly PIER System, Inc.) [EMAIL PROTECTED] voice : 360.671.8708 x12 fax : 360.647.5351 www.audiencecentral.com > -----Original Message----- > From: S. Isaac Dealey [mailto:[EMAIL PROTECTED] > Sent: Wednesday, March 26, 2003 11:47 AM > To: CF-Talk > Subject: RE: J2EE session replication -- application replication? > > > > I agree with the incremental regeneration of app vars. > > Passing the whole > > structure probably isn't smart, unless you need to. I'm > > not sure that > > making each server recreate the data is a good idea, > > though. I typically > > put stuff in the app scope that is expensive to create and > > rarely changes. > > Thus, passing around a large seralized packet isn't a huge > > deal, since it's > > not going to happen all that often. And some situations > > (like content > > caching) can probably go without synching, althuogh it > > might be faster to > > sync up, rather that regenerate on each server. > > Maybe I should elaborate a bit more. > > As an example, in my application there are a lot of objects drawn out of a > view tap_Library, however, one of the columns in that view contains an xml > packet. So because deserializing that xml packet is somewhat expensive, > rather than deserializing it every time I want to get the object, > I store it > in an application variable application.myap.objects.F3UD3F-blah > blah... you > get the idea... it's a number or it's a UUID but it's stored in a > structure > there ... > > Now individual objects (these are content objects like Spectra has) may be > updated "frequently" meaning daily or so or multiple times a day (you're > working on a page and you make periodic edits throughout the day and you > save changes any time you step away from the workstation)... but they're > never going as frequent as the views for that object. You may > open an object > to look at it and then close it because it's not the one you needed, or > someone else or a couple other people may open it in "read-only" > mode at the > same time, or people may view the "details" or fire other of its methods > (add content, upload image), that don't update the object itself. > > It does use up some memory, but memory is less expensive than processors / > processor speed, and certianly less expensive in terms of retreival than a > combination cfquery / cfwddx|xmlparse, so there's a noticeable difference > (believe me, on some pages it's the difference between 5 and 50 seconds to > the end-user) when the data is cached. Which is why I've stuck with this > cacheing routine for the past few years. > > My gut feeling about passing an xml packet through cfhttp to > accomplish this > is that it wouldn't speed anything up actually it could potentially slow > things down. Lets say for instance you have 5 servers, and the object is > being viewed on 2 and it gets updated on 1. The routine would pass the xml > packet to all 4 other servers. One of them would have to deserialize the > object and store it in the application scope. The other 3 may or > may not. It > could be wasted processor power and memory if they do since no > one can tell > if someone else is going to view the object on that server. Even if they > don't deserialize the xml packet then you've spent what could in the long > run turn out to be considerable bandwidth passing around those xml packets > between servers when you have dozens or hundreds of people > working with the > content on 4 or 5 servers. > > > The "processing after CFFLUSH in OnRequestEnd" idea is a > > damn good one that I'd never though of. Being a FB guy, > > OnRequestEnd.cfm is usually reserved for debugigng output > > during development and such, but that's a > > very good use for it. > > Thanks. :) Yea, FB sort of does it's own thing. > > Interrestingly enough (and I'm sure some folks on this list will think I'm > smoking crack) in the prototype for the next version of Tapestry I've > started working with a file-upload routine which uses OnRequestEnd.cfm to > ensure that files being uploaded for new objects are properly associated > with the object in the db and stored in the proper directory in the file > system. The problem was that the way the application is designed, > the files > would be uploaded before the object was inserted into the db, so no > relationship could be established at the time the object is uploaded > (violation of foreign key constraint). > > So instead what I've done is I've established the object's new ID (UUID), > uploaded the file and stored information about it in the request > scope. Then > in OnRequestEnd.cfm after the db record for the object has been > inserted it > moves those files from a user-specific upload directory to the appropriate > directory for the object and inserts the related record in the database. > There's a try-catch block around each file insert/move so that if it fails > on one it doesn't kill the rest of the OnRequestEnd code, and > then it scans > the user-upload directory and attempts to delete any files that are left > there for garbage collection. > > > barneyb > > > --- > > Barney Boisvert, Senior Development Engineer > > AudienceCentral (formerly PIER System, Inc.) > > [EMAIL PROTECTED] > > voice : 360.671.8708 x12 > > fax : 360.647.5351 > > > www.audiencecentral.com > > >> -----Original Message----- > >> From: S. Isaac Dealey [mailto:[EMAIL PROTECTED] > >> Sent: Wednesday, March 26, 2003 10:45 AM > >> To: CF-Talk > >> Subject: RE: J2EE session replication -- application > >> replication? > >> > >> > >> > I know the use of an HTTP request was previously marked > >> > as > >> > a poor idea, however I'm not sure I agree as I've > >> > thought > >> > this through. Here's a solution that I think would work > >> > quite well, assuming you encapsulate all your app vars > >> > in > >> > a single object (which is a good idea anyway, IMO). Of > >> > course, I might be suggesting rebuilding JMS, which I > >> > know > >> > exactly nothing about, but oh well. > >> > >> Yea, my one large project (Tapestry) does encapsulate all > >> its application > >> variables into its own structure, however, in this case > >> it > >> doesn't help much > >> using what you've described below, and I'll explain why > >> here in a moment. > >> > >> > - server 1 learns it needs to update it's app vars > >> > (exactly how is app dependant). > >> > - server 1 updates the app vars object however it > >> > needs to. > >> > >> This much is already being done prior to clustering, so > >> we're okay here. > >> > >> > - server 1 serializes the app vars object > >> > (probably want XML, rather than the built in binary > >> > format). > >> > >> That's an idea... Although if the application is rather > >> large > >> (and caches a > >> lot of data), serializing the structure into an xml > >> packet > >> whether it's wddx > >> or otherwise could be rather time consuming. At some > >> point, xml > >> serialization then defeats the purpose of cacheing data > >> in the application > >> scope instead of going to the db server every time, so > >> imho it's > >> much better > >> to just deal with the data that's been changed. It could > >> be that you pass > >> both an xml packet and a string representing the path to > >> the data you're > >> updating, such as > >> > >> <xml blah blah> ... </xml> > >> application.myapp.someobject.stuff.3XLN7 > >> > >> and the clustered app receiving this messag then coppies > >> the deserialized > >> content of that xml packet into the indicated variable. > >> Should take a lot > >> less time than serializing the entire application.myapp > >> structure. > >> > >> > - server 1 gets a list of cluster members (exactly how > >> > will be app/container/cluster controller dependant). > >> > >> I'd go for a cachedafter query here. This allows me to > >> not have to > >> continually go to the db to get the list of server and > >> use the same > >> application sharing routine to flush the cachedafter > >> timestamp and update > >> the query to then add a new server to the list without > >> having to > >> do anything > >> special like restarting the application or worse > >> restarting the cf server. > >> > >> > - for each server, server 1 calls a secured web > >> > service, > >> > passing the serialized app vars object, which the > >> > other servers then > >> > deserialize and store in their app scope. > >> > >> I don't know that it needs to be a web service. I could > >> do the same thing > >> with CF5 and "secure" it by using a password or > >> password-like > >> algorithm, or > >> better yet by only allowing other servers in the cluster > >> to submit to the > >> page receiving the data using the same server list that > >> was used on the > >> sending end and matching that list against the > >> cgi.remote_addr > >> variable. And > >> what might actually be a best case scenario is to have > >> each individual > >> server in the cluster have a single receive-from and > >> send-to address, such > >> that the cluster is linked in a ring like a token-ring > >> network or a > >> web-ring. ;P The signal sent from the originating server > >> is sent > >> to only one > >> machine which then receives it and serializes it in a > >> queue using <cflock > >> name="[myappname]_appshare"> with a long timeout limit in > >> addition to any > >> locking on the application scope -- this would just > >> ensure that all the > >> appshare commands are processed in sequential order. If a > >> given machine in > >> the ring is down or otherwise inaccessible, the machine > >> attempting to send > >> to it moves on to the next machine in the list when it > >> doesn't get an "ok" > >> response, so that any individual server in the ring won't > >> break the entire > >> mechanism. The ring itself ensures that the load created > >> by the routine is > >> distributed evenly amongst the servers and that refresh > >> commands are > >> processed in sequence. And this of course solves the > >> problem described > >> below. > >> > >> > The last piece of the puzzle would be making sure more > >> > than one server > >> > doesn't start that process at any given time, but that > >> > shouldn't be a big > >> > deal. Call another web service method after step one > >> > that > >> > tells all servers > >> > not to start updating, and handle collisions (if two > >> > servers start down > >> > that new step at the same time) by alphabetic ordering > >> > of > >> > server name or > >> > something. > >> > >> > Feel free to tell me I'm a jackass, but that seems to > >> > me > >> > like it'd work > >> > well, and be very app independant, so it could be > >> > reused > >> > over and over. > >> > >> Not a jackass, it was much the same sort of mechanism I > >> had in mind but > >> wasn't entirely keen on using. Although now that I've > >> read yours and put > >> some more thought into it I'm a little less anxious about > >> actually > >> implementing it. > >> > >> I'm still concerned about sending an xml packet between > >> the machines and I > >> think I have a solution for that as well. All servers in > >> the cluster > >> retrieve their cache data initially from the same source > >> (a single db or a > >> clustered db, but it's all the same data in either case). > >> That > >> data from the > >> db is then cached in the application scope. And in the > >> case of my > >> application in many instances it's already choosing to > >> simply delete data > >> from the application scope rather than writing over it, > >> which then forces > >> the application to go back to the db when the data > >> doesn't exist in the > >> application scope. So the simplest solution (without > >> going to JMS) is to > >> send a one-line command to each machine in the ring with > >> two variables > >> indicating the originating server and the application > >> data to refresh. The > >> receiving server deletes the variable, sends the "ok" > >> response > >> back and then > >> sends the same command on to the next server in the list > >> and waits for the > >> ok back. If it doesn't receive the ok, it moves on to the > >> next > >> server in the > >> ring until it receives an ok or reaches the originating > >> server. > >> > >> And best of all, to prevent this process from slowing the > >> users down, you > >> run the routine from the originating server using request > >> scope > >> data in the > >> OnRequestEnd.cfm template after a <cfflush> tag. Only the > >> first request > >> occurs on a user requested page (and that only after the > >> page has been > >> delivered) -- remaining requests occur in the "ether" > >> between the machines > >> and so no one is affected (speed of delivery) by those > >> transactions. And > >> since the data's being deleted and then automatically > >> recreated > >> from source > >> on demand, you don't have to worry about anyone having > >> cache data that's > >> "out of sequence". > >> > >> s. isaac dealey 954-776-0046 > >> > >> new epoch http://www.turnkey.to > >> > >> lead architect, tapestry cms http://products.turnkey.to > >> > >> tapestry api is opensource http://www.turnkey.to/tapi > >> > >> certified advanced coldfusion 5 developer > >> http://www.macromedia.com/v1/handlers/index.cfm?ID=21816 > >> > >> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > ~~~~~~~~~~~| > > Archives: > > http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 > > Subscription: http://www.houseoffusion.com/cf_lists/index. > > cfm?method=subscribe&forumid=4 > > FAQ: http://www.thenetprofits.co.uk/coldfusion/faq > > Get the mailserver that powers this list at > > http://www.coolfusion.com > > > Unsubscribe: http://www.houseoffusion.com/cf_lists/uns > ubscribe.cfm?user=633.558.4 s. isaac dealey 954-776-0046 new epoch http://www.turnkey.to lead architect, tapestry cms http://products.turnkey.to tapestry api is opensource http://www.turnkey.to/tapi certified advanced coldfusion 5 developer http://www.macromedia.com/v1/handlers/index.cfm?ID=21816 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/cf_lists/index.cfm?method=subscribe&forumid=4 FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Your ad could be here. Monies from ads go to support these lists and provide more resources for the community. http://www.fusionauthority.com/ads.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
