Re: [CFCDev] CFC Performance question

[Joe Eugene]

Just a thought, you can try to store the Raw data as Collection Objects in the application scope and have methods manipulating the Collection Object. I think this might be a lot more faster, since Java Collection Objects are meant to do this Job very well.   Joe Eugene       ----- Original Message ----- From: Jim Davis To: [EMAIL PROTECTED] Sent: Monday, November 10, 2003 2:12 AM Subject: RE: [CFCDev] CFC Performance question Sort of � the idea is to load the entire festival into memory and use it from there � which I (stupidly) assumed would be faster than constant DB hits.   The component methods all hit the DB as needed � but cache their information from then on.  So if I call �venue.getChildren()� it checks in the DB for it�s children and creates them (as a special �venue collection� component containing other venues) � but the second time it pulls from cache instead (unless a special �ForceRefesh� argument is passed).   In this case I AM using the components as a datastore � sort of.  More of a cache, really.  I do this for several reasons:   1) I can cache the whole festival (which is never going to be larger than a certain size) in memory with all of it�s assets �set up�.  For example an event event is linked to several schedules which are linked to venues.  In the DB I do a join on several tables which creates an �event� containing a �ScheduleCollection� which contains �schedules� which are linked to �Venues�   To access this I do (for example) �Event.getSchedules().sortByBeginTime()�.  The first time through this hits the DB (of course) � but if I persist it following calls never touch the DB again (eliminating massively redundant calls).  Many of these calls are somewhat difficult to cache using traditional query caching (I used to cache the queries in the application scope and just pull single records from them - but that was in CF 4.5).   2) I can build the supporting systems completely blind to the database.  In other words my basic component model knows about the database, but my administration and end-user system don�t have to.  Even my component model is only aware a little of the database � all of my DB access code is abstracted into include files.  I can add libraries of these include files to take advantage of any DBMS I wish (or not use one at all � it would be pretty trivial to use XML files as the datastore for example).   Actually with the way it�s built the same component model can take advantage of multiple DBMS systems simultaneously with no modification.  I�m actually quite pleased with that little trick.   3) Since the Festival information is stored in the application scope I leave the database free to do other things� like store user preferences.  We get a LOT of traffic on the day of our festival but have never had the DB resources to spare to store much metrics information � this year we�re storing clickstreams, events selected (via an �Interactive Festival Planner�) and so forth.  In other words were not leaving the DB idle.  ;^)   4) By caching the festival information in the Application scope we more easily control when edits are exposed.  We didn�t have the time (or money) to build a real content workflow system so this makes for a usable kludge.  The Festival editor can makes changes, review them and them even save them � but they won�t be seen by the end users until they refresh the cached festival.   By storing multiple festivals in cache we can actually present multiple bodies of information depending on audience type in this way.  Nothing that couldn�t be done in other ways, but this way it�s just much simpler and cleaner (in my opinion).   5) Because the data is maintained as CFCs (instead of application.scoped queries as I did with CF 4.5) I can easily create a web service interface to allow interested media outlets to fetch the data in a �raw� form.  This won�t happen this year� but I hope to do it for next year�s festival.     The model doesn�t require everything to be cached at all.  The �festival� is the top level object and works as a container for all others, but you can easily just instantiate a festival, add one event, edit that, save it (to the database), then throw it away.  It�s fast and painless.   In use however I want to cache everything to the application scope and use it from there.  This is to improve performance.  Once this is done performance actually does improve markedly � however the caching process itself was dead-dog-slow.  Doing it �right� (calling the proper getters and instantiating each component correctly) was taking upwards of eight minutes to run on my dev box.   However doing it wrong (just for this mass cache method) and setting all of the properties for the assets directly dropped the cache time down to the 10-20 second range.  This works � and the result is a (for the most part) lightning fast cache.   Even then however there are odd weird performance dips: a method that takes 0ms 1,000 times will be reported as taking 60 seconds once.  There�s no specific load involved (in fact the machine is barely sweating) and no outward sign of trouble � but it just hangs (seemingly, I�m guessing waiting for some timeout someplace).   I�d like to track down why that�s happening� but it�s so inscrutable I doubt I�ll find it�   Jim Davis   -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Joe Eugene Sent: Monday, November 10, 2003 1:10 AM To: [EMAIL PROTECTED] Subject: RE: [CFCDev] CFC Performance question   Jim,   It seems to me, you are trying to simulate a DataBase with Objects OR I am really missing something with your OOD. --------------- I�m seeing similar issues: create a CFC (or several) that work together and all is well � but use them several times and performance tanks. In my case I�ve got a �Festival� CFC that contains many �Venues� (about 60), �Events� (about 100), �Artists� (about 100) and �Schedules� (which link events to Venues at particular times).  -------------   You can use a Class as a representation of your Business Object Model, Not to as middle tier data repository. I recall someone posted something about creating object instances of every single DB record...I dont see the point on this.   I belive you have to do Abstraction towards Enterprise Objective, so your application can be easily maintained as your business rules change.   I might be mis-understanding what you are trying to accomplish with your design.   Joe Eugene   -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]On Behalf Of Jim Davis Sent: Sunday, November 09, 2003 10:34 PM To: [EMAIL PROTECTED] Subject: RE: [CFCDev] CFC Performance question I�m seeing similar issues: create a CFC (or several) that work together and all is well � but use them several times and performance tanks.   In my case I�ve got a �Festival� CFC that contains many �Venues� (about 60), �Events� (about 100), �Artists� (about 100) and �Schedules� (which link events to Venues at particular times).  When this is empty everything runs fine.  Add in all the information and performance get really loopy.  Sometimes fully populating the festival (which is then cached in the Application scope) takes as little as 8 seconds � other times as long as 2 minutes.   This is on a completely dedicated development system with no (observable) other load (at least nothing that�s appearing in PerfMon) and a single user.  In the debug output everything will fly then I�ll see one otherwise simple method call (something like a Venue.new()) taking 70 seconds or a series of simple calls (something like venue.getparent() which runs once for each venue and normally averaging 0ms taking 1200ms each).   I admit this may be an issue with servers� but I see the exact same behavior on my hosted production server and on another box I set up to test.   That being said even when I�m not seeing inexplicable performance dips CFCs definitely don�t seem to scale linearly � or, to be fair, at least MY CFCs don�t. ;^)   In general I guess all I�m saying is make sure to test them both under load and with as many instances as you�ll be using in production.   And it anybody has any suggestions for me to tame my beast I�m all ears!   Jim Davis       -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andy Ousterhout Sent: Sunday, November 09, 2003 2:48 PM To: Cfcdev Subject: [CFCDev] CFC Performance question   I have a CFC that processes sales orders.  It has a Customer.cfc and Product.cfc to validate each of these properties.  On my dev machine, it took 591ms to create and save 1 order with 5 items.  Should I be satisfied with this performance or should I look at ways to improve this?  The majority of this time was in the 5 calls to the  AddItem method which has a product CFC verify item information, with the first call taking 80ms, then 50ms, 50ms, 30ms and 30ms.   I am using an Access DB and am in the process of moving to MS SQL 2000.  Should this improve performance?   Andy