Hi Dorioo, Here are the results of our performance comparisons. Firstly the disclaimer, I am not trying to bag out MM here, he has done a great job on Transfer.
We run a large Transfer/Mach-II/Coldspring app (3.6k lines of Transfer XML config). We have run into the typical memory issues that large apps on this stack have. I believe these issues are being addressed with the pluggable cache updates MM is doing. We tested against Transfer 1.1 Our production environment is based on 64bit/20 gig/8proc machines. The tests were modelling a small sub-system of our application, without mach-II or coldspring. The tests were executed using a comparably small server (dual procs, 512 mb allocated to JVM). Scale wise, this machine approximated our production environment taking into account how small both the database for the tests was, and that the test classes, object graph, etc are fairly simple. SQL db was on the same physical test server. JMeter was also running on the same physical test server to eliminate network latency issues. The test code consisted on 5 level (class) One2Many hierarchy, at the bottom of that hierarchy was a M2M class. (ie, fairly linear object model) Loading a single root object would cause the loading of around 50 child objects. A percentage of the M2M’s at the bottom of the hierarchy were shared between the root objects. The fairly linear object model allowed us to test something called Bulk-Loading, ie, use a CFQuery to grab the parent, joined to each child, then a grouped CfOutput to create/load each object. The tests compared Bulk-Loading on CF8, CF9, Transfer on CF8 & CF9 and Hibernate on CF9. CF9 was a RC version. We developed a simple class caching system for both Hibernate/Bulk loading objects consisting of a manager class which is stored in application scope. It had an internal struct to store objects and automatically reaped objects based on MaxObjects and/or low memory. This only stored the Root objects, with the root object containing references to all of their child objects, which stored references to their children, etc. Although transfer caches the child object individually, we did not need to do this as our production code for this sub-system would allow us to cache the root object and let that manage its children. For the transfer test, each object had a decorator class (as they do in production), and the test would execute a method on the root object, which would execute a method on it’s child objects and so on. Transfer was set to cache in application scope. The non-transfer classes had a similar number of methods to the transfer decorator class (as well as setters/getters) Each test was run 2000 iterations (two J-Meter threads running 1000 times) & repeated 3 times with results averaged. Each config was tested against a pool of 50 objects (ie, every object ends up in cache) and 500 objects (a small percentage of objects is able to be cached and therefore we get a lot of cache thrashing, as in production) As you can see, my tests are very specific and designed to find the best way for our project to move forward. You should probably create your own test-rigs to decide which way to move forward. These tests took around 1 week to build and conduct & were very worthwhile for us. The results: CF8, 500 Root-objects (cache thrashing) Transfer with transfer cache: 1403ms/request Transfer with no cache: 1730ms/request Bulk-Loading with custom cache: 97ms/request Bulk-Loading with no cache: 106ms/request CF8, 50 Root-objects (designed for all objects to end up in cache) Transfer with transfer cache: 72ms/request Transfer with no cache: 1671ms/request Bulk-Loading with custom cache: 28ms/request Bulk-Loading with no cache: 95ms/request CF9, 500 Root-objects (cache thrashing) Transfer with transfer cache: 4880ms/request Transfer with no cache: 2455ms/request Bulk-Loading with custom cache: 91ms/request Bulk-Loading with no cache: 126ms/request Hibernate with custom cache: 66ms/request Hibernate with no cache: 63ms/request CF9, 50 Root-objects (designed for all objects to end up in cache) Transfer with transfer cache: 66ms/request Transfer with no cache: 2207ms/request Bulk-Loading with custom cache: 25ms/request Bulk-Loading with no cache: 114ms/request Hibernate with custom cache: 23ms/request Hibernate with no cache: 64ms/request Dave On Feb 7, 11:53 am, Dorioo <[email protected]> wrote: > please post results. Thank you > > - Gabriel > > > > On Sat, Feb 6, 2010 at 7:49 PM, Dave <[email protected]> wrote: > > I did some profiling of Transfer on CF9 vs CF8, and transfer was a lot > > slower on CF9 in tests caused a lot of new objects to be loaded from > > the DB (ie, cache exhaustion). > > > I suspect this is due to the fact that transfer has some custom code > > for CF8, and treating is CF9 as CF7. MM confirmed this may be the > > cause at CFObjective. > > > The results were very interesting, with Hibernate performing a lot > > better then transfer under load. > > > I can post comparisons of transfer vs hand-coded objects vs hibernate > > on CF8/CF9 if you like (obviously no results for hibernate on CF8). > > > The clear winner was Hibernate, followed by hand-coded objects, then > > transfer. > > > dave > > > On Feb 5, 5:53 am, Brian G <[email protected]> wrote: > > > Assuming you've upgraded your CF8 JVM beyond the classloader issue, > > > can anyone estimate the performance improvement for Transfer-based > > > apps from upgrading to CF9? > > > > I'm familiar with tests like Jamie Krug's object creation test but I'm > > > more interested in what kind of actual impact that CFC creation > > > improvement has on your app. E.g., on the same hardware, same app, > > > same load, etc, what kind of a performance boost, if any, did you see > > > in upgrading? > > > > Ballparks and gut feels are ok with me, thanks! > > > > Brian > > > -- > > Before posting questions to the group please read: > > >http://groups.google.com/group/transfer-dev/web/how-to-ask-support-qu... > > > You received this message because you are subscribed to the Google Groups > > "transfer-dev" group. > > To post to this group, send email to [email protected] > > To unsubscribe from this group, send email to > > [email protected]<transfer-dev%2bunsubscr...@google > > groups.com> > > For more options, visit this group at > >http://groups.google.com/group/transfer-dev?hl=en -- Before posting questions to the group please read: http://groups.google.com/group/transfer-dev/web/how-to-ask-support-questions-on-transfer You received this message because you are subscribed to the Google Groups "transfer-dev" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/transfer-dev?hl=en
