Hi please see inline ...
Werner wg> -----Original Message----- wg> From: Vijayanand Sukumar [mailto:[EMAIL PROTECTED] wg> Sent: Thursday, May 12, 2005 4:07 AM wg> To: [email protected]; [EMAIL PROTECTED]; wg> [EMAIL PROTECTED]; [EMAIL PROTECTED] wg> Subject: RE: [castor-user] Transaction locks and muti wg> threaded access wg> wg> wg> wg> All, wg> wg> Thanks for all your advice on how to improve the performance. wg> wg> Here are the things I did. wg> wg> 1. Switched to Castor 0.9.6. wg> 2. Used JDO2. (Though even earlier we were using a single wg> JDO instance with wg> JDO). wg> 3. Made my transactions as short. wg> 4. Made the transactions as read-only. wg> 5. used db.load() instead of db.execute() wg> I could not use db.load() where I need to query for a wg> list of data. I don't know if I can do that, but from looking at wg> the javadoc , I think we have to use db.execute() for querying a list of wg> data(i.e when a non primary keys are used to query) Correct. Database.load() is only useful with lookups by primary key. wg> 6. Cached the objects time-limited. wg> wg> We have been using DBCP JNDI with tomcat with wg> org.apache.commons.dbcp.BasicDataSourceFactory. wg> wg> We had performance improvement by a factor of 6. :). Is wg> there anything more that I can do ? Wait for 0.9.9, a feature release scheduled for end of June. In general, please consult with the Castor roadmap at http://castor.codehaus.org/browse/CASTOR for details about the upcoming releases. In particular, this release will include a complete rewrite of the Castor-internal transaction context that will greatly improve performance, especially when the number of objects loaded as part of a transaction is high. wg> Does castor have a distributed cache to use in clustered environment ? Not yet. But as per release 0.9.9, we will support several distributed caches, incl. oscache, jcs, jgroup and coherence. For details about this feature, please have a look at http://jira.codehaus.org/browse/CASTOR-1102. wg> Since the objects we configured in castor's cache are wg> time-limited, (in our case 15 mins), in a clustered environment different wg> instances of castor(in diff app servers) have their own cache. How will the cache wg> in the second instance be updated if the cache in the first instance is wg> updated via the> application. With the current cache implementatiosn available, not at all. You'll have to wait for support for distributed caches to be added with release 0.9.9. Or even better, use the patch from the issue mentioned above, apply it to Castor sources and help us on testsing. wg> wg> Thanks for all your help. wg> wg> - Vijay wg> wg> wg> wg> wg> -----Original Message----- wg> From: Gregory Block [mailto:[EMAIL PROTECTED] wg> Sent: Saturday, April 30, 2005 4:30 AM wg> To: [email protected] wg> Subject: Re: [castor-user] Transaction locks and muti wg> threaded access wg> wg> On 27 Apr 2005, at 11:30, Ralf Joachim wrote: wg> > I know that Gregory Block (another castor commiter) is wg> using castor in wg> > a high-volume application. wg> wg> Indeed we do. wg> wg> > 1. Switch to version 0.9.6 of castor as we have fixed wg> some bugs that wg> > may cause some of your problems. wg> wg> Sidenote: Performance has, generally, improved recently. wg> If you're not wg> seeing performance improvements, then it's worth spending wg> some time thinking wg> about why. wg> wg> > 2. Initialize your JDO or JDO2 (will be renamed to wg> JDOManager at next wg> > release) instance once and reuse it all over your application. wg> > Don't reuse the Database instances. wg> wg> Again: Never, ever reuse a database instance. Creating them is wg> inexpensive, and JDBC rules state that one thread -> one wg> JDBC connection is wg> the rule. Do not multithread inside of a Database wg> instance; as a corrolary, wg> do not multithread on a single JDBC connection. wg> wg> > 3. Use a Datasource instead of a Driver configuration as wg> they enable wg> > connection pooling which gives you a great performance wg> improvement. wg> wg> I highly suggest DBCP, here, with the beneficial use of wg> prepared statement wg> caching. wg> wg> Should you be running on a system where read performance is wg> critical, feel wg> free to take the SQL code generated by castor, and dumped wg> to logs during the wg> DB mapping load in debug output, and turn those into stored wg> procedures that wg> you then invoke via call to perform those loads; however, I wg> find personally wg> that stored procedures would be a minimal improvement over wg> the DBCP prepared wg> statement cache; your mileage may vary. db.load() has wg> performance benefits wg> that are worth keeping, IMO, and the pleasure of having wg> pretty stored wg> procedures in your database is far outweighed by the wg> nightmare of change wg> management. wg> wg> > 4. Always commit or rollback your transactions and close wg> your Database wg> > instances properly also in fail situations as suggested by Nick wg> > previously. wg> wg> Just the obvious general rule on Java objects that hold wg> resources: wg> Don't wait for the VM to finalize to have something happen wg> to your objects wg> when you could have released critical resources at the wg> appropriate point in wg> the codebase. wg> wg> > 5. Keep your transactions as short as possible. If you wg> have an open wg> > transaction that holds a write lock on an object no other wg> transaction wg> > can get a write lock on the same object which will lead to a wg> > LockNotGrantedException. wg> wg> Also keep in mind that folks using lockmode of dblocked do wg> FOR UPDATE calls wg> on things they read while the transaction is open; if you're wg> using dblocked mode, be aware of how your application does wg> things. wg> If you're in one of the other modes, locks happen inside wg> castor, and it's wg> your responsibility to always use the right access mode wg> when accessing wg> content. wg> wg> If you can, for example, decide at the API layer whether or wg> not an operation wg> is going to ever need to modify an object, and know that wg> you will only ever wg> use an instance in read only mode, load objects with access wg> mode read only, wg> and not shared. wg> wg> Limit use of read-write objects to situations in which it wg> is likely you will wg> need to perform updates. wg> wg> Read-write performance will change dramatically once the wg> TransactionContext wg> patches have been checked in; if you'd like to guinea pig wg> them, check JIRA wg> and try the patch out; we're already using it in production here. wg> wg> > 6. Query or load your objects read only whenever wg> possible. Even if wg> > castor creates a lock on them this does not prevent other wg> threads from wg> > reading or writing them. Read only queries are also about 7 times wg> > faster compared with default shared mode. wg> wg> Cannot stress how important this is: If 99% of your wg> application never wg> writes an object, and you as a programmer know it won't, wg> then do something wg> about it. If you're in a situation where you want the object to be wg> read-only most of the time, and only want a writable every wg> now and then, do wg> so just-in-time by performing a load-modify- store wg> operation in a single wg> transaction for the shareable you want. wg> wg> In other words: Don't use read-write objects unless you wg> know you're likely wg> to want to write them. wg> wg> > 7. If there is a possibility you should prefer db.load(Class, wg> > object) over db.execute(String). I suggest that as wg> db.load() first wg> > tries to load the requested object from cache and only wg> retrieves it wg> > from database when it is not availble there. When wg> executing queries wg> > with db.execute() the object will always be loaded from database wg> > without looking at the cache. You may gain a improvement wg> by a factor wg> > of 10 and more when changing from db.execute() to db.load(). wg> wg> I've never touched db.execute() - it never struck me that wg> you'd want to. :) wg> wg> > a. If you have a look at http://jira.codehaus.org/browse/ wg> > CASTOR-1085 where a patch to TransactionContext is attached that wg> > improves read/write performance with a factor of 3. Even wg> if the patch wg> > passes all tests of castor test framework it requires wg> more testing wg> > before we will integrate it in our next major release. As wg> stated in wg> > the comment Gregory will use the patch in his production wg> environment wg> > sooon. wg> wg> It's in production now; and several large content imports wg> of guide content wg> have been run through it without any difficulties or problems in the wg> generated content. wg> wg> wg> Now, there's lots left to do - there is still the issue, wg> for example, of wg> dependent objects being slightly sub-optimal in performance wg> both in terms of wg> the SQL that gets generated and the way it gets managed - wg> but there will be wg> improvements over time to the way that this and other operations are wg> performed. wg> wg> But performance *should be good right now*. If it isn't, wg> you'll need wg> to think about whether you are using the optimal set of wg> operations. wg> No environment can predict your requirements - hinting to wg> the system when wg> objects can be safely assumed to be read-only is vital to a high- wg> performance implementation. wg> wg> Cheers, wg> Greg wg> wg> wg>
