On 6/18/10 12:47 PM, Adam Hardy wrote: > 1) I'm using tomcat, so no Java EE container. I'm using Spring to handle > the transactions. > > 2) I'm using one EMF for the whole app from app start, and one EM for > the transaction or sometimes 2, launched by a tomcat filter which opens > it at the start of the HTTP request and closes it at the end. And the > two transactions I'm talking about are 2 distinct user operations, so > each gets one EM. > > 3) Build-time enhancement. > > 4) Actually no because in OpenJPA 1.2.2 there is a bug that throws NPEs > and means I can't run with it enabled. I just upgraded to 2.0.0 I think > I've got the same bug - albeit a different stack trace - but a quick > google didn't show up any immediately available work-around. I guess > this is the key issue, right? >
Which JIRA and what does your 2.0.0 stack trace look like? > > > Here's my persistence xml > <persistence-unit name="OpenJpaJdbc"> > <description>Pattern Repo JPA Config with OpenJPA</description> > > <provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider> > > <mapping-file>org/permacode/atomic/domain/entity/AtomicEntity.xml</mapping-file> > > > <mapping-file>org/permacode/patternrepo/orm/BrokerAccount.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/Category.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/DollarReturn.xml</mapping-file> > <mapping-file>org/permacode/patternrepo/orm/Exchange.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/ExchangeSymbol.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/Fill.xml</mapping-file> > <mapping-file>org/permacode/patternrepo/orm/Market.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/MarketSymbol.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/MarketSystem.xml</mapping-file> > <mapping-file>org/permacode/patternrepo/orm/Order.xml</mapping-file> > <mapping-file>org/permacode/patternrepo/orm/Pattern.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/Portfolio.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/PortfolioItem.xml</mapping-file> > > > <mapping-file>org/permacode/patternrepo/orm/RequiredParam.xml</mapping-file> > > > <mapping-file>org/permacode/patternrepo/orm/TestAnalysis.xml</mapping-file> > <mapping-file>org/permacode/patternrepo/orm/TestRun.xml</mapping-file> > <mapping-file>org/permacode/patternrepo/orm/Trade.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/TradingParam.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/TradingSystem.xml</mapping-file> > > > <mapping-file>org/permacode/patternrepo/orm/Weighting.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/WeightingGroup.xml</mapping-file> > > > <mapping-file>org/permacode/patternrepo/orm/MiniAnalysis.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/MiniResult.xml</mapping-file> > > <mapping-file>org/permacode/patternrepo/orm/TestQueries.xml</mapping-file> > <properties> > <property name="openjpa.RuntimeUnenhancedClasses" > value="unsupported" /> > <property name="openjpa.Log" value="SQL=TRACE"/> > <property name="openjpa.ConnectionFactoryProperties" > value="PrettyPrint=true, PrettyPrintLineLength=72" /> > </properties> > </persistence-unit> > > > > > Donald Woods on 18/06/10 13:36, wrote: >> Some quick initial questions - >> 1) Are you using Java SE or a Java EE container (more of a transactions >> and XA question)? >> 2) Are you using one EMF and EM for both the parent, child and >> grandchild operations? Are you calling flush() anywhere? >> 3) Are you using build time or run time enhancement of entities? >> 4) Have you enabled the DataCache or QueryCache? >> http://openjpa.apache.org/builds/2.0.0/apache-openjpa-2.0.0/docs/manual/manual.html#ref_guide_cache_query >> >> >> Also, what do the entities and your persistence.xml look like? >> >> >> -Donald >> >> >> On 6/18/10 8:11 AM, Adam Hardy wrote: >>> I have a transaction that submits around 20K records, followed by >>> another transaction in the user process which inserts the same into a >>> related table. >>> >>> The performance issues I talked about below initially caused processing >>> times of 45 mins, but I worked on the Java in the code and I tweaked the >>> mySQL database and reduced it to 15 mins. >>> >>> This though is still a problem - I've been over the optimization >>> guidelines in the documentation and there's nothing there that I can >>> implement that I'm not already. >>> >>> The first transaction I mentioned takes 5 mins, but the second takes >>> 15mins and is inserting child records of the records created in the >>> first transaction. It looks like OpenJPA is fetching all of those first >>> records again. Shouldn't they already be in memory? >>> >>> >>> Thanks >>> Adam >>> >>> Adam Hardy on 12/06/10 13:21, wrote: >>>> I am trying to get a handle on what I should be able to achieve. >>>> >>>> Can someone give me some idea of the metrics I should be able to get >>>> optimistically when persisting an object that has a child list with >>>> 20,000 child objects and 20,000 grandchildren? (one-to-one child -> >>>> grandchild) >>>> >>>> Can I reasonably expect to get this done in under a minute? >>>> >>>> I think that would work out at a rate of about 1.5 milliseconds per >>>> object. >>>> >>>> Thanks >>>> Adam >>>> >>>> Adam Hardy on 11/06/10 17:34, wrote: >>>>> I have performance problems with large lists of beans due to the base >>>>> class I am using for my entities. >>>>> >>>>> This is slightly non-OpenJPA specific, so I hope nobody minds, but it >>>>> is Friday afternoon so I'm hoping you give me a bit of slack here. >>>>> >>>>> The problem arises when I start building lists with over 10,000 items >>>>> on a parent class. >>>>> >>>>> The trouble is in the base class for the entities, which is quite >>>>> clever (but obviously not clever enough) and it has non-facile >>>>> equals() and hashcode() algorithms which make use of reflection. It's >>>>> here that the slow-down comes. >>>>> >>>>> When I link the child with a parent that already has 10,000 children, >>>>> the equals() method is called by ArrayList before the new child is >>>>> placed in the index. >>>>> >>>>> As far as I can tell I have a couple of options. >>>>> >>>>> (1) ditch the reflection-based equals method and hard-code an equals >>>>> method. >>>>> >>>>> (2) don't use ArrayList but find a Collection-based class that uses >>>>> hashes or similar to identify items instead of equals. This is just >>>>> speculation - perhaps there is no such thing or it wouldn't help >>>>> anyway: >>>>> >>>>> - would a collection using hashes caches the hashes of the items >>>>> already indexed? >>>>> - would such a collection be persistable? >>>>> >>>>> If anyone has been in this situation before, or has an idea about it >>>>> , I'd really appreciate the help. >>> >> > >
