Since most production systems don't really rely upon generated schemas, it seems reasonable to hand craft a schema for each supported database that uses reasonable assumptions comparable to those that would be made in a production system (primary/surrogate keys, unique keys, foreign keys, proper column types and sizes, etc). Each db schema would be part of the source of the test, and each test starts with a virgin database with its given schema. It doesn't seem to difficult given that the domain model objects have been hand crafted.
That way, the benchmark wouldn't have to worry about schema creation or sychronization or the timings of those operations and standard JPA OR mapping metadata could be used. -matthew On Thu, Nov 11, 2010 at 8:49 AM, Michael Dick <[email protected]>wrote: > The schema generation tools might operate in unpredictable ways - > potentially skewing the results. > > Currently the test harness assumes that the schema tools will execute and > be > done executing by the time you've created the first EntityManager. This > isn't guaranteed, some tools may not execute until the first time you > access > an entity. In that case your test is, unknowingly, skewed by the shema > tools. > > I haven't looked at the tests to see whether this affects the OpenJPA > results, I'll try to take a look later today if I can find some cycles. I > do > know that when SyncrhonizeMappings is found OpenJPA obtains a lot of > connections. I put in a quick fix that reduced the use of connections, but > I > wouldn't say it's complete by any means. > > My recommendation would be to do the table generation separately. OpenJPA > provides a standalone > tool< > http://openjpa.apache.org/builds/latest/docs/manual/manual.html#ref_guide_mapping_mappingtool > >to > generate tables, maybe the other providers have similar tools. That > would > save you the time of creating a separate persistence unit, at least. > > One other thing. It looks like the test expects each EntityManager to use a > single connection until the EntityManager is closed. If that's the case you > might want to add this property > <property name="openjpa.ConnectionRetainMode" value="always"/> > > This setting is suitable when the application tightly controls the > EntityManager, and wants to minimize connection pool use. It's not suitable > in all scenarios (e.g. container managed EntityManagers), but if you want > to > force a single connection to be used this is the way to go. > > Hope this helps, > -mike > > > > On Wed, Nov 10, 2010 at 8:16 PM, Ilan Kirsh <[email protected]> wrote: > > > > > Hi Mike, > > > > Automatic schema generation is used by all the tested JPA implementations > - > > in order to simplify the benchmark. I agree, however, that it should be > > excluded from time measurement because it is not a real factor in > > production. I will try to explain what I did and I will be glad to get > your > > opinion. > > > > The time of every test run is divided into 2 phases. The first phase is > > always excluded from time measurement. Only the performance in the second > > phase is taken into account. This way, initialization operations > including > > class loading, enhancement by the Java Agent enhancer, JIT compilation, > JVM > > tuning (inlining and other optimizations) and schema generation - are all > > excluded from time measurement. > > > > Connecting to the database (invocation of createEntityManager) is also > > completed before starting to measure time to avoid the influence of > > connection pools. > > > > Given the above, do you still think that the SynchronizeMappings property > > might affect the results. If so, how? > > > > If necessary I may generate 2 persistence units for every test (per > > ORM/DBMS) - one for initialization (including schema generation) and the > > other for the time measurement. But I want to keep the benchmark simple > as > > possible, so I will do that only if it is really necessary. > > > > Best regards, > > Ilan > > -- > > View this message in context: > > http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5727348.html > > Sent from the OpenJPA Users mailing list archive at Nabble.com. > > > -- mailto:[email protected] skype:matthewadams12 yahoo:matthewadams aol:matthewadams12 google-talk:[email protected]<google-talk%[email protected]> msn:[email protected] <msn%[email protected]> http://matthewadams.me http://www.linkedin.com/in/matthewadams
