Dave wrote:
Part of the plan for Roller 4.0 is to replace Hibernate with an ORM
system that is compatible with Apache license and policy. I think
we're ready to replace Hibernate now with the new JPA based back-end
that is currently in the roller_4.0_newbackend branch. I'll explain
why but first some background so we're all on the same page.
The datamapper backend for JDO and JPA
Roller committer Craig Russell with the help of Mitesh Meswani created
a new JPA back-end earlier this year. Actually, they created a new
"datamapper" back-end that supports JDO and JPA.
The straight-to-JPA backend w/OpenJPA
Some folks questioned the need for Craig and Mitesh's extra datamapper
layer, so I created a straight-to-JPA back-end based on their work. I
got things working and tested against Toplink Essentials JPA and
Apache OpenJPA. OpenJPA seemed to work better and it's an Apache
project, so I've been working almost exclusively with it.
IBM's iBatis backend
IBM also had problems with Hibernate and it's licensing. They created
an iBatis based Roller back-end for the 3.1 code-base, but have yet to
donate that to Apache. I assume they still have to do some work to
upgrade their work to the current trunk. Also, Elias (our IBM
committer) has indicated that they'd be willing to switch to JPA as
long as JPA has reasonable performance.
Where we are now
I've been working to keep the roller_4.0_newbackend branch in sync
with the Roller trunk for a couple of months now. The newbackend
branch is configured to use OpenJPA for persistence and it's been
passing 100% of the Roller unit tests for some time.
sounds right to me.
Over the past weeks I've been doing some informal testing with JMeter
on the newbackend branch to compare it to trunk -- testing Roller
w/Hibernate native vs. Roller w/OpenJPA. I'm testing with a database
full of blogs, entries and comments and I'm trying to test a variety
of URLs. These are the forms of URLs that I'm testing:
${blog1}
${blog1}?page=3
${blog1}/entry/${entry1}
${blog1}/feed/entries/atom
${blog1}/category/General
${blog2}
${blog2}?page=3
${blog2}/entry/${entry2}
${blog2}/feed/entries/atom
${blog2}/category/General
Here are some notes on my test environment:
- MacBook Pro (2.33Mhz, 2GB RAM)
- Tomcat 5.5.23
- Derby 10.2
- Roller caching turned off
My tests are not really complete as I'm not testing all of the
different forms of URL and I'm not measuring load and memory size. Its
a very simple test. Perhaps too simple. That said, what I'm finding
with these informal tests is that OpenJPA performs the same or better
than Hibernate in terms of response time and through-put. The attached
screen shots show an example of the results I'm getting. Overall:
Hibernate: 25.73 pages/minute
OpenJPA: 34.3 pages/minute
So, straight-to-JPA appears to be working well and we're still waiting
for the IBM iBatis implementation. I think it's time to stop waiting
and make the switch to OpenJPA. What do you guys think?
personally I am still hesitant to make that switch yet, but i haven't
had time to look at the code recently. also, even if Roller were to
switch to JPA right now i am not ready to do that for our installation
so i am going to be maintaining the Hibernate backend until i have
enough time to actually fully evaluate the JPA backend and make sure
it's ready.
Here are some questions to consider:
- What additional testing & work would you like to see?
from the previous time that i spent with the JPA backend I am not truly
convinced that if JPA is passing our current tests that it means
everything is green. there are still gaps in the tests and since JPA
has some different semantics than Hibernate there is the possibility
that an operation can pass even though it didn't really succeed. i know
this is a little tough to qualify without explicit examples, but my
general feeling is still unease.
the performance tests you have are valid, but ultimately those aren't
tests of the backend. what i would really like to see us have is some
tests which work directly against the backend manager classes because
that is truly testing the performance of the backend. i don't think
this required to promote the JPA backend, but we had previously talked
about a performance comparison and IMO this is the real way to achieve
it. plus, this would just be a great analytical tool to work on
performance tuning the backend code.
- Should we wait any longer for IBM to donate an iBatis implementation?
i don't think so. unfortunately we can't wait forever :/
- Has anybody else been doing any testing?
not me, not since the testing that i did a month or two ago.
- What other objections do you have to making the switch?
echoing the concerns mentioned above, my general feeling is that there
really hasn't been enough eyes looking at the code, playing with it, and
testing it.
aside from unit testing has anyone else actually fired up a real
deployment using the JPA backend? tested it against a db full of
existing data? walked through all the operations and made sure the data
is properly being handled properly?
anyways, general uncertainty from me.
-- Allen
- How should we time & execute the switch?
Thoughts?
- Dave
