I'm an engineer at BEA/Weblogic and wish to respond to William
Louth's statements.
------------------------------------------------------------------------------
"Louth, William (Exchange)" wrote:
>
> I have recently created a report(html based) on this subject for my current
> employer if you wish for a copy I can send it to you zipped up. It has not
> been proof so aplogies for any inaccuracies. I have pasted into my email
> editor so things might not be correctly layed out and tables and charts not
> included. The result that comes out is that if your where to use cmp with
> IAS you would only suffer 10% overhead. If you chose web logic you would
> suffer a 1868% peformance overhead. IAS 4.0 is roughly 16 times faster than
> WL ...that is if your where to not right any plumbing for them i..e
> isModified. Also with isModified we have issues with clustering as stated
> into the WL documentation.
Thanks, William, for your report.
I wish we had gotten some time to respond before you sent this missive out.
Needless to say, we challenge all these numbers.
> 2. Tests
>
> The following test where done using 2 application servers for the purpose of
> ensuring that a method chosen is ideal across different EJB containers. The
> 2 application servers used are Inprise Application Server 4.0 and Web Logic
> 4.51.
We inadvertently forgot to turn off detailed assertion checking
("debug=true") in the basic 4.51 release. This was discovered quickly
enough and has been fixed in service pack 2 (out since last
september). Please verify you have this SP.
> 2.1 Inprise Application Server 4.0 Test Results
>
> Session Bean + SQL (ms)
>
> .... some table.....
>
> * The naming service was run out of process along with the ejb container.
> ** This figure includes the cost of creation of the initial database
> connection which is incurred by Web Logic at server startup.
I don't understand what the cost of the creation of weblogic
connections is doing inside an IAS test result.
> Session Bean + Container Managed Persistence Entity Bean (ms)
> ...........some table..........
> * The naming service was run out of process along with the ejb container.
> ** This figure includes the cost of creation of the initial database
> connection which is incurred by Web Logic at server startup.
Again, what's weblogic doing here in an IAS test result?
> >From the figures above we can see that with the entity bean method in a
> single user enviroment we suffer a 10% peformance overhead. This seems quite
> acceptable considering that we obtain database independence (i.e persistence
> mapping) and a considerably reduction in code related to JDBC. Also with the
> entity method we only have one code base for both read only and read write
> access.
>
> 2.2 Web Logic 4.51 Test Results
>
> Session Bean + SQL (ms)
> ....some table.....
> >From the figures above we can see that with the entity bean method in a
> single user enviroment we suffer a 1868% peformance overhead. This is quite
> unacceptable. With these figures we see that the Inprise Application Server
> 4.0 is roughly 16 times faster than Web Logic 4.51.
Hmm. I assume you are talking about entity beans here, not SessionBean
+ sql. More on this next.
> The overhead can be attributed to the fact that at the end of the
> transaction Web Logic will call ejbStore on all entity beans accessed. This
> will result in n (size of collection) update sql calls to the database. The
> Inprise EJB Container does perform the ejbStore after which its checks to
> see has the state of the bean changed since the ejbLoad call. If nothing has
> changed, which is the case, then no update sql calls will be made.
Yes, which is the reason for the isModified pattern, which you
subsequently discovered. (More on isModified below) Meanwhile, can you
please verify whether the dbIsShared flag is set to true or false? It
should be false in your case, because you are clearly not sharing the
database with anyone. The default is "true", which forces the server
to go refresh the bean from the database every single time.
> After further investigation a properitary solution was found to reduce this
> overhead. This involved implementing an isModified method in the entity
> bean. Web Logic can then use this to determine if a bean has changed. It is
> the reponsibility of the bean developer to ensure that the value returned
> from this function is correct. This means that code must be placed in each
> set method for the bean and also in the standard ejb methods to set the flag
> appropriately. There are many issues including vendor lock in of our beans,
> production of more code and we know what that means the more chance of
> errors creeping in.
Yes, it is a pattern that we implement, and I'll grant you that it is
somewhat inconvenient. However, it does NOT mean that the bean is
locked into our solution. The weblogic container merely makes use of
such a method if one is available. It would be locked in if
if the bean is expected to import weblogic.ejb.extensions.* or something
like that. As it stands, the bean is inherently portable.
The advantage of isModified is that the check is not proportional to
data size. If you have a reasonably complex object, the server doesn't
waste any time capturing a snapshot before the transaction and
subsequently doing reflection on the bean and every nested object to
see which fields have changed: it merely relies on the programmer to
know best.
Further, given the direction the EJB2.0 spec is taking, beans will get
abstract accessor methods, something we have been lobbying for since
before the 1.0 spec was released to the public. Since the container is
expected to implement the concrete accessor methods, the isModified
pattern is a temporary measure.
> Session Bean + Container Managed Persistence Entity Bean + isModified (ms)
> .....some table....
> >From the figures above we can see that with the entity bean method + the
> isModified properitary extension in a single user enviroment we suffer a 50%
> ((1895-1278)/1278) peformance overhead in retrieving the list. This is more
> acceptable. With these figures we see that the Inprise Application Server
> 4.0 is roughly 25% times faster than Web Logic 4.51.
All I can say is, check that you have the latest service pack on 4.5
to avoid paying a penalty for assertion checks that were inadvertently
left in, and set the dbIsShared flag to false to get rid of
unnecessary reads.
> The 25% is calculated on just the retrieving list results. The Inprise
> Application server value of 1.434 secs excludes the initial connection cost
> that web logic incurrs at startup and one large value that was caused by
> myself (working on something). The weblogic value used was 1.895 secs. 25% =
> (1.895 - 1.434) / 1.895
I still don't get this. How come the IAS server excludes the connection
cost that weblogic incurs?
Could this possibly mean that you haven't set up the connection pool
to start all the required connections when the weblogic server starts
up?
> 2.3 Vendor Independent Test Results (Bean Managed Persistence)
>
> Session Bean + Bean Managed Persistence Entity Bean
> ..............some table.........
> >From the figures above we can see that with the bean managed persistence
> entity bean method in a single user enviroment we suffer a 3380%
> ((44482-1278)/1278) peformance overhead in retrieving the list. This is
> totally unacceptable and one of the reasons I am totally against bean
> managed persistence without the aid of some persistent cache framework i.e.
> TopLink.
I agree with you here. The problem is that with BMP the container
merely gets an enumeration that contains primary keys, so all bulk
reads are wasted. There are two solutions.
1) The bean or the persistence manager (such as TopLink) caches
the result set
2) The bean returns a weblogic specific enumeration
(weblogic.ejb.pstore.FindEnumeration) that represents a collection of
key-value pairs, the values being instances of the bean itself. That
way, the weblogic cache can make use of the result directly. We are
most hesitant to have the bean import any weblogic specific classes,
but in this case there were no other respectable defaults. This is
also a problem that we have told the ejb spec time a while back.
Of course, CMP implementations don't necessarily have to have this
problem. Ours doesn't.
> i. The tests above were done using a single client. I would expect the
> performance difference between IAS 4.0 and WL 4.51 to be considerably more
> since IAS is based on Visibroker 4.0 and 2.3 CORBA compliant ORB which
> avails of the new POA.
> ii. The cost of activation as session is 75% slower with Web Logic 4.51.
> This attributed to their reliance on RMI technology.
Huh? I think you are on extremely thin ice here.
First, we have our own solidly-optimized clean room implementation of
RMI.
Second, activation as a concept has existed way before CORBA, and
doesn't require a CORBA POA to support it. weblogic RMI has supported
activation for some time now. Try doing javap
weblogic.rmi.extensions.ActivationHelper in case you don't
believe me. This facility has not been for public consumption
because EJB came along with a reasonably satisfactory activation
model.
Finally, allow me to substantiate the performance of the weblogic
server. These are internal test runs from last week. The test
involves 10,000 simultaneous RMI client threads hitting a cluster
of WebLogic Servers. There is no database activity going on.
Without the Performance Pack: The cluster of WebLogic Servers was able to
average 8,307 calls per second.
If you use the native performance pack for NT or Solaris, the figure
jumps to 16,686 calls per second.
The benchmark is run on a cluster of 10 Pentium II 256MB-512MB
300-400mhz PC servers (on 10 single CPU machines running Windows NT)
20 heterogeneous client machines are used (a mix of single CPU NT
boxes & multiprocessor Unix boxes running a total of 76 VMs).
Hopefully you'll substantiate your response in a similar
way. Roundtrip testing is one thing, and we believe we are at least as
fast at it if not way faster than our competitors. But where the
server is truly optimized for is for server throughput, and I hope you
will do us all a favor by testing throughput also.
Also, it'd be nice if we can see the example you posted. Feel free to
post it here on this list.
Ciao for niao!
-Sriram
Principal Engineer, BEA/Weblogic.
"Advanced Perl Programming", O'Reilly and Associates.
===========================================================================
To unsubscribe, send email to [EMAIL PROTECTED] and include in the body
of the message "signoff EJB-INTEREST". For general help, send email to
[EMAIL PROTECTED] and include in the body of the message "help".
