Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time
Hi Johan -
I will be able to look into this error more later today but wanted to
get this observation to you ASAP since this issue seems to have set for
a week. The root cause of the failure is this exception taken from one
of the repeated stack traces from the MCWrapper class:
original exception message: Cannot call 'cleanup' on a ManagedConnection while
it is
still in a transaction..
Based on this message I recommend placing a check in your code to insure
there are no open transactions prior to calling cleanup. I can't
explain why this would be an issue when Derby is used and not DB2 but I
am hoping it is as simple as doing a check and handling all open
transactions.
Johan Hoogenboezem wrote:
Hi,
I'm using Derby 10.1.1.0 with derbyclient in a WebSphere 5.1 server
environment. Here's the first part of the console output that shows some
versioning info:
**Start
WebSphere Platform 5.1 [BASE 5.1.0.3 cf30412.02] [JDK 1.4.1 b0344.02]
running with process name localhost\localhost\server1 and process id 2304
Host Operating System is Windows XP, version 5.1
Java version = J2RE 1.4.1 IBM Windows 32 build cn1411-20031011 (JIT enabled:
jitc), Java Compiler = jitc, Java VM name = Classic VM
...
...
**End
I've set up a data source using org.apache.derby.jdbc.ClientXADataSource.
Hibernate is configured as follows:
**Start
hibernate.connection.datasource = java:comp/env/jdbc/DefaultDS
hibernate.transaction.factory_class=net.sf.hibernate.transaction.JTATransact
ionFactory
hibernate.transaction.manager_lookup_class=net.sf.hibernate.transaction.WebS
phereTransactionManagerLookup
hibernate.dialect=net.sf.hibernate.dialect.DB2Dialect
hibernate.connection.username=APP
hibernate.connection.password=APP
hibernate.show_sql=false
**End
I get the current session from a ThreadLocal variable, so that in the
context of the current thread, anything asking for a hibernate session will
get the same session. I'm using Stateless Session Beans and Message Driven
Beans. Specifically in this instance I'm using an MDB with WebSphere MQ as
the JMS implementation. The transaction is managed by the container and in a
finally clause inside the onMessage() method I call a closeCurrentSession()
utility method that just flushes and closes the hibernate session stored in
the ThreadLocal.
Everything works fine when using a datasource for DB2. However, when I
switch to Derby, I get this:
**Start
[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0304E: XAException
occurred. XAException contents and details are: The cause is :
org.apache.derby.client.am.SqlException: Error executing a
XAResource.commit(), Server returned XAER_NOTA.
[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0302E: XAException
occurred. Error code is: XAER_NOTA. Exception is: XAER_NOTA : Error
executing a XAResource.commit(), Server returned XAER_NOTA
[9/13/05 8:23:49:889 CAT] 1e59f386 XATransaction E J2CA0027E: An exception
occurred while invoking commit on an XA Resource Adapter from dataSource
jdbc/derby/scvwebdev, within transaction ID {XID: formatId(57415344),
gtrid_length(39), bqual_length(28),
data(000200044c4c40cbd49eaa1530e4b7146f080d8853876c7a7365727
66572314c4c40cbd49eaa1530e4b7146f080d8853876c7a00048333bb35)}:
org.apache.derby.client.am.XaException: XAER_NOTA : Error executing a
XAResource.commit(), Server returned XAER_NOTA
at
org.apache.derby.client.net.NetXAResource.throwXAException(Unknown Source)
at org.apache.derby.client.net.NetXAResource.commit(Unknown Source)
at
com.ibm.ws.rsadapter.spi.WSRdbXaResourceImpl.commit(WSRdbXaResourceImpl.java
:182)
at
com.ibm.ejs.j2c.XATransactionWrapper.commit(XATransactionWrapper.java:432)
at
com.ibm.ws.Transaction.JTA.JTAXAResourceImpl.commit(JTAXAResourceImpl.java:2
42)
at
com.ibm.ws.Transaction.JTA.RegisteredResources.distributeOutcome(RegisteredR
esources.java:1044)
at
com.ibm.ws.Transaction.JTA.RegisteredResources.distributeCommit(RegisteredRe
sources.java:1555)
at
com.ibm.ws.Transaction.JTA.TransactionImpl.internalCommit(TransactionImpl.ja
va:1432)
at
com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java:1012)
at
com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java:150)
at
com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java:182)
at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:712)
at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:167)
at
Re: Wish List addition
Hi Mag, This issue (bug 472) is certainly on my wishlist but since it's a modest sized feature, it won't get into the October release. We need someone to work on this feature and no-one has been donated to this effort yet. You're welcome to help out! Cheers, -Rick Mag Gam wrote: I would like to add 'full text indexing' as a wishlist item. There has been some good tutorials with Lucene, but I prefer an integrated solution. Does anyone have some schema or triggers to get this done?
Security attacks moving to the web tier
FYI. I think it underscores that those of us building web-tier apps need to be very conscious of security during design, implementation and deployment. It's no longer core IT that is threatened. http://www.geekzone.co.nz/content.asp?contentid=5222 begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: derby performance and 'order by'
As craig points out it is important in performance testing to say exactly what you are measuring. In general Derby will try to stream rows to the user before it has finished looking at all rows. So often looking at the first row will and stopping will mean that many rows have not been processed. BUT when an order by is involved and the query plan either has no appropriate matching index, or decides to use a different index then all the rows are processed, then they are sent to the sorter and finally after all rows are processed they are streamed to the client. So as you have seen reading the first 1000 rows of a much larger data set can happen very quickly. As subsequent mail threads have pointed out, returning the top 1000 sorted rows is an interesting problem which could be costed and executed differently if that information was pushed into the optimizer and the sorter (and medium level projects were done in those areas). scotto wrote: The test was set up and run using the SQuirreL client, not ij. All 3 of the queries return the top 1000 rows and the times I reported are to return these top 1000 rows, not just the first row. *From:* Craig Russell [mailto:[EMAIL PROTECTED] *Sent:* Saturday, September 17, 2005 2:35 PM *To:* Derby Discussion *Subject:* Re: derby performance and 'order by' Hi Scott, How have you set up the test? Are you using ij and displaying all of the data or using jdbc to access the data? What do you do in 0.010 seconds? Do you read all of the rows into memory, or just record the time until you get the first row? Are you measuring the time taken to return all the rows or just the first row? Another reader has already commented on the fact that the second query is doing a lot more work than the first. The second query must sort the results after filtering the data, whereas the first and third queries can simply use the indexes and filter on the fly. I'm a little suspicious of the third query returning 720,000 results in 0.010 seconds. Craig On Sep 16, 2005, at 4:42 PM, Scott Ogden wrote: I have observed some interesting query performance behavior and am hoping someone here can explain. In my scenario, it appears that an existing index is not being used for the ‘order by’ part of the operation and as a result the performance of certain queries is suffering. Can someone explain if this is supposed to be what is happening and why? Please see below for the specific queries and their performance characteristics. Here are the particulars: - create table orders( order_id varchar(50) NOT NULL CONSTRAINT ORDERS_PK PRIMARY KEY, amount numeric(31,2), time date, inv_num varchar(50), line_num varchar(50), phone varchar(50), prod_num varchar(50)); --Load a large amount of data (720,000 records) into the ‘orders’ table --Create an index on the time column as that will be used in the ‘where’ clause. create index IX_ORDERS_TIME on orders(time); --When I run a query against this table returning top 1,000 records, this query returns very quickly, consistently less than .010 seconds. select * from orders where time '10/01/2002' and time '11/30/2002' order by time; --Now run a similarly query against same table, returning the top 1,000 records. --The difference is that the results are now sorted by the primary key (‘order_id’) rather than ‘time’. --This query returns slowly, approximately 15 seconds. Why?? select * from orders where time '10/01/2002' and time '11/30/2002' order by order_id; --Now run a third query against the same ‘orders’ table, removing the where clause --This query returns quickly, around .010 seconds. select * from orders order by order_id; - Craig Russell Architect, Sun Java Enterprise System http://java.sun.com/products/jdo 408 276-5638 mailto:[EMAIL PROTECTED] P.S. A good JDO? O, Gasp!
Re: derby performance and 'order by'
Another little detail about optimization is that Statement.setMaxRows() kind of functions on the JDBC side may not be sufficientsince it iscalled after SQL statement is prepared and returned as an object (after query plan is built). Therefore, it may be necessary to have language syntax to indicate the intention to fetch first 1000 rows only, so that when the query is prepared, this intention can be taken into account. Regards, Ali Mike Matrigali [EMAIL PROTECTED] wrote: As craig points out it is important in performance testing to sayexactly what you are measuring. In general Derby will try tostream rows to the user before it has finished looking at all rows.So often looking at the first row will and stopping will mean thatmany rows have not been processed. BUT when an order by is involvedand the query plan either has no appropriate matching index, or decidesto use a different index then all the rows are processed, then they aresent to the sorter and finally after all rows are processed they arestreamed to the client.So as you have seen reading the first 1000 rows of a much larger dataset can happen very quickly.As subsequent mail threads have pointed out, returning the top 1000sorted rows is an interesting problem which could be costed and executeddifferently if that information was pushed into the optimizer and thesorter (and medium level projects were done in those areas).scotto wrote: The test was set up and run using the SQuirreL client, not ij. All 3 of the queries return the top 1000 rows and the times I reported are to return these top 1000 rows, not just the first row. *From:* Craig Russell [mailto:[EMAIL PROTECTED] *Sent:* Saturday, September 17, 2005 2:35 PM *To:* Derby Discussion *Subject:* Re: derby performance and 'order by'Hi Scott,How have you set up the test? Are you using ij and displaying all of the data or using jdbc to access the data?What do you do in 0.010 seconds? Do you read all of the rows into memory, or just record the ti me until you get the first row? Are you measuring the time taken to return all the rows or just the first row? Another reader has already commented on the fact that the second query is doing a lot more work than the first. The second query must sort the results after filtering the data, whereas the first and third queries can simply use the indexes and filter on the fly.I'm a little suspicious of the third query returning 720,000 results in 0.010 seconds.CraigOn Sep 16, 2005, at 4:42 PM, Scott Ogden wrote:I have observed some interesting query performance behavior and am hoping someone here can explain. In my scenario, it appears that an existing index is not being used for the order by part of the operation and as a result the perfo rmance of certain queries is suffering. Can someone explain if this is supposed to be what is happening and why? Please see below for the specific queries and their performance characteristics. Here are the particulars: - create table orders( order_id varchar(50) NOT NULL CONSTRAINT ORDERS_PK PRIMARY KEY, amount numeric(31,2), time date, inv_num varchar(50), line_num varchar(50), phone varchar(50), prod_num varchar(50)); --Load a large amount of data (720,000 records) into the orders table --Create an index on the time column as that will be used i n the where clause.create index IX_ORDERS_TIME on orders(time); --When I run a query against this table returning top 1,000 records, this query returns very quickly, consistently less than .010 seconds.select * from orders where time '10/01/2002' and time '11/30/2002' order by time; --Now run a similarly query against same table, returning the top 1,000 records. --The difference is that the results are now sorted by the primary key (order_id) rather than time. --This query returns slowly, approximately 15 seconds. Why?? select * from orders where time '10/01/2002' and time '11/30/2002' order by order_id; --Now run a third query against the same orders table, removing the where clause --This query returns quickly, around .010 seconds. select * from orders order by order_id; - Craig Russell Architect, Sun Java Enterprise System http://java.sun.com/products/jdo 408 276-5638 mailto:[EMAIL PROTECTED] P.S. A good JDO? O, Gasp!
Re: Class loading deadlock
Daniel John Debrunner wrote: Andreas Fredriksson wrote: On Thu, 2005-09-15 at 09:59 +0200, Knut Anders Hatlen wrote: Your patch is against Derby 10.1.1.0, but I see that the development sources have changed since that version. Some of the changes are similar to those you proposed. Could you try to run your test using the latest development sources? Thanks! Knut Anders, I can confirm that the version on the SVN trunk doesn't deadlock with the testcase I posted previously. Thanks for testing it, I couldn't reproduce it on the trunk but assumed it was due to running on a single cpu machine. ! 2cpu machine was my next step! I will merge the associated changes into 10.1 from the trunk and hopefully they will fix the problem and become part of the upcoming 10.1.2 release. The changes Knut pointed out have been merged to 10.1. If you can build the 10.1 branch and try them out that would be great, otherwise you could wait for a snapshot and try that out. Thanks, Dan.
RE: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time
Hi Stanley
Thank you for responding.
I don't know how to do what you ask, though. By the nature of the message
driven bean in which context the code runs I do not and should not know
about any open transactions. The container starts the transaction, Hibernate
senses it is running inside a container transaction (through the way it is
configured) and defers all transaction handling to the container. The only
thing I'm allowed to do is to call setRollbackOnly() in case I handle a
checked exception and decide a rollback is in order. I cannot begin, end or
commit a transaction without getting an error thrown at me by the container.
Unless I decouple my persistence code from the container, which I don't want
to do, because then I don't have distributed transactions anymore.
Everything seems ok if XA is not involved. In other words if I have a
stateless session bean with container managed transactions that do not
involved messaging or any other XA participants, then I happily do creates,
reads, updates and deletes through Derby.
I assume Derby does cleanup as a result of some event and that perhaps it
should be moved to a different event in case of an XA tranasction? Sorry if
I re-state the obvious. I'm amazed at how often, once I've fixed an error, I
notice that the exception's been telling me what was wrong all along.
Here's the onMessage() method of my MDB:
start
public void onMessage(javax.jms.Message msg) {
try {
if (msg.getJMSRedelivered()) { // needs to be looked at in more
detail.
L.debug(Message was redelivered. Not reprocessing it.);
return;
}
getServiceLocator().getUpdatesDispatcher().dispatch((ObjectMessage)msg);
}
catch (Exception ex) {
L.error(Error, ex);
L.debug(setting rollback);
context.setRollbackOnly();
// Display the error but swallow it. We don't want the message bean to
die
}
finally {
BaseDomainAccessHibernate.closeCurrentSession();
}
}
end
Inside the updates dispatcher I have code that gets the current hibernate
session from a ThreadLocal (creating it if not present) for doing database
updates.
This is what it looks like (it uses the singleton
net.sf.hibernate.SessionFactory instance and currentSession is the
ThreadLocal):
start
public static Session getCurrentSession()
throws HibernateException {
Session s = (Session)currentSession.get();
if (s == null) {
s = getSessionFactory().openSession();
currentSession.set(s);
}
return s;
}
end
And then the closing if the current session is done like this:
start
public static void closeCurrentSession() {
Session s = (Session)currentSession.get();
if (s != null) {
try {
s.flush();
s.close();
}
catch (HibernateException e) {
processStandardHibernateException(e);
}
finally {
currentSession.set(null);
}
}
}
end
I've been looking at some of the Hibernate code, and it defers to the
container for transaction handling, as it should.
Please let me know if there is anything else I can do.
This is important as I want my whole team to convert to Derby for
development.
Regards
Johan
-Original Message-
From: Stanley Bradbury [mailto:[EMAIL PROTECTED]
Sent: Tuesday, September 20, 2005 4:19 PM
To: Derby Discussion
Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
XAER_NOTA at commit time
Hi Johan -
I will be able to look into this error more later today but wanted to
get this observation to you ASAP since this issue seems to have set for
a week. The root cause of the failure is this exception taken from one
of the repeated stack traces from the MCWrapper class:
original exception message: Cannot call 'cleanup' on a ManagedConnection
while it is
still in a transaction..
Based on this message I recommend placing a check in your code to insure
there are no open transactions prior to calling cleanup. I can't
explain why this would be an issue when Derby is used and not DB2 but I
am hoping it is as simple as doing a check and handling all open
transactions.
Johan Hoogenboezem wrote:
Hi,
I'm using Derby 10.1.1.0 with derbyclient in a WebSphere 5.1 server
environment. Here's the first part of the console output that shows some
versioning info:
***
*
**Start
WebSphere Platform 5.1 [BASE 5.1.0.3 cf30412.02] [JDK 1.4.1 b0344.02]
running with process name localhost\localhost\server1 and process id 2304
Host Operating System is Windows XP, version 5.1
Java version = J2RE 1.4.1 IBM Windows 32 build cn1411-20031011 (JIT
enabled:
jitc), Java Compiler = jitc, Java VM
