Re: JTA JMS Spec question, connection leakage
Nice work! So one final grievance. This has to do with MDBs... we see this message spammed to sysout all the time when an MDB consumes a message: Sep 04, 2019 9:41:37 AM org.apache.geronimo.connector.work.WorkerContext run INFO: Removing non-required WorkContextHandler with no context: org.apache.geronimo.connector.work.TransactionContextHandler@51541c25 Sep 04, 2019 9:41:37 AM org.apache.geronimo.connector.work.WorkerContext run INFO: Removing non-required WorkContextHandler with no context: org.apache.openejb.core.security.SecurityContextHandler@5f93f87f Sep 04, 2019 9:41:37 AM org.apache.geronimo.connector.work.WorkerContext run INFO: Removing non-required WorkContextHandler with no context: org.apache.geronimo.connector.work.HintsContextHandler@5bac7755 Any clue how to squelch just these messages? We want the rest of the INFO startup/stop messages. If possible I'd like to turn this off by default. On Wed, Sep 4, 2019 at 9:36 AM Jonathan Gallimore < jonathan.gallim...@gmail.com> wrote: > Yep, should be on 7.0.x, 7.1.x and master. > > Jon > > On Wed, Sep 4, 2019 at 3:35 PM Jonathan S. Fisher > wrote: > > > Nice, thanks for cleaning that merge up. This got ported to both 7.1.x > and > > master? > > > > > > > > On Wed, Sep 4, 2019 at 8:10 AM Jonathan Gallimore < > > jonathan.gallim...@gmail.com> wrote: > > > > > Ok, think I found it. The code in > AutoConnectionTracker.handleObtained() > > > changed ever so slightly, but the effect was proxyConnection() didn't > get > > > called. I've fixed that, and the test passes. I've also run the JMS > > > Arquillian tests which work ok too. > > > > > > Cheers! > > > > > > Jon > > > > > > On Wed, Sep 4, 2019 at 12:16 PM Jonathan Gallimore < > > > jonathan.gallim...@gmail.com> wrote: > > > > > > > Thanks Jonathan! I merged this in, and ported to 7.1.x. I've run > into a > > > > test failure > > > > with > org.apache.openejb.resource.GeronimoConnectionManagerFactoryTest. > > No > > > > idea what the root cause is yet - checking it out. I haven't pushed > > this > > > to > > > > master yet (will try and resolve that test issue first). > > > > > > > > Thanks > > > > > > > > Jon > > > > > > > > On Tue, Sep 3, 2019 at 9:07 PM Jonathan S. Fisher < > exabr...@gmail.com> > > > > wrote: > > > > > > > >> Cool! And you're welcome! JMS, ActiveMQ, and XA transactions are > > pretty > > > >> key > > > >> for us; they form the building blocks for us to scale horizontally > > > (Kafka > > > >> like patterns). > > > >> > > > >> I just ran a 100,000 messages through the code in the PR without > > > problems > > > >> or memory leaks, so I'm not worried about that Arquillian test > failing > > > at > > > >> 10k messages. > > > >> > > > >> On Tue, Sep 3, 2019 at 2:37 PM Jonathan Gallimore < > > > >> jonathan.gallim...@gmail.com> wrote: > > > >> > > > >> > I knew you'd know some magic to help with this - thank you! Just > > > >> looking at > > > >> > your PR now. Will give it a test shortly, but it looks good to me. > > > >> > > > > >> > Jon > > > >> > > > > >> > On Tue, Sep 3, 2019 at 3:08 PM Jonathan S. Fisher < > > exabr...@gmail.com > > > > > > > >> > wrote: > > > >> > > > > >> > > There are actually a few log messages we regularly ignore all > the > > > time > > > >> > from > > > >> > > the transaction manager ::wince face:: I'm not sure if we should > > be > > > >> > > concerned with that one. > > > >> > > > > > >> > > On your test, first, how is the broker xml declared? Often > > something > > > >> that > > > >> > > trips our newbies up to TomEE is having a persistent broker that > > is > > > >> > storing > > > >> > > messages between TomEE runs. The tricky thing is that the broker > > > store > > > >> > does > > > >> > > always appear in /target, so it might not get cleaned up with > mvn > > > >> clean > > > >> > > install. As such, for local development we use this > > > >> > > URL: > > > >> > > > > > >> > > > > >> > > > > > > broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true. > > > >> > > Next, on JMSReceiverBean, you're missing transactionAttribute. > > > >> > Technically > > > >> > > it should work without it, but I'd add it just in case. Finally, > > I'd > > > >> add > > > >> > a > > > >> > > small thread.sleep, or check to the queue length == 0 before > doing > > > >> your > > > >> > > assert on messagecounter. It could be you're beating the > receiver > > > >> bean to > > > >> > > the finish line. The default messaging mode is auto-ack, so > > > >> technically > > > >> > the > > > >> > > message just has to be on the broker, it doesn't have to be > > > processed > > > >> > > before your sender bean will return. > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore < > > > >> > > jonathan.gallim...@gmail.com> wrote: > > > >> > > > > > >> > > > In case it means anything to anyone, the unexpected output I'm > > > >> getting > > > >> > is > > > >> > > > an abandoned connection
Re: JTA JMS Spec question, connection leakage
Yep, should be on 7.0.x, 7.1.x and master.
Jon
On Wed, Sep 4, 2019 at 3:35 PM Jonathan S. Fisher
wrote:
> Nice, thanks for cleaning that merge up. This got ported to both 7.1.x and
> master?
>
>
>
> On Wed, Sep 4, 2019 at 8:10 AM Jonathan Gallimore <
> jonathan.gallim...@gmail.com> wrote:
>
> > Ok, think I found it. The code in AutoConnectionTracker.handleObtained()
> > changed ever so slightly, but the effect was proxyConnection() didn't get
> > called. I've fixed that, and the test passes. I've also run the JMS
> > Arquillian tests which work ok too.
> >
> > Cheers!
> >
> > Jon
> >
> > On Wed, Sep 4, 2019 at 12:16 PM Jonathan Gallimore <
> > jonathan.gallim...@gmail.com> wrote:
> >
> > > Thanks Jonathan! I merged this in, and ported to 7.1.x. I've run into a
> > > test failure
> > > with org.apache.openejb.resource.GeronimoConnectionManagerFactoryTest.
> No
> > > idea what the root cause is yet - checking it out. I haven't pushed
> this
> > to
> > > master yet (will try and resolve that test issue first).
> > >
> > > Thanks
> > >
> > > Jon
> > >
> > > On Tue, Sep 3, 2019 at 9:07 PM Jonathan S. Fisher
> > > wrote:
> > >
> > >> Cool! And you're welcome! JMS, ActiveMQ, and XA transactions are
> pretty
> > >> key
> > >> for us; they form the building blocks for us to scale horizontally
> > (Kafka
> > >> like patterns).
> > >>
> > >> I just ran a 100,000 messages through the code in the PR without
> > problems
> > >> or memory leaks, so I'm not worried about that Arquillian test failing
> > at
> > >> 10k messages.
> > >>
> > >> On Tue, Sep 3, 2019 at 2:37 PM Jonathan Gallimore <
> > >> jonathan.gallim...@gmail.com> wrote:
> > >>
> > >> > I knew you'd know some magic to help with this - thank you! Just
> > >> looking at
> > >> > your PR now. Will give it a test shortly, but it looks good to me.
> > >> >
> > >> > Jon
> > >> >
> > >> > On Tue, Sep 3, 2019 at 3:08 PM Jonathan S. Fisher <
> exabr...@gmail.com
> > >
> > >> > wrote:
> > >> >
> > >> > > There are actually a few log messages we regularly ignore all the
> > time
> > >> > from
> > >> > > the transaction manager ::wince face:: I'm not sure if we should
> be
> > >> > > concerned with that one.
> > >> > >
> > >> > > On your test, first, how is the broker xml declared? Often
> something
> > >> that
> > >> > > trips our newbies up to TomEE is having a persistent broker that
> is
> > >> > storing
> > >> > > messages between TomEE runs. The tricky thing is that the broker
> > store
> > >> > does
> > >> > > always appear in /target, so it might not get cleaned up with mvn
> > >> clean
> > >> > > install. As such, for local development we use this
> > >> > > URL:
> > >> > >
> > >> >
> > >>
> >
> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
> > >> > > Next, on JMSReceiverBean, you're missing transactionAttribute.
> > >> > Technically
> > >> > > it should work without it, but I'd add it just in case. Finally,
> I'd
> > >> add
> > >> > a
> > >> > > small thread.sleep, or check to the queue length == 0 before doing
> > >> your
> > >> > > assert on messagecounter. It could be you're beating the receiver
> > >> bean to
> > >> > > the finish line. The default messaging mode is auto-ack, so
> > >> technically
> > >> > the
> > >> > > message just has to be on the broker, it doesn't have to be
> > processed
> > >> > > before your sender bean will return.
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
> > >> > > jonathan.gallim...@gmail.com> wrote:
> > >> > >
> > >> > > > In case it means anything to anyone, the unexpected output I'm
> > >> getting
> > >> > is
> > >> > > > an abandoned connection notification:
> > >> > > >
> > >> > > > WARNING: Transaction complete, but connection still has handles
> > >> > > associated:
> > >> > > > ManagedConnectionInfo:
> > >> > > >
> > >> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
> > >> > > mc:
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> > >> > > > ,ActiveMQConnection
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> > >> > > > Abandoned connection information:
> > >> > > > Connection handle opened at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
Re: JTA JMS Spec question, connection leakage
Nice, thanks for cleaning that merge up. This got ported to both 7.1.x and
master?
On Wed, Sep 4, 2019 at 8:10 AM Jonathan Gallimore <
jonathan.gallim...@gmail.com> wrote:
> Ok, think I found it. The code in AutoConnectionTracker.handleObtained()
> changed ever so slightly, but the effect was proxyConnection() didn't get
> called. I've fixed that, and the test passes. I've also run the JMS
> Arquillian tests which work ok too.
>
> Cheers!
>
> Jon
>
> On Wed, Sep 4, 2019 at 12:16 PM Jonathan Gallimore <
> jonathan.gallim...@gmail.com> wrote:
>
> > Thanks Jonathan! I merged this in, and ported to 7.1.x. I've run into a
> > test failure
> > with org.apache.openejb.resource.GeronimoConnectionManagerFactoryTest. No
> > idea what the root cause is yet - checking it out. I haven't pushed this
> to
> > master yet (will try and resolve that test issue first).
> >
> > Thanks
> >
> > Jon
> >
> > On Tue, Sep 3, 2019 at 9:07 PM Jonathan S. Fisher
> > wrote:
> >
> >> Cool! And you're welcome! JMS, ActiveMQ, and XA transactions are pretty
> >> key
> >> for us; they form the building blocks for us to scale horizontally
> (Kafka
> >> like patterns).
> >>
> >> I just ran a 100,000 messages through the code in the PR without
> problems
> >> or memory leaks, so I'm not worried about that Arquillian test failing
> at
> >> 10k messages.
> >>
> >> On Tue, Sep 3, 2019 at 2:37 PM Jonathan Gallimore <
> >> jonathan.gallim...@gmail.com> wrote:
> >>
> >> > I knew you'd know some magic to help with this - thank you! Just
> >> looking at
> >> > your PR now. Will give it a test shortly, but it looks good to me.
> >> >
> >> > Jon
> >> >
> >> > On Tue, Sep 3, 2019 at 3:08 PM Jonathan S. Fisher >
> >> > wrote:
> >> >
> >> > > There are actually a few log messages we regularly ignore all the
> time
> >> > from
> >> > > the transaction manager ::wince face:: I'm not sure if we should be
> >> > > concerned with that one.
> >> > >
> >> > > On your test, first, how is the broker xml declared? Often something
> >> that
> >> > > trips our newbies up to TomEE is having a persistent broker that is
> >> > storing
> >> > > messages between TomEE runs. The tricky thing is that the broker
> store
> >> > does
> >> > > always appear in /target, so it might not get cleaned up with mvn
> >> clean
> >> > > install. As such, for local development we use this
> >> > > URL:
> >> > >
> >> >
> >>
> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
> >> > > Next, on JMSReceiverBean, you're missing transactionAttribute.
> >> > Technically
> >> > > it should work without it, but I'd add it just in case. Finally, I'd
> >> add
> >> > a
> >> > > small thread.sleep, or check to the queue length == 0 before doing
> >> your
> >> > > assert on messagecounter. It could be you're beating the receiver
> >> bean to
> >> > > the finish line. The default messaging mode is auto-ack, so
> >> technically
> >> > the
> >> > > message just has to be on the broker, it doesn't have to be
> processed
> >> > > before your sender bean will return.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
> >> > > jonathan.gallim...@gmail.com> wrote:
> >> > >
> >> > > > In case it means anything to anyone, the unexpected output I'm
> >> getting
> >> > is
> >> > > > an abandoned connection notification:
> >> > > >
> >> > > > WARNING: Transaction complete, but connection still has handles
> >> > > associated:
> >> > > > ManagedConnectionInfo:
> >> > > >
> >> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
> >> > > mc:
> >> > > >
> >> > >
> >> >
> >>
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> >> > > > ,ActiveMQConnection
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> >> > > > Abandoned connection information:
> >> > > > Connection handle opened at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
Re: JTA JMS Spec question, connection leakage
Ok, think I found it. The code in AutoConnectionTracker.handleObtained()
changed ever so slightly, but the effect was proxyConnection() didn't get
called. I've fixed that, and the test passes. I've also run the JMS
Arquillian tests which work ok too.
Cheers!
Jon
On Wed, Sep 4, 2019 at 12:16 PM Jonathan Gallimore <
jonathan.gallim...@gmail.com> wrote:
> Thanks Jonathan! I merged this in, and ported to 7.1.x. I've run into a
> test failure
> with org.apache.openejb.resource.GeronimoConnectionManagerFactoryTest. No
> idea what the root cause is yet - checking it out. I haven't pushed this to
> master yet (will try and resolve that test issue first).
>
> Thanks
>
> Jon
>
> On Tue, Sep 3, 2019 at 9:07 PM Jonathan S. Fisher
> wrote:
>
>> Cool! And you're welcome! JMS, ActiveMQ, and XA transactions are pretty
>> key
>> for us; they form the building blocks for us to scale horizontally (Kafka
>> like patterns).
>>
>> I just ran a 100,000 messages through the code in the PR without problems
>> or memory leaks, so I'm not worried about that Arquillian test failing at
>> 10k messages.
>>
>> On Tue, Sep 3, 2019 at 2:37 PM Jonathan Gallimore <
>> jonathan.gallim...@gmail.com> wrote:
>>
>> > I knew you'd know some magic to help with this - thank you! Just
>> looking at
>> > your PR now. Will give it a test shortly, but it looks good to me.
>> >
>> > Jon
>> >
>> > On Tue, Sep 3, 2019 at 3:08 PM Jonathan S. Fisher
>> > wrote:
>> >
>> > > There are actually a few log messages we regularly ignore all the time
>> > from
>> > > the transaction manager ::wince face:: I'm not sure if we should be
>> > > concerned with that one.
>> > >
>> > > On your test, first, how is the broker xml declared? Often something
>> that
>> > > trips our newbies up to TomEE is having a persistent broker that is
>> > storing
>> > > messages between TomEE runs. The tricky thing is that the broker store
>> > does
>> > > always appear in /target, so it might not get cleaned up with mvn
>> clean
>> > > install. As such, for local development we use this
>> > > URL:
>> > >
>> >
>> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
>> > > Next, on JMSReceiverBean, you're missing transactionAttribute.
>> > Technically
>> > > it should work without it, but I'd add it just in case. Finally, I'd
>> add
>> > a
>> > > small thread.sleep, or check to the queue length == 0 before doing
>> your
>> > > assert on messagecounter. It could be you're beating the receiver
>> bean to
>> > > the finish line. The default messaging mode is auto-ack, so
>> technically
>> > the
>> > > message just has to be on the broker, it doesn't have to be processed
>> > > before your sender bean will return.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
>> > > jonathan.gallim...@gmail.com> wrote:
>> > >
>> > > > In case it means anything to anyone, the unexpected output I'm
>> getting
>> > is
>> > > > an abandoned connection notification:
>> > > >
>> > > > WARNING: Transaction complete, but connection still has handles
>> > > associated:
>> > > > ManagedConnectionInfo:
>> > > >
>> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
>> > > mc:
>> > > >
>> > >
>> >
>> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
>> > > > ,ActiveMQConnection
>> > > >
>> > > >
>> > >
>> >
>> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
>> > > > Abandoned connection information:
>> > > > Connection handle opened at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
>> > > >
>> > > >
>> > >
>> >
>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
>> > > >
>> > > >
>> > >
>> >
>>
Re: JTA JMS Spec question, connection leakage
Thanks Jonathan! I merged this in, and ported to 7.1.x. I've run into a
test failure
with org.apache.openejb.resource.GeronimoConnectionManagerFactoryTest. No
idea what the root cause is yet - checking it out. I haven't pushed this to
master yet (will try and resolve that test issue first).
Thanks
Jon
On Tue, Sep 3, 2019 at 9:07 PM Jonathan S. Fisher
wrote:
> Cool! And you're welcome! JMS, ActiveMQ, and XA transactions are pretty key
> for us; they form the building blocks for us to scale horizontally (Kafka
> like patterns).
>
> I just ran a 100,000 messages through the code in the PR without problems
> or memory leaks, so I'm not worried about that Arquillian test failing at
> 10k messages.
>
> On Tue, Sep 3, 2019 at 2:37 PM Jonathan Gallimore <
> jonathan.gallim...@gmail.com> wrote:
>
> > I knew you'd know some magic to help with this - thank you! Just looking
> at
> > your PR now. Will give it a test shortly, but it looks good to me.
> >
> > Jon
> >
> > On Tue, Sep 3, 2019 at 3:08 PM Jonathan S. Fisher
> > wrote:
> >
> > > There are actually a few log messages we regularly ignore all the time
> > from
> > > the transaction manager ::wince face:: I'm not sure if we should be
> > > concerned with that one.
> > >
> > > On your test, first, how is the broker xml declared? Often something
> that
> > > trips our newbies up to TomEE is having a persistent broker that is
> > storing
> > > messages between TomEE runs. The tricky thing is that the broker store
> > does
> > > always appear in /target, so it might not get cleaned up with mvn clean
> > > install. As such, for local development we use this
> > > URL:
> > >
> >
> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
> > > Next, on JMSReceiverBean, you're missing transactionAttribute.
> > Technically
> > > it should work without it, but I'd add it just in case. Finally, I'd
> add
> > a
> > > small thread.sleep, or check to the queue length == 0 before doing your
> > > assert on messagecounter. It could be you're beating the receiver bean
> to
> > > the finish line. The default messaging mode is auto-ack, so technically
> > the
> > > message just has to be on the broker, it doesn't have to be processed
> > > before your sender bean will return.
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
> > > jonathan.gallim...@gmail.com> wrote:
> > >
> > > > In case it means anything to anyone, the unexpected output I'm
> getting
> > is
> > > > an abandoned connection notification:
> > > >
> > > > WARNING: Transaction complete, but connection still has handles
> > > associated:
> > > > ManagedConnectionInfo:
> > > >
> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
> > > mc:
> > > >
> > >
> >
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> > > > ,ActiveMQConnection
> > > >
> > > >
> > >
> >
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> > > > Abandoned connection information:
> > > > Connection handle opened at
> > > >
> > > >
> > >
> >
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
> > > >
> > > >
> > >
> >
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
> > > >
> > > >
> > >
> >
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
> > > >
> > > >
> > >
> >
> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
> > > >
> > > >
> > >
> >
> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
> > > >
> > > >
> > >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
> > > >
> > > >
> > >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
> > > >
> > > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
> > > >
> > > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
> > > >
> > > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
> > > >
> > > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
> > > >
> > > >
> > >
> >
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
> > > >
> > > >
> > >
> >
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
> > > > sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
> > > >
> > > >
> > >
> >
>
Re: JTA JMS Spec question, connection leakage
Thanks for the explanation; I think your change should be good for everyone, not just amq. Iirc it used to be common (very bad) practice to get a connection handle and cache it in a non Singleton session bean and expect the connection infrastructure to associate it to whatever managed connection was associated to the current transaction. If that worked this certainly ought to! Thanks David Jencks Sent from my iPhone > On Sep 3, 2019, at 6:43 PM, Jonathan S. Fisher wrote: > > Ahhh... The CDI spec for JMSContext says it's either scoped to either the > current RequestScope, or in our case TransactionScope. I'll take an > educated guess and bet TransactionScoped beans are destroyed using a > [Transaction] Synchronization's afterCompletion() call. When the > TransactionScope is destroyed is when the JMSContext is destroyed, and it's > at that point it closes the connection. I'm guessing AutoConnectionTracker > was probably written before this was possible, so users were calling > connection.close() ergo handleClosed() before beforeCompletion(). Now the > CDI Extension for JMS2.0 in TomEE is calling close() much later. > > My PR moves what was in beforeCompletion() into afterCompletion(). For > ActiveMQ I'm fairly certain this won't be an issue, as they allow you to > really abuse when you open connections and even how man sessions per > connection you open :) > > On Tue, Sep 3, 2019 at 7:50 PM David Jencks > wrote: > >> Well, beforeCompletion() is called as a result of commit() being called on >> the transaction, presumably by an EJB “interceptor”, and handleClosed() is >> called as a result of the “user level” connection being closed. I’m used to >> the latter being called by user code... perhaps with all the CDI and >> dependency injection I haven’t kept up with this is no longer the case? >> >> Thanks. >> David Jencks >> Sent from my iPhone >> >>> On Sep 3, 2019, at 3:25 PM, Jonathan S. Fisher >> wrote: >>> >>> Honestly I have no idea. And the interface specification is silent >>> unfortunately: >>> >> https://github.com/apache/geronimo-txmanager/blob/trunk/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTracker.java >>> >>> On Tue, Sep 3, 2019 at 3:32 PM David Jencks >>> wrote: >>> You might have already explained this, but… why is beforeCompletion() called before handleReleased()? If that’s happening, I’d expect >> something is wrong. However, I haven’t looked at this code in years. thanks! David Jencks > On Sep 3, 2019, at 12:27 PM, Jonathan S. Fisher wrote: > > Two more updates: > > For the log message, it looks like beforeCompletion() Is being called > before handleReleased(), leading to that warning. I ran a couple >> thousand > messages through and took a heap dump and didn't get any leak suspects, so > I think it's working correctly despite the warning. I'll add that to my > existing PR with your tests. > > Second, on my 10K messages test, it looks like it might be Arquillian > crashing, not TomEE. I'm going to try the same test but without Arquillian, > just deploy those classes to a server and see what happens. > > On Tue, Sep 3, 2019 at 10:54 AM Jonathan S. Fisher >> > wrote: > >> If I bump the number of messages up to 10k or so I get a VM Crash... I >> cannot figure out how to get arquillian to give me a heap dump on exit >> though. >> >> On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher < >> exabr...@gmail.com> >> wrote: >> >>> https://github.com/apache/tomee/pull/546/files >>> >>> This passes consistently for me with no issues >>> >>> On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher < >> exabr...@gmail.com> >>> wrote: >>> There are actually a few log messages we regularly ignore all the >> time from the transaction manager ::wince face:: I'm not sure if we >> should be concerned with that one. On your test, first, how is the broker xml declared? Often something that trips our newbies up to TomEE is having a persistent broker >> that is storing messages between TomEE runs. The tricky thing is that the broker store does always appear in /target, so it might not get cleaned up with mvn clean install. As such, for local development we use this URL: >> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true. Next, on JMSReceiverBean, you're missing transactionAttribute. Technically it should work without it, but I'd add it just in case. Finally, I'd add a small thread.sleep, or check to the queue length == 0 before doing your assert on messagecounter. It could be you're beating the receiver bean to the finish line. The default messaging
Re: JTA JMS Spec question, connection leakage
Ahhh... The CDI spec for JMSContext says it's either scoped to either the current RequestScope, or in our case TransactionScope. I'll take an educated guess and bet TransactionScoped beans are destroyed using a [Transaction] Synchronization's afterCompletion() call. When the TransactionScope is destroyed is when the JMSContext is destroyed, and it's at that point it closes the connection. I'm guessing AutoConnectionTracker was probably written before this was possible, so users were calling connection.close() ergo handleClosed() before beforeCompletion(). Now the CDI Extension for JMS2.0 in TomEE is calling close() much later. My PR moves what was in beforeCompletion() into afterCompletion(). For ActiveMQ I'm fairly certain this won't be an issue, as they allow you to really abuse when you open connections and even how man sessions per connection you open :) On Tue, Sep 3, 2019 at 7:50 PM David Jencks wrote: > Well, beforeCompletion() is called as a result of commit() being called on > the transaction, presumably by an EJB “interceptor”, and handleClosed() is > called as a result of the “user level” connection being closed. I’m used to > the latter being called by user code... perhaps with all the CDI and > dependency injection I haven’t kept up with this is no longer the case? > > Thanks. > David Jencks > Sent from my iPhone > > > On Sep 3, 2019, at 3:25 PM, Jonathan S. Fisher > wrote: > > > > Honestly I have no idea. And the interface specification is silent > > unfortunately: > > > https://github.com/apache/geronimo-txmanager/blob/trunk/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTracker.java > > > > On Tue, Sep 3, 2019 at 3:32 PM David Jencks > > wrote: > > > >> You might have already explained this, but… why is beforeCompletion() > >> called before handleReleased()? If that’s happening, I’d expect > something > >> is wrong. However, I haven’t looked at this code in years. > >> > >> thanks! > >> David Jencks > >> > >>> On Sep 3, 2019, at 12:27 PM, Jonathan S. Fisher > >> wrote: > >>> > >>> Two more updates: > >>> > >>> For the log message, it looks like beforeCompletion() Is being called > >>> before handleReleased(), leading to that warning. I ran a couple > thousand > >>> messages through and took a heap dump and didn't get any leak suspects, > >> so > >>> I think it's working correctly despite the warning. I'll add that to my > >>> existing PR with your tests. > >>> > >>> Second, on my 10K messages test, it looks like it might be Arquillian > >>> crashing, not TomEE. I'm going to try the same test but without > >> Arquillian, > >>> just deploy those classes to a server and see what happens. > >>> > >>> On Tue, Sep 3, 2019 at 10:54 AM Jonathan S. Fisher > > >>> wrote: > >>> > If I bump the number of messages up to 10k or so I get a VM Crash... I > cannot figure out how to get arquillian to give me a heap dump on exit > though. > > On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher < > exabr...@gmail.com> > wrote: > > > https://github.com/apache/tomee/pull/546/files > > > > This passes consistently for me with no issues > > > > On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher < > exabr...@gmail.com> > > wrote: > > > >> There are actually a few log messages we regularly ignore all the > time > >> from the transaction manager ::wince face:: I'm not sure if we > should > >> be > >> concerned with that one. > >> > >> On your test, first, how is the broker xml declared? Often something > >> that trips our newbies up to TomEE is having a persistent broker > that > >> is > >> storing messages between TomEE runs. The tricky thing is that the > >> broker > >> store does always appear in /target, so it might not get cleaned up > >> with > >> mvn clean install. As such, for local development we use this > >> URL: > >> > broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true. > >> Next, on JMSReceiverBean, you're missing transactionAttribute. > >> Technically > >> it should work without it, but I'd add it just in case. Finally, I'd > >> add a > >> small thread.sleep, or check to the queue length == 0 before doing > >> your > >> assert on messagecounter. It could be you're beating the receiver > >> bean to > >> the finish line. The default messaging mode is auto-ack, so > >> technically the > >> message just has to be on the broker, it doesn't have to be > processed > >> before your sender bean will return. > >> > >> > >> > >> > >> > >> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore < > >> jonathan.gallim...@gmail.com> wrote: > >> > >>> In case it means anything to anyone, the unexpected output I'm > >> getting > >>> is > >>> an abandoned connection notification: > >>> > >>> WARNING: Transaction complete, but connection still has handles >
Re: JTA JMS Spec question, connection leakage
Well, beforeCompletion() is called as a result of commit() being called on the
transaction, presumably by an EJB “interceptor”, and handleClosed() is called
as a result of the “user level” connection being closed. I’m used to the latter
being called by user code... perhaps with all the CDI and dependency injection
I haven’t kept up with this is no longer the case?
Thanks.
David Jencks
Sent from my iPhone
> On Sep 3, 2019, at 3:25 PM, Jonathan S. Fisher wrote:
>
> Honestly I have no idea. And the interface specification is silent
> unfortunately:
> https://github.com/apache/geronimo-txmanager/blob/trunk/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTracker.java
>
> On Tue, Sep 3, 2019 at 3:32 PM David Jencks
> wrote:
>
>> You might have already explained this, but… why is beforeCompletion()
>> called before handleReleased()? If that’s happening, I’d expect something
>> is wrong. However, I haven’t looked at this code in years.
>>
>> thanks!
>> David Jencks
>>
>>> On Sep 3, 2019, at 12:27 PM, Jonathan S. Fisher
>> wrote:
>>>
>>> Two more updates:
>>>
>>> For the log message, it looks like beforeCompletion() Is being called
>>> before handleReleased(), leading to that warning. I ran a couple thousand
>>> messages through and took a heap dump and didn't get any leak suspects,
>> so
>>> I think it's working correctly despite the warning. I'll add that to my
>>> existing PR with your tests.
>>>
>>> Second, on my 10K messages test, it looks like it might be Arquillian
>>> crashing, not TomEE. I'm going to try the same test but without
>> Arquillian,
>>> just deploy those classes to a server and see what happens.
>>>
>>> On Tue, Sep 3, 2019 at 10:54 AM Jonathan S. Fisher
>>> wrote:
>>>
If I bump the number of messages up to 10k or so I get a VM Crash... I
cannot figure out how to get arquillian to give me a heap dump on exit
though.
On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher
wrote:
> https://github.com/apache/tomee/pull/546/files
>
> This passes consistently for me with no issues
>
> On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher
> wrote:
>
>> There are actually a few log messages we regularly ignore all the time
>> from the transaction manager ::wince face:: I'm not sure if we should
>> be
>> concerned with that one.
>>
>> On your test, first, how is the broker xml declared? Often something
>> that trips our newbies up to TomEE is having a persistent broker that
>> is
>> storing messages between TomEE runs. The tricky thing is that the
>> broker
>> store does always appear in /target, so it might not get cleaned up
>> with
>> mvn clean install. As such, for local development we use this
>> URL:
>> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
>> Next, on JMSReceiverBean, you're missing transactionAttribute.
>> Technically
>> it should work without it, but I'd add it just in case. Finally, I'd
>> add a
>> small thread.sleep, or check to the queue length == 0 before doing
>> your
>> assert on messagecounter. It could be you're beating the receiver
>> bean to
>> the finish line. The default messaging mode is auto-ack, so
>> technically the
>> message just has to be on the broker, it doesn't have to be processed
>> before your sender bean will return.
>>
>>
>>
>>
>>
>> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
>> jonathan.gallim...@gmail.com> wrote:
>>
>>> In case it means anything to anyone, the unexpected output I'm
>> getting
>>> is
>>> an abandoned connection notification:
>>>
>>> WARNING: Transaction complete, but connection still has handles
>>> associated:
>>> ManagedConnectionInfo:
>>>
>> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
>>> mc:
>>>
>>>
>> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
>>> ,ActiveMQConnection
>>>
>>>
>> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
>>> Abandoned connection information:
>>> Connection handle opened at
>>>
>>>
>> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
>>>
>>>
>> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
>>>
>>>
>> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
>>>
>>>
>> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
>>>
>>>
>> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
Re: JTA JMS Spec question, connection leakage
Honestly I have no idea. And the interface specification is silent
unfortunately:
https://github.com/apache/geronimo-txmanager/blob/trunk/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTracker.java
On Tue, Sep 3, 2019 at 3:32 PM David Jencks
wrote:
> You might have already explained this, but… why is beforeCompletion()
> called before handleReleased()? If that’s happening, I’d expect something
> is wrong. However, I haven’t looked at this code in years.
>
> thanks!
> David Jencks
>
> > On Sep 3, 2019, at 12:27 PM, Jonathan S. Fisher
> wrote:
> >
> > Two more updates:
> >
> > For the log message, it looks like beforeCompletion() Is being called
> > before handleReleased(), leading to that warning. I ran a couple thousand
> > messages through and took a heap dump and didn't get any leak suspects,
> so
> > I think it's working correctly despite the warning. I'll add that to my
> > existing PR with your tests.
> >
> > Second, on my 10K messages test, it looks like it might be Arquillian
> > crashing, not TomEE. I'm going to try the same test but without
> Arquillian,
> > just deploy those classes to a server and see what happens.
> >
> > On Tue, Sep 3, 2019 at 10:54 AM Jonathan S. Fisher
> > wrote:
> >
> >> If I bump the number of messages up to 10k or so I get a VM Crash... I
> >> cannot figure out how to get arquillian to give me a heap dump on exit
> >> though.
> >>
> >> On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher
> >> wrote:
> >>
> >>> https://github.com/apache/tomee/pull/546/files
> >>>
> >>> This passes consistently for me with no issues
> >>>
> >>> On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher
> >>> wrote:
> >>>
> There are actually a few log messages we regularly ignore all the time
> from the transaction manager ::wince face:: I'm not sure if we should
> be
> concerned with that one.
>
> On your test, first, how is the broker xml declared? Often something
> that trips our newbies up to TomEE is having a persistent broker that
> is
> storing messages between TomEE runs. The tricky thing is that the
> broker
> store does always appear in /target, so it might not get cleaned up
> with
> mvn clean install. As such, for local development we use this
> URL:
> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
> Next, on JMSReceiverBean, you're missing transactionAttribute.
> Technically
> it should work without it, but I'd add it just in case. Finally, I'd
> add a
> small thread.sleep, or check to the queue length == 0 before doing
> your
> assert on messagecounter. It could be you're beating the receiver
> bean to
> the finish line. The default messaging mode is auto-ack, so
> technically the
> message just has to be on the broker, it doesn't have to be processed
> before your sender bean will return.
>
>
>
>
>
> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
> jonathan.gallim...@gmail.com> wrote:
>
> > In case it means anything to anyone, the unexpected output I'm
> getting
> > is
> > an abandoned connection notification:
> >
> > WARNING: Transaction complete, but connection still has handles
> > associated:
> > ManagedConnectionInfo:
> >
> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
> > mc:
> >
> >
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> > ,ActiveMQConnection
> >
> >
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> > Abandoned connection information:
> > Connection handle opened at
> >
> >
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
> >
> >
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
> >
> >
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
> >
> >
> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
> >
> >
> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
> >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
> >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
> >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
> >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
> >
> >
>
Re: JTA JMS Spec question, connection leakage
You might have already explained this, but… why is beforeCompletion() called
before handleReleased()? If that’s happening, I’d expect something is wrong.
However, I haven’t looked at this code in years.
thanks!
David Jencks
> On Sep 3, 2019, at 12:27 PM, Jonathan S. Fisher wrote:
>
> Two more updates:
>
> For the log message, it looks like beforeCompletion() Is being called
> before handleReleased(), leading to that warning. I ran a couple thousand
> messages through and took a heap dump and didn't get any leak suspects, so
> I think it's working correctly despite the warning. I'll add that to my
> existing PR with your tests.
>
> Second, on my 10K messages test, it looks like it might be Arquillian
> crashing, not TomEE. I'm going to try the same test but without Arquillian,
> just deploy those classes to a server and see what happens.
>
> On Tue, Sep 3, 2019 at 10:54 AM Jonathan S. Fisher
> wrote:
>
>> If I bump the number of messages up to 10k or so I get a VM Crash... I
>> cannot figure out how to get arquillian to give me a heap dump on exit
>> though.
>>
>> On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher
>> wrote:
>>
>>> https://github.com/apache/tomee/pull/546/files
>>>
>>> This passes consistently for me with no issues
>>>
>>> On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher
>>> wrote:
>>>
There are actually a few log messages we regularly ignore all the time
from the transaction manager ::wince face:: I'm not sure if we should be
concerned with that one.
On your test, first, how is the broker xml declared? Often something
that trips our newbies up to TomEE is having a persistent broker that is
storing messages between TomEE runs. The tricky thing is that the broker
store does always appear in /target, so it might not get cleaned up with
mvn clean install. As such, for local development we use this
URL:
broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
Next, on JMSReceiverBean, you're missing transactionAttribute. Technically
it should work without it, but I'd add it just in case. Finally, I'd add a
small thread.sleep, or check to the queue length == 0 before doing your
assert on messagecounter. It could be you're beating the receiver bean to
the finish line. The default messaging mode is auto-ack, so technically the
message just has to be on the broker, it doesn't have to be processed
before your sender bean will return.
On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
jonathan.gallim...@gmail.com> wrote:
> In case it means anything to anyone, the unexpected output I'm getting
> is
> an abandoned connection notification:
>
> WARNING: Transaction complete, but connection still has handles
> associated:
> ManagedConnectionInfo:
> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
> mc:
>
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> ,ActiveMQConnection
>
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> Abandoned connection information:
> Connection handle opened at
>
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
>
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
>
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
>
> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
>
> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
>
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
>
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
>
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
>
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
>
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
>
> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
>
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
>
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
> sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
>
>
Re: JTA JMS Spec question, connection leakage
Cool! And you're welcome! JMS, ActiveMQ, and XA transactions are pretty key
for us; they form the building blocks for us to scale horizontally (Kafka
like patterns).
I just ran a 100,000 messages through the code in the PR without problems
or memory leaks, so I'm not worried about that Arquillian test failing at
10k messages.
On Tue, Sep 3, 2019 at 2:37 PM Jonathan Gallimore <
jonathan.gallim...@gmail.com> wrote:
> I knew you'd know some magic to help with this - thank you! Just looking at
> your PR now. Will give it a test shortly, but it looks good to me.
>
> Jon
>
> On Tue, Sep 3, 2019 at 3:08 PM Jonathan S. Fisher
> wrote:
>
> > There are actually a few log messages we regularly ignore all the time
> from
> > the transaction manager ::wince face:: I'm not sure if we should be
> > concerned with that one.
> >
> > On your test, first, how is the broker xml declared? Often something that
> > trips our newbies up to TomEE is having a persistent broker that is
> storing
> > messages between TomEE runs. The tricky thing is that the broker store
> does
> > always appear in /target, so it might not get cleaned up with mvn clean
> > install. As such, for local development we use this
> > URL:
> >
> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
> > Next, on JMSReceiverBean, you're missing transactionAttribute.
> Technically
> > it should work without it, but I'd add it just in case. Finally, I'd add
> a
> > small thread.sleep, or check to the queue length == 0 before doing your
> > assert on messagecounter. It could be you're beating the receiver bean to
> > the finish line. The default messaging mode is auto-ack, so technically
> the
> > message just has to be on the broker, it doesn't have to be processed
> > before your sender bean will return.
> >
> >
> >
> >
> >
> > On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
> > jonathan.gallim...@gmail.com> wrote:
> >
> > > In case it means anything to anyone, the unexpected output I'm getting
> is
> > > an abandoned connection notification:
> > >
> > > WARNING: Transaction complete, but connection still has handles
> > associated:
> > > ManagedConnectionInfo:
> > > org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
> > mc:
> > >
> >
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> > > ,ActiveMQConnection
> > >
> > >
> >
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> > > Abandoned connection information:
> > > Connection handle opened at
> > >
> > >
> >
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
> > >
> > >
> >
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
> > >
> > >
> >
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
> > >
> > >
> >
> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
> > >
> > >
> >
> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
> > >
> > >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
> > >
> > >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
> > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
> > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
> > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
> > >
> > >
> >
> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
> > >
> > >
> >
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
> > >
> > >
> >
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
> > > sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
> > > java.lang.reflect.Method.invoke(Method.java:498),
> > >
> > >
> >
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
> > >
> > >
> >
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
> > >
> > >
> >
> org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191),
> > >
> > >
> >
> org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102),
> > > sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source),
> > >
> > >
> >
>
Re: JTA JMS Spec question, connection leakage
I knew you'd know some magic to help with this - thank you! Just looking at
your PR now. Will give it a test shortly, but it looks good to me.
Jon
On Tue, Sep 3, 2019 at 3:08 PM Jonathan S. Fisher
wrote:
> There are actually a few log messages we regularly ignore all the time from
> the transaction manager ::wince face:: I'm not sure if we should be
> concerned with that one.
>
> On your test, first, how is the broker xml declared? Often something that
> trips our newbies up to TomEE is having a persistent broker that is storing
> messages between TomEE runs. The tricky thing is that the broker store does
> always appear in /target, so it might not get cleaned up with mvn clean
> install. As such, for local development we use this
> URL:
> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
> Next, on JMSReceiverBean, you're missing transactionAttribute. Technically
> it should work without it, but I'd add it just in case. Finally, I'd add a
> small thread.sleep, or check to the queue length == 0 before doing your
> assert on messagecounter. It could be you're beating the receiver bean to
> the finish line. The default messaging mode is auto-ack, so technically the
> message just has to be on the broker, it doesn't have to be processed
> before your sender bean will return.
>
>
>
>
>
> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
> jonathan.gallim...@gmail.com> wrote:
>
> > In case it means anything to anyone, the unexpected output I'm getting is
> > an abandoned connection notification:
> >
> > WARNING: Transaction complete, but connection still has handles
> associated:
> > ManagedConnectionInfo:
> > org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
> mc:
> >
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> > ,ActiveMQConnection
> >
> >
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> > Abandoned connection information:
> > Connection handle opened at
> >
> >
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
> >
> >
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
> >
> >
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
> >
> >
> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
> >
> >
> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
> >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
> >
> >
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
> >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
> >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
> >
> >
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
> >
> >
> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
> >
> >
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
> >
> >
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
> > sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
> > java.lang.reflect.Method.invoke(Method.java:498),
> >
> >
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
> >
> >
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
> >
> >
> org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191),
> >
> >
> org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102),
> > sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source),
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
> > java.lang.reflect.Method.invoke(Method.java:498),
> >
> >
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
> >
> >
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
> >
> >
> org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:85),
> >
> >
> org.apache.openejb.core.singleton.SingletonContainer._invoke(SingletonContainer.java:272),
> >
> >
> org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:221),
> >
> >
>
Re: JTA JMS Spec question, connection leakage
Two more updates:
For the log message, it looks like beforeCompletion() Is being called
before handleReleased(), leading to that warning. I ran a couple thousand
messages through and took a heap dump and didn't get any leak suspects, so
I think it's working correctly despite the warning. I'll add that to my
existing PR with your tests.
Second, on my 10K messages test, it looks like it might be Arquillian
crashing, not TomEE. I'm going to try the same test but without Arquillian,
just deploy those classes to a server and see what happens.
On Tue, Sep 3, 2019 at 10:54 AM Jonathan S. Fisher
wrote:
> If I bump the number of messages up to 10k or so I get a VM Crash... I
> cannot figure out how to get arquillian to give me a heap dump on exit
> though.
>
> On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher
> wrote:
>
>> https://github.com/apache/tomee/pull/546/files
>>
>> This passes consistently for me with no issues
>>
>> On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher
>> wrote:
>>
>>> There are actually a few log messages we regularly ignore all the time
>>> from the transaction manager ::wince face:: I'm not sure if we should be
>>> concerned with that one.
>>>
>>> On your test, first, how is the broker xml declared? Often something
>>> that trips our newbies up to TomEE is having a persistent broker that is
>>> storing messages between TomEE runs. The tricky thing is that the broker
>>> store does always appear in /target, so it might not get cleaned up with
>>> mvn clean install. As such, for local development we use this
>>> URL:
>>> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
>>> Next, on JMSReceiverBean, you're missing transactionAttribute. Technically
>>> it should work without it, but I'd add it just in case. Finally, I'd add a
>>> small thread.sleep, or check to the queue length == 0 before doing your
>>> assert on messagecounter. It could be you're beating the receiver bean to
>>> the finish line. The default messaging mode is auto-ack, so technically the
>>> message just has to be on the broker, it doesn't have to be processed
>>> before your sender bean will return.
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
>>> jonathan.gallim...@gmail.com> wrote:
>>>
In case it means anything to anyone, the unexpected output I'm getting
is
an abandoned connection notification:
WARNING: Transaction complete, but connection still has handles
associated:
ManagedConnectionInfo:
org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
mc:
[org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
,ActiveMQConnection
{id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
Abandoned connection information:
Connection handle opened at
org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
java.lang.reflect.Method.invoke(Method.java:498),
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191),
Re: JTA JMS Spec question, connection leakage
If I bump the number of messages up to 10k or so I get a VM Crash... I
cannot figure out how to get arquillian to give me a heap dump on exit
though.
On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher
wrote:
> https://github.com/apache/tomee/pull/546/files
>
> This passes consistently for me with no issues
>
> On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher
> wrote:
>
>> There are actually a few log messages we regularly ignore all the time
>> from the transaction manager ::wince face:: I'm not sure if we should be
>> concerned with that one.
>>
>> On your test, first, how is the broker xml declared? Often something that
>> trips our newbies up to TomEE is having a persistent broker that is storing
>> messages between TomEE runs. The tricky thing is that the broker store does
>> always appear in /target, so it might not get cleaned up with mvn clean
>> install. As such, for local development we use this
>> URL:
>> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
>> Next, on JMSReceiverBean, you're missing transactionAttribute. Technically
>> it should work without it, but I'd add it just in case. Finally, I'd add a
>> small thread.sleep, or check to the queue length == 0 before doing your
>> assert on messagecounter. It could be you're beating the receiver bean to
>> the finish line. The default messaging mode is auto-ack, so technically the
>> message just has to be on the broker, it doesn't have to be processed
>> before your sender bean will return.
>>
>>
>>
>>
>>
>> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
>> jonathan.gallim...@gmail.com> wrote:
>>
>>> In case it means anything to anyone, the unexpected output I'm getting is
>>> an abandoned connection notification:
>>>
>>> WARNING: Transaction complete, but connection still has handles
>>> associated:
>>> ManagedConnectionInfo:
>>> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
>>> mc:
>>>
>>> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
>>> ,ActiveMQConnection
>>>
>>> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
>>> Abandoned connection information:
>>> Connection handle opened at
>>>
>>> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
>>>
>>> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
>>>
>>> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
>>>
>>> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
>>>
>>> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
>>>
>>> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
>>>
>>> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
>>>
>>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
>>>
>>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
>>>
>>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
>>>
>>> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
>>>
>>> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
>>>
>>> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
>>> sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
>>> java.lang.reflect.Method.invoke(Method.java:498),
>>>
>>> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
>>>
>>> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
>>>
>>> org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191),
>>>
>>> org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102),
>>> sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source),
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
>>> java.lang.reflect.Method.invoke(Method.java:498),
>>>
>>> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
>>>
>>> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
>>>
>>> org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:85),
>>>
>>>
Re: JTA JMS Spec question, connection leakage
https://github.com/apache/tomee/pull/546/files
This passes consistently for me with no issues
On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher
wrote:
> There are actually a few log messages we regularly ignore all the time
> from the transaction manager ::wince face:: I'm not sure if we should be
> concerned with that one.
>
> On your test, first, how is the broker xml declared? Often something that
> trips our newbies up to TomEE is having a persistent broker that is storing
> messages between TomEE runs. The tricky thing is that the broker store does
> always appear in /target, so it might not get cleaned up with mvn clean
> install. As such, for local development we use this
> URL:
> broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
> Next, on JMSReceiverBean, you're missing transactionAttribute. Technically
> it should work without it, but I'd add it just in case. Finally, I'd add a
> small thread.sleep, or check to the queue length == 0 before doing your
> assert on messagecounter. It could be you're beating the receiver bean to
> the finish line. The default messaging mode is auto-ack, so technically the
> message just has to be on the broker, it doesn't have to be processed
> before your sender bean will return.
>
>
>
>
>
> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
> jonathan.gallim...@gmail.com> wrote:
>
>> In case it means anything to anyone, the unexpected output I'm getting is
>> an abandoned connection notification:
>>
>> WARNING: Transaction complete, but connection still has handles
>> associated:
>> ManagedConnectionInfo:
>> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed.
>> mc:
>> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
>> ,ActiveMQConnection
>>
>> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
>> Abandoned connection information:
>> Connection handle opened at
>>
>> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
>>
>> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
>>
>> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
>>
>> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
>>
>> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
>>
>> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
>>
>> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
>>
>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
>>
>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
>>
>> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
>>
>> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
>>
>> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
>>
>> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
>> sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
>> java.lang.reflect.Method.invoke(Method.java:498),
>>
>> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
>>
>> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
>>
>> org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191),
>>
>> org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102),
>> sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source),
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
>> java.lang.reflect.Method.invoke(Method.java:498),
>>
>> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
>>
>> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
>>
>> org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:85),
>>
>> org.apache.openejb.core.singleton.SingletonContainer._invoke(SingletonContainer.java:272),
>>
>> org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:221),
>>
>> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265),
>>
>>
Re: JTA JMS Spec question, connection leakage
There are actually a few log messages we regularly ignore all the time from
the transaction manager ::wince face:: I'm not sure if we should be
concerned with that one.
On your test, first, how is the broker xml declared? Often something that
trips our newbies up to TomEE is having a persistent broker that is storing
messages between TomEE runs. The tricky thing is that the broker store does
always appear in /target, so it might not get cleaned up with mvn clean
install. As such, for local development we use this
URL:
broker:(vm://localhost)?persistent=falsedeleteAllMessagesOnStartup=true.
Next, on JMSReceiverBean, you're missing transactionAttribute. Technically
it should work without it, but I'd add it just in case. Finally, I'd add a
small thread.sleep, or check to the queue length == 0 before doing your
assert on messagecounter. It could be you're beating the receiver bean to
the finish line. The default messaging mode is auto-ack, so technically the
message just has to be on the broker, it doesn't have to be processed
before your sender bean will return.
On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore <
jonathan.gallim...@gmail.com> wrote:
> In case it means anything to anyone, the unexpected output I'm getting is
> an abandoned connection notification:
>
> WARNING: Transaction complete, but connection still has handles associated:
> ManagedConnectionInfo:
> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed. mc:
> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43
> ,ActiveMQConnection
>
> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
> Abandoned connection information:
> Connection handle opened at
>
> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
>
> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
>
> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
>
> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
>
> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
>
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
>
> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
>
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
>
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
>
> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
>
> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
>
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
>
> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
> sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
> java.lang.reflect.Method.invoke(Method.java:498),
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
>
> org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191),
>
> org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102),
> sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source),
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
> java.lang.reflect.Method.invoke(Method.java:498),
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
>
> org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:85),
>
> org.apache.openejb.core.singleton.SingletonContainer._invoke(SingletonContainer.java:272),
>
> org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:221),
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265),
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260),
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89),
>
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:347),
>
>
Re: JTA JMS Spec question, connection leakage
In case it means anything to anyone, the unexpected output I'm getting is
an abandoned connection notification:
WARNING: Transaction complete, but connection still has handles associated:
ManagedConnectionInfo:
org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed. mc:
[org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43,ActiveMQConnection
{id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]]
Abandoned connection information:
Connection handle opened at
org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119),
org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57),
org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39),
org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66),
org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81),
org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94),
org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67),
org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83),
org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99),
org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309),
org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371),
org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37),
org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33),
sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source),
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
java.lang.reflect.Method.invoke(Method.java:498),
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191),
org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102),
sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source),
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
java.lang.reflect.Method.invoke(Method.java:498),
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205),
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186),
org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:85),
org.apache.openejb.core.singleton.SingletonContainer._invoke(SingletonContainer.java:272),
org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:221),
org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265),
org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260),
org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89),
org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:347),
org.apache.openejb.arquillian.tests.jms.JMSSenderBean$$LocalBeanProxy.sendToQueue(org/apache/openejb/arquillian/tests/jms/JMSSenderBean.java),
org.apache.openejb.arquillian.tests.jms.JMSContextInjectionTest.testShouldSendAndReceiveMessages(JMSContextInjectionTest.java:61),
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method),
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62),
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
java.lang.reflect.Method.invoke(Method.java:498),
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50),
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12),
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47),
org.jboss.arquillian.junit.Arquillian$8$1.invoke(Arquillian.java:379),
org.jboss.arquillian.container.test.impl.execution.LocalTestExecuter.execute(LocalTestExecuter.java:60),
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method),
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62),
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
java.lang.reflect.Method.invoke(Method.java:498),
org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96),
Re: JTA JMS Spec question, connection leakage
I had a play around with this, and got my previously failing JMSContextInjectionTest to work -- some of the time. ( https://github.com/apache/tomee/blob/e58ff848a3a80938d4d99fc9bcfeaade5a72d644/arquillian/arquillian-tomee-tests/arquillian-tomee-jms-tests/src/test/java/org/apache/openejb/arquillian/tests/jms/JMSContextInjectionTest.java ). I'm getting some log output that I'm not expecting so I've rolled back my removal of @Ignore so I can dig into it some more. Jon On Tue, Aug 27, 2019 at 5:36 PM Jonathan Gallimore < jonathan.gallim...@gmail.com> wrote: > Hi Jonathan > > Thanks for this. I think I had run into some of the things you fixed I > little while back. I'll take a look at your change later today. Happy for > you to commit and I can review after, too. :) > > Cheers > > The other Jonathan > > > On Tue, Aug 27, 2019 at 2:59 AM Jonathan S. Fisher > wrote: > >> >> https://github.com/exabrial/tomee/commit/08dd4c744818702f3be5edfd8a1c4cf2b69d524d >> >> With these patches the JMS2.0 API works pretty well now :) If anyone wants >> to review, please go ahead, otherwise I'll commit them tomorrow. >> >> cheers, >> - [The other] Jonathan >> >> On Mon, Aug 26, 2019 at 3:45 PM Jonathan S. Fisher >> wrote: >> >> > I've narrowed down the problem to AutoConnectionTracker. It's not >> > completing, which isn't allowing the connections to be returned to the >> pool. >> > >> > >> https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 >> > >> > getResource() is throwing an IllegalStateException. The JavaDoc ( >> > >> https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object- >> ) >> > states it should throw an ISE if a current transaction is not Active. >> The >> > transaction is in the state ROLLED_BACK when AutoConnectionTracker >> tries to >> > call getResource(). >> > >> > I think the Geronimo implementation ( >> > >> https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203 >> ) >> > maybe be a little too strict. The JTA Spec pdf doesn't offer exact >> hints of >> > which statuses ( >> > https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) >> > should be have getResource _not_ throw an ISE unfortunately. I was >> thinking >> > of changing Geronimo's implementation to check for anything >> > but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. >> > >> > The other way is to cast Transaction to the Geronimo implementation and >> > use Geronimo specific APIs to get call getResource(). Do you guys have >> any >> > preference which route I should take to fix? >> > >> > >> > On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher >> > wrote: >> > >> >> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak >> >> >> >> Here's a project that reproduces the bug. This project intentionally >> >> exceeds the transaction timeout (of 1s). Each invocation, the >> connection is >> >> not returned to the pool and eventually you run out, causing your >> >> application to freeze. >> >> >> >> >> >> >> >> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher > > >> >> wrote: >> >> >> >>> Hello Apache friends :) I have a question about the JTA and JMS/RA >> specs: >> >>> >> >>> If you borrow something from a RA, like a JMS Connection, and you're >> in >> >>> XA Transaction, is it necessary to call connection.close()? It would >> seem >> >>> JTA should be smart enough to know the connection is enrolled for 2 >> phase >> >>> commit and should be smart enough to close it, but I'm not sure if >> that's >> >>> part of the spec. >> >>> >> >>> In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with >> >>> connectionFactory.createConnection(), and your code fails to call >> close() >> >>> before the transaction completion, the connection leaks. (And >> >>> unfortunately, calling close() after the transaction completes doesn't >> >>> mitigate the problem). It took awhile for us to track this down. >> >>> >> >>> This becomes a huge problem if you're calling external services in >> your >> >>> transaction. Let's say you have a reasonable transaction timeout of >> 30s >> >>> set. You call three services, and they end up taking 15s a piece. >> Even if >> >>> you're doing the right thing and you have connection.close() in a >> finally >> >>> block, because your transaction isn't active when you call close, it >> leaks >> >>> and it just gets "stuck" as an active connection, which eventually >> you hit >> >>> the pool limit and your app freezes. >> >>> >> >>> On a separate note, we noticed if you open a connection outside of the >> >>> scope of a transaction, then start a transaction, then create a >> session >> >>> with session_transacted option, the session does not participate in >> the JTA >> >>> (which seems out of spec). One most open the connection
Re: JTA JMS Spec question, connection leakage
Green build here: https://ci.apache.org/builders/tomee-7.0.x-ubuntu/builds/52 Not sure why this got is all pissy: https://ci.apache.org/builders/tomee-7.0.x-ubuntu-jvm8 Seems to be a different test failure each time. Builds fine on OSX/AdoptOpenJdk8 locally. Any hints? On Wed, Aug 28, 2019 at 10:08 AM Jonathan S. Fisher wrote: > Alrighty. I've got "transactionSupport" ready. Apparently that hasn't > worked for some time. > > The nice thing too is that gives the user an "out" if they want to revert > to the non-spec behavior: Right now, connection factories are non-xa, by > the spec says they should be by default. If someone upgrades and desires > the old non-xa behavior, they can set transactionSupport=none on the > connection factory. > > I'm fairly confident in this merge request, the only part I really don't > know about is the enlistResource() when I enlist XASession. It works > though, and from the examples I've seen it seems to be the correct way to > enlist a resource. I've just never messed with that API before. > > Anyway, eyes appreciated, after I update my PRs on github I'll give it a > day for review and then merge it in. > > Thanks! > > > > > On Tue, Aug 27, 2019 at 9:27 PM Jonathan S. Fisher > wrote: > >> Just noticed in JmsConnectionFactoryBuilder this is present already with >> the attribute "transactionSupport". I need to tie that into my patch before >> it's merged >> >> On Tue, Aug 27, 2019 at 4:30 PM Jonathan S. Fisher >> wrote: >> >>> I just checked Wildfly, they do the same thing as Liberty. I agree with >>> your statement for the "completely correct" fix, ideally that's the place >>> to do it, but might take awhile to get a release out. >>> >>> On another note: I know the spec says, "Ignore all arguments to >>> connection.create*(int mode)" methods. Yet I can think of a lot of >>> scenarios where having a non-JTA connection pool is very handy (for >>> instance, logging over JMS). We have the option to have non-JTA Database >>> connections, I feel though we should be able to declare whether or not a >>> jms connection pool participates in JTA. >>> >>> I'm thinking maybe we should have an `xa=true/false` parameter in the >>> connection pool declaration. Would that be ok? >>> >>> >>> On Tue, Aug 27, 2019 at 3:43 PM David Jencks >>> wrote: >>> I checked the Open Liberty TransactionSynchronizationRegistry, and it interprets “active transaction” to mean “any transaction on the thread, no matter it’s state”. So I think that it would be best to decide to do the same in the Geronimo TM, deciding that the java doc is ambiguous as to the meaning of “active” and the most useful meaning can be picked rather than the most literal. Whether this is practical for the next TomEE, I don’t know. David Jencks > On Aug 27, 2019, at 8:25 AM, David Jencks wrote: > > I think the java doc for getResource might have been written thoughtlessly, and more appropriate behavior would be an ISE only for STATUS_NO_TRANSACTION; literally the geronimo implementation is too lax, as “marked rollback” is not status “active”. Is there anyone who’s opinion we might ask? > > I rather thought the “ignore session type” logic was supposed to be put into the RA, but I don’t recall if or how I dealt with this in Geronimo. > > So I’d prefer these issues be dealt with elsewhere but don’t see much practical alternative to your implementation. > > Nice to see someone working on XA:-) > > thanks! > David Jencks > >> On Aug 26, 2019, at 1:45 PM, Jonathan S. Fisher wrote: >> >> I've narrowed down the problem to AutoConnectionTracker. It's not >> completing, which isn't allowing the connections to be returned to the pool. >> https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 >> >> getResource() is throwing an IllegalStateException. The JavaDoc ( >> https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object- ) >> states it should throw an ISE if a current transaction is not Active. The >> transaction is in the state ROLLED_BACK when AutoConnectionTracker tries to >> call getResource(). >> >> I think the Geronimo implementation ( >> https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203 ) >> maybe be a little too strict. The JTA Spec pdf doesn't offer exact hints of >> which statuses ( >> https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) should >> be have getResource _not_ throw an ISE unfortunately. I was thinking of >> changing
Re: JTA JMS Spec question, connection leakage
Alrighty. I've got "transactionSupport" ready. Apparently that hasn't worked for some time. The nice thing too is that gives the user an "out" if they want to revert to the non-spec behavior: Right now, connection factories are non-xa, by the spec says they should be by default. If someone upgrades and desires the old non-xa behavior, they can set transactionSupport=none on the connection factory. I'm fairly confident in this merge request, the only part I really don't know about is the enlistResource() when I enlist XASession. It works though, and from the examples I've seen it seems to be the correct way to enlist a resource. I've just never messed with that API before. Anyway, eyes appreciated, after I update my PRs on github I'll give it a day for review and then merge it in. Thanks! On Tue, Aug 27, 2019 at 9:27 PM Jonathan S. Fisher wrote: > Just noticed in JmsConnectionFactoryBuilder this is present already with > the attribute "transactionSupport". I need to tie that into my patch before > it's merged > > On Tue, Aug 27, 2019 at 4:30 PM Jonathan S. Fisher > wrote: > >> I just checked Wildfly, they do the same thing as Liberty. I agree with >> your statement for the "completely correct" fix, ideally that's the place >> to do it, but might take awhile to get a release out. >> >> On another note: I know the spec says, "Ignore all arguments to >> connection.create*(int mode)" methods. Yet I can think of a lot of >> scenarios where having a non-JTA connection pool is very handy (for >> instance, logging over JMS). We have the option to have non-JTA Database >> connections, I feel though we should be able to declare whether or not a >> jms connection pool participates in JTA. >> >> I'm thinking maybe we should have an `xa=true/false` parameter in the >> connection pool declaration. Would that be ok? >> >> >> On Tue, Aug 27, 2019 at 3:43 PM David Jencks >> wrote: >> >>> I checked the Open Liberty TransactionSynchronizationRegistry, and it >>> interprets “active transaction” to mean “any transaction on the thread, no >>> matter it’s state”. So I think that it would be best to decide to do the >>> same in the Geronimo TM, deciding that the java doc is ambiguous as to the >>> meaning of “active” and the most useful meaning can be picked rather than >>> the most literal. >>> >>> Whether this is practical for the next TomEE, I don’t know. >>> >>> David Jencks >>> >>> > On Aug 27, 2019, at 8:25 AM, David Jencks >>> wrote: >>> > >>> > I think the java doc for getResource might have been written >>> thoughtlessly, and more appropriate behavior would be an ISE only for >>> STATUS_NO_TRANSACTION; literally the geronimo implementation is too lax, as >>> “marked rollback” is not status “active”. Is there anyone who’s opinion we >>> might ask? >>> > >>> > I rather thought the “ignore session type” logic was supposed to be >>> put into the RA, but I don’t recall if or how I dealt with this in Geronimo. >>> > >>> > So I’d prefer these issues be dealt with elsewhere but don’t see much >>> practical alternative to your implementation. >>> > >>> > Nice to see someone working on XA:-) >>> > >>> > thanks! >>> > David Jencks >>> > >>> >> On Aug 26, 2019, at 1:45 PM, Jonathan S. Fisher >>> wrote: >>> >> >>> >> I've narrowed down the problem to AutoConnectionTracker. It's not >>> >> completing, which isn't allowing the connections to be returned to >>> the pool. >>> >> >>> https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 >>> >> >>> >> getResource() is throwing an IllegalStateException. The JavaDoc ( >>> >> >>> https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object- >>> ) >>> >> states it should throw an ISE if a current transaction is not Active. >>> The >>> >> transaction is in the state ROLLED_BACK when AutoConnectionTracker >>> tries to >>> >> call getResource(). >>> >> >>> >> I think the Geronimo implementation ( >>> >> >>> https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203 >>> ) >>> >> maybe be a little too strict. The JTA Spec pdf doesn't offer exact >>> hints of >>> >> which statuses ( >>> >> https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) >>> should >>> >> be have getResource _not_ throw an ISE unfortunately. I was thinking >>> of >>> >> changing Geronimo's implementation to check for anything >>> >> but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. >>> >> >>> >> The other way is to cast Transaction to the Geronimo implementation >>> and use >>> >> Geronimo specific APIs to get call getResource(). Do you guys have any >>> >> preference which route I should take to fix? >>> >> >>> >> >>> >> On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher < >>> exabr...@gmail.com> >>> >> wrote: >>> >> >>> >>>
Re: JTA JMS Spec question, connection leakage
Just noticed in JmsConnectionFactoryBuilder this is present already with the attribute "transactionSupport". I need to tie that into my patch before it's merged On Tue, Aug 27, 2019 at 4:30 PM Jonathan S. Fisher wrote: > I just checked Wildfly, they do the same thing as Liberty. I agree with > your statement for the "completely correct" fix, ideally that's the place > to do it, but might take awhile to get a release out. > > On another note: I know the spec says, "Ignore all arguments to > connection.create*(int mode)" methods. Yet I can think of a lot of > scenarios where having a non-JTA connection pool is very handy (for > instance, logging over JMS). We have the option to have non-JTA Database > connections, I feel though we should be able to declare whether or not a > jms connection pool participates in JTA. > > I'm thinking maybe we should have an `xa=true/false` parameter in the > connection pool declaration. Would that be ok? > > > On Tue, Aug 27, 2019 at 3:43 PM David Jencks > wrote: > >> I checked the Open Liberty TransactionSynchronizationRegistry, and it >> interprets “active transaction” to mean “any transaction on the thread, no >> matter it’s state”. So I think that it would be best to decide to do the >> same in the Geronimo TM, deciding that the java doc is ambiguous as to the >> meaning of “active” and the most useful meaning can be picked rather than >> the most literal. >> >> Whether this is practical for the next TomEE, I don’t know. >> >> David Jencks >> >> > On Aug 27, 2019, at 8:25 AM, David Jencks >> wrote: >> > >> > I think the java doc for getResource might have been written >> thoughtlessly, and more appropriate behavior would be an ISE only for >> STATUS_NO_TRANSACTION; literally the geronimo implementation is too lax, as >> “marked rollback” is not status “active”. Is there anyone who’s opinion we >> might ask? >> > >> > I rather thought the “ignore session type” logic was supposed to be put >> into the RA, but I don’t recall if or how I dealt with this in Geronimo. >> > >> > So I’d prefer these issues be dealt with elsewhere but don’t see much >> practical alternative to your implementation. >> > >> > Nice to see someone working on XA:-) >> > >> > thanks! >> > David Jencks >> > >> >> On Aug 26, 2019, at 1:45 PM, Jonathan S. Fisher >> wrote: >> >> >> >> I've narrowed down the problem to AutoConnectionTracker. It's not >> >> completing, which isn't allowing the connections to be returned to the >> pool. >> >> >> https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 >> >> >> >> getResource() is throwing an IllegalStateException. The JavaDoc ( >> >> >> https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object- >> ) >> >> states it should throw an ISE if a current transaction is not Active. >> The >> >> transaction is in the state ROLLED_BACK when AutoConnectionTracker >> tries to >> >> call getResource(). >> >> >> >> I think the Geronimo implementation ( >> >> >> https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203 >> ) >> >> maybe be a little too strict. The JTA Spec pdf doesn't offer exact >> hints of >> >> which statuses ( >> >> https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) >> should >> >> be have getResource _not_ throw an ISE unfortunately. I was thinking of >> >> changing Geronimo's implementation to check for anything >> >> but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. >> >> >> >> The other way is to cast Transaction to the Geronimo implementation >> and use >> >> Geronimo specific APIs to get call getResource(). Do you guys have any >> >> preference which route I should take to fix? >> >> >> >> >> >> On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher > > >> >> wrote: >> >> >> >>> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak >> >>> >> >>> Here's a project that reproduces the bug. This project intentionally >> >>> exceeds the transaction timeout (of 1s). Each invocation, the >> connection is >> >>> not returned to the pool and eventually you run out, causing your >> >>> application to freeze. >> >>> >> >>> >> >>> >> >>> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher < >> exabr...@gmail.com> >> >>> wrote: >> >>> >> Hello Apache friends :) I have a question about the JTA and JMS/RA >> specs: >> >> If you borrow something from a RA, like a JMS Connection, and you're >> in >> XA Transaction, is it necessary to call connection.close()? It would >> seem >> JTA should be smart enough to know the connection is enrolled for 2 >> phase >> commit and should be smart enough to close it, but I'm not sure if >> that's >> part of the spec. >> >> In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection >> with >>
Re: JTA JMS Spec question, connection leakage
I just checked Wildfly, they do the same thing as Liberty. I agree with your statement for the "completely correct" fix, ideally that's the place to do it, but might take awhile to get a release out. On another note: I know the spec says, "Ignore all arguments to connection.create*(int mode)" methods. Yet I can think of a lot of scenarios where having a non-JTA connection pool is very handy (for instance, logging over JMS). We have the option to have non-JTA Database connections, I feel though we should be able to declare whether or not a jms connection pool participates in JTA. I'm thinking maybe we should have an `xa=true/false` parameter in the connection pool declaration. Would that be ok? On Tue, Aug 27, 2019 at 3:43 PM David Jencks wrote: > I checked the Open Liberty TransactionSynchronizationRegistry, and it > interprets “active transaction” to mean “any transaction on the thread, no > matter it’s state”. So I think that it would be best to decide to do the > same in the Geronimo TM, deciding that the java doc is ambiguous as to the > meaning of “active” and the most useful meaning can be picked rather than > the most literal. > > Whether this is practical for the next TomEE, I don’t know. > > David Jencks > > > On Aug 27, 2019, at 8:25 AM, David Jencks > wrote: > > > > I think the java doc for getResource might have been written > thoughtlessly, and more appropriate behavior would be an ISE only for > STATUS_NO_TRANSACTION; literally the geronimo implementation is too lax, as > “marked rollback” is not status “active”. Is there anyone who’s opinion we > might ask? > > > > I rather thought the “ignore session type” logic was supposed to be put > into the RA, but I don’t recall if or how I dealt with this in Geronimo. > > > > So I’d prefer these issues be dealt with elsewhere but don’t see much > practical alternative to your implementation. > > > > Nice to see someone working on XA:-) > > > > thanks! > > David Jencks > > > >> On Aug 26, 2019, at 1:45 PM, Jonathan S. Fisher > wrote: > >> > >> I've narrowed down the problem to AutoConnectionTracker. It's not > >> completing, which isn't allowing the connections to be returned to the > pool. > >> > https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 > >> > >> getResource() is throwing an IllegalStateException. The JavaDoc ( > >> > https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object- > ) > >> states it should throw an ISE if a current transaction is not Active. > The > >> transaction is in the state ROLLED_BACK when AutoConnectionTracker > tries to > >> call getResource(). > >> > >> I think the Geronimo implementation ( > >> > https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203 > ) > >> maybe be a little too strict. The JTA Spec pdf doesn't offer exact > hints of > >> which statuses ( > >> https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) > should > >> be have getResource _not_ throw an ISE unfortunately. I was thinking of > >> changing Geronimo's implementation to check for anything > >> but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. > >> > >> The other way is to cast Transaction to the Geronimo implementation and > use > >> Geronimo specific APIs to get call getResource(). Do you guys have any > >> preference which route I should take to fix? > >> > >> > >> On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher > >> wrote: > >> > >>> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak > >>> > >>> Here's a project that reproduces the bug. This project intentionally > >>> exceeds the transaction timeout (of 1s). Each invocation, the > connection is > >>> not returned to the pool and eventually you run out, causing your > >>> application to freeze. > >>> > >>> > >>> > >>> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher > > >>> wrote: > >>> > Hello Apache friends :) I have a question about the JTA and JMS/RA > specs: > > If you borrow something from a RA, like a JMS Connection, and you're > in > XA Transaction, is it necessary to call connection.close()? It would > seem > JTA should be smart enough to know the connection is enrolled for 2 > phase > commit and should be smart enough to close it, but I'm not sure if > that's > part of the spec. > > In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with > connectionFactory.createConnection(), and your code fails to call > close() > before the transaction completion, the connection leaks. (And > unfortunately, calling close() after the transaction completes doesn't > mitigate the problem). It took awhile for us to track this down. > > This becomes a huge problem if you're calling external services in >
Re: JTA JMS Spec question, connection leakage
I checked the Open Liberty TransactionSynchronizationRegistry, and it interprets “active transaction” to mean “any transaction on the thread, no matter it’s state”. So I think that it would be best to decide to do the same in the Geronimo TM, deciding that the java doc is ambiguous as to the meaning of “active” and the most useful meaning can be picked rather than the most literal. Whether this is practical for the next TomEE, I don’t know. David Jencks > On Aug 27, 2019, at 8:25 AM, David Jencks wrote: > > I think the java doc for getResource might have been written thoughtlessly, > and more appropriate behavior would be an ISE only for STATUS_NO_TRANSACTION; > literally the geronimo implementation is too lax, as “marked rollback” is not > status “active”. Is there anyone who’s opinion we might ask? > > I rather thought the “ignore session type” logic was supposed to be put into > the RA, but I don’t recall if or how I dealt with this in Geronimo. > > So I’d prefer these issues be dealt with elsewhere but don’t see much > practical alternative to your implementation. > > Nice to see someone working on XA:-) > > thanks! > David Jencks > >> On Aug 26, 2019, at 1:45 PM, Jonathan S. Fisher wrote: >> >> I've narrowed down the problem to AutoConnectionTracker. It's not >> completing, which isn't allowing the connections to be returned to the pool. >> https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 >> >> getResource() is throwing an IllegalStateException. The JavaDoc ( >> https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object-) >> states it should throw an ISE if a current transaction is not Active. The >> transaction is in the state ROLLED_BACK when AutoConnectionTracker tries to >> call getResource(). >> >> I think the Geronimo implementation ( >> https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203) >> maybe be a little too strict. The JTA Spec pdf doesn't offer exact hints of >> which statuses ( >> https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) should >> be have getResource _not_ throw an ISE unfortunately. I was thinking of >> changing Geronimo's implementation to check for anything >> but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. >> >> The other way is to cast Transaction to the Geronimo implementation and use >> Geronimo specific APIs to get call getResource(). Do you guys have any >> preference which route I should take to fix? >> >> >> On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher >> wrote: >> >>> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak >>> >>> Here's a project that reproduces the bug. This project intentionally >>> exceeds the transaction timeout (of 1s). Each invocation, the connection is >>> not returned to the pool and eventually you run out, causing your >>> application to freeze. >>> >>> >>> >>> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher >>> wrote: >>> Hello Apache friends :) I have a question about the JTA and JMS/RA specs: If you borrow something from a RA, like a JMS Connection, and you're in XA Transaction, is it necessary to call connection.close()? It would seem JTA should be smart enough to know the connection is enrolled for 2 phase commit and should be smart enough to close it, but I'm not sure if that's part of the spec. In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with connectionFactory.createConnection(), and your code fails to call close() before the transaction completion, the connection leaks. (And unfortunately, calling close() after the transaction completes doesn't mitigate the problem). It took awhile for us to track this down. This becomes a huge problem if you're calling external services in your transaction. Let's say you have a reasonable transaction timeout of 30s set. You call three services, and they end up taking 15s a piece. Even if you're doing the right thing and you have connection.close() in a finally block, because your transaction isn't active when you call close, it leaks and it just gets "stuck" as an active connection, which eventually you hit the pool limit and your app freezes. On a separate note, we noticed if you open a connection outside of the scope of a transaction, then start a transaction, then create a session with session_transacted option, the session does not participate in the JTA (which seems out of spec). One most open the connection inside the transaction, AND open the session in the transaction, and close the connection in the transaction for everything to work. I'll get a reproducing project created,
Re: JTA JMS Spec question, connection leakage
Hi Jonathan Thanks for this. I think I had run into some of the things you fixed I little while back. I'll take a look at your change later today. Happy for you to commit and I can review after, too. :) Cheers The other Jonathan On Tue, Aug 27, 2019 at 2:59 AM Jonathan S. Fisher wrote: > > https://github.com/exabrial/tomee/commit/08dd4c744818702f3be5edfd8a1c4cf2b69d524d > > With these patches the JMS2.0 API works pretty well now :) If anyone wants > to review, please go ahead, otherwise I'll commit them tomorrow. > > cheers, > - [The other] Jonathan > > On Mon, Aug 26, 2019 at 3:45 PM Jonathan S. Fisher > wrote: > > > I've narrowed down the problem to AutoConnectionTracker. It's not > > completing, which isn't allowing the connections to be returned to the > pool. > > > > > https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 > > > > getResource() is throwing an IllegalStateException. The JavaDoc ( > > > https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object- > ) > > states it should throw an ISE if a current transaction is not Active. The > > transaction is in the state ROLLED_BACK when AutoConnectionTracker tries > to > > call getResource(). > > > > I think the Geronimo implementation ( > > > https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203 > ) > > maybe be a little too strict. The JTA Spec pdf doesn't offer exact hints > of > > which statuses ( > > https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) > > should be have getResource _not_ throw an ISE unfortunately. I was > thinking > > of changing Geronimo's implementation to check for anything > > but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. > > > > The other way is to cast Transaction to the Geronimo implementation and > > use Geronimo specific APIs to get call getResource(). Do you guys have > any > > preference which route I should take to fix? > > > > > > On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher > > wrote: > > > >> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak > >> > >> Here's a project that reproduces the bug. This project intentionally > >> exceeds the transaction timeout (of 1s). Each invocation, the > connection is > >> not returned to the pool and eventually you run out, causing your > >> application to freeze. > >> > >> > >> > >> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher > >> wrote: > >> > >>> Hello Apache friends :) I have a question about the JTA and JMS/RA > specs: > >>> > >>> If you borrow something from a RA, like a JMS Connection, and you're in > >>> XA Transaction, is it necessary to call connection.close()? It would > seem > >>> JTA should be smart enough to know the connection is enrolled for 2 > phase > >>> commit and should be smart enough to close it, but I'm not sure if > that's > >>> part of the spec. > >>> > >>> In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with > >>> connectionFactory.createConnection(), and your code fails to call > close() > >>> before the transaction completion, the connection leaks. (And > >>> unfortunately, calling close() after the transaction completes doesn't > >>> mitigate the problem). It took awhile for us to track this down. > >>> > >>> This becomes a huge problem if you're calling external services in your > >>> transaction. Let's say you have a reasonable transaction timeout of 30s > >>> set. You call three services, and they end up taking 15s a piece. Even > if > >>> you're doing the right thing and you have connection.close() in a > finally > >>> block, because your transaction isn't active when you call close, it > leaks > >>> and it just gets "stuck" as an active connection, which eventually you > hit > >>> the pool limit and your app freezes. > >>> > >>> On a separate note, we noticed if you open a connection outside of the > >>> scope of a transaction, then start a transaction, then create a session > >>> with session_transacted option, the session does not participate in > the JTA > >>> (which seems out of spec). One most open the connection inside the > >>> transaction, AND open the session in the transaction, and close the > >>> connection in the transaction for everything to work. > >>> > >>> I'll get a reproducing project created, but I was curious if anyone > knew > >>> offhand what the spec says. > >>> > >>> cheers, and thanks, > >>> -[the other] Jonathan > >>> > >>> -- > >>> Jonathan | exabr...@gmail.com > >>> Pessimists, see a jar as half empty. Optimists, in contrast, see it as > >>> half full. > >>> Engineers, of course, understand the glass is twice as big as it needs > >>> to be. > >>> > >> > >> > >> -- > >> Jonathan | exabr...@gmail.com > >> Pessimists, see a jar as half empty. Optimists, in contrast, see it as > >> half
Re: JTA JMS Spec question, connection leakage
I think the java doc for getResource might have been written thoughtlessly, and more appropriate behavior would be an ISE only for STATUS_NO_TRANSACTION; literally the geronimo implementation is too lax, as “marked rollback” is not status “active”. Is there anyone who’s opinion we might ask? I rather thought the “ignore session type” logic was supposed to be put into the RA, but I don’t recall if or how I dealt with this in Geronimo. So I’d prefer these issues be dealt with elsewhere but don’t see much practical alternative to your implementation. Nice to see someone working on XA:-) thanks! David Jencks > On Aug 26, 2019, at 1:45 PM, Jonathan S. Fisher wrote: > > I've narrowed down the problem to AutoConnectionTracker. It's not > completing, which isn't allowing the connections to be returned to the pool. > https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 > > getResource() is throwing an IllegalStateException. The JavaDoc ( > https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object-) > states it should throw an ISE if a current transaction is not Active. The > transaction is in the state ROLLED_BACK when AutoConnectionTracker tries to > call getResource(). > > I think the Geronimo implementation ( > https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203) > maybe be a little too strict. The JTA Spec pdf doesn't offer exact hints of > which statuses ( > https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) should > be have getResource _not_ throw an ISE unfortunately. I was thinking of > changing Geronimo's implementation to check for anything > but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. > > The other way is to cast Transaction to the Geronimo implementation and use > Geronimo specific APIs to get call getResource(). Do you guys have any > preference which route I should take to fix? > > > On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher > wrote: > >> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak >> >> Here's a project that reproduces the bug. This project intentionally >> exceeds the transaction timeout (of 1s). Each invocation, the connection is >> not returned to the pool and eventually you run out, causing your >> application to freeze. >> >> >> >> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher >> wrote: >> >>> Hello Apache friends :) I have a question about the JTA and JMS/RA specs: >>> >>> If you borrow something from a RA, like a JMS Connection, and you're in >>> XA Transaction, is it necessary to call connection.close()? It would seem >>> JTA should be smart enough to know the connection is enrolled for 2 phase >>> commit and should be smart enough to close it, but I'm not sure if that's >>> part of the spec. >>> >>> In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with >>> connectionFactory.createConnection(), and your code fails to call close() >>> before the transaction completion, the connection leaks. (And >>> unfortunately, calling close() after the transaction completes doesn't >>> mitigate the problem). It took awhile for us to track this down. >>> >>> This becomes a huge problem if you're calling external services in your >>> transaction. Let's say you have a reasonable transaction timeout of 30s >>> set. You call three services, and they end up taking 15s a piece. Even if >>> you're doing the right thing and you have connection.close() in a finally >>> block, because your transaction isn't active when you call close, it leaks >>> and it just gets "stuck" as an active connection, which eventually you hit >>> the pool limit and your app freezes. >>> >>> On a separate note, we noticed if you open a connection outside of the >>> scope of a transaction, then start a transaction, then create a session >>> with session_transacted option, the session does not participate in the JTA >>> (which seems out of spec). One most open the connection inside the >>> transaction, AND open the session in the transaction, and close the >>> connection in the transaction for everything to work. >>> >>> I'll get a reproducing project created, but I was curious if anyone knew >>> offhand what the spec says. >>> >>> cheers, and thanks, >>> -[the other] Jonathan >>> >>> -- >>> Jonathan | exabr...@gmail.com >>> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >>> half full. >>> Engineers, of course, understand the glass is twice as big as it needs to >>> be. >>> >> >> >> -- >> Jonathan | exabr...@gmail.com >> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >> half full. >> Engineers, of course, understand the glass is twice as big as it needs to >> be. >> > > > -- > Jonathan | exabr...@gmail.com >
Re: JTA JMS Spec question, connection leakage
https://github.com/exabrial/tomee/commit/08dd4c744818702f3be5edfd8a1c4cf2b69d524d With these patches the JMS2.0 API works pretty well now :) If anyone wants to review, please go ahead, otherwise I'll commit them tomorrow. cheers, - [The other] Jonathan On Mon, Aug 26, 2019 at 3:45 PM Jonathan S. Fisher wrote: > I've narrowed down the problem to AutoConnectionTracker. It's not > completing, which isn't allowing the connections to be returned to the pool. > > https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 > > getResource() is throwing an IllegalStateException. The JavaDoc ( > https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object-) > states it should throw an ISE if a current transaction is not Active. The > transaction is in the state ROLLED_BACK when AutoConnectionTracker tries to > call getResource(). > > I think the Geronimo implementation ( > https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203) > maybe be a little too strict. The JTA Spec pdf doesn't offer exact hints of > which statuses ( > https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) > should be have getResource _not_ throw an ISE unfortunately. I was thinking > of changing Geronimo's implementation to check for anything > but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. > > The other way is to cast Transaction to the Geronimo implementation and > use Geronimo specific APIs to get call getResource(). Do you guys have any > preference which route I should take to fix? > > > On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher > wrote: > >> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak >> >> Here's a project that reproduces the bug. This project intentionally >> exceeds the transaction timeout (of 1s). Each invocation, the connection is >> not returned to the pool and eventually you run out, causing your >> application to freeze. >> >> >> >> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher >> wrote: >> >>> Hello Apache friends :) I have a question about the JTA and JMS/RA specs: >>> >>> If you borrow something from a RA, like a JMS Connection, and you're in >>> XA Transaction, is it necessary to call connection.close()? It would seem >>> JTA should be smart enough to know the connection is enrolled for 2 phase >>> commit and should be smart enough to close it, but I'm not sure if that's >>> part of the spec. >>> >>> In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with >>> connectionFactory.createConnection(), and your code fails to call close() >>> before the transaction completion, the connection leaks. (And >>> unfortunately, calling close() after the transaction completes doesn't >>> mitigate the problem). It took awhile for us to track this down. >>> >>> This becomes a huge problem if you're calling external services in your >>> transaction. Let's say you have a reasonable transaction timeout of 30s >>> set. You call three services, and they end up taking 15s a piece. Even if >>> you're doing the right thing and you have connection.close() in a finally >>> block, because your transaction isn't active when you call close, it leaks >>> and it just gets "stuck" as an active connection, which eventually you hit >>> the pool limit and your app freezes. >>> >>> On a separate note, we noticed if you open a connection outside of the >>> scope of a transaction, then start a transaction, then create a session >>> with session_transacted option, the session does not participate in the JTA >>> (which seems out of spec). One most open the connection inside the >>> transaction, AND open the session in the transaction, and close the >>> connection in the transaction for everything to work. >>> >>> I'll get a reproducing project created, but I was curious if anyone knew >>> offhand what the spec says. >>> >>> cheers, and thanks, >>> -[the other] Jonathan >>> >>> -- >>> Jonathan | exabr...@gmail.com >>> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >>> half full. >>> Engineers, of course, understand the glass is twice as big as it needs >>> to be. >>> >> >> >> -- >> Jonathan | exabr...@gmail.com >> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >> half full. >> Engineers, of course, understand the glass is twice as big as it needs to >> be. >> > > > -- > Jonathan | exabr...@gmail.com > Pessimists, see a jar as half empty. Optimists, in contrast, see it as > half full. > Engineers, of course, understand the glass is twice as big as it needs to > be. > -- Jonathan | exabr...@gmail.com Pessimists, see a jar as half empty. Optimists, in contrast, see it as half full. Engineers, of course, understand the glass is twice as big as it needs to be.
Re: JTA JMS Spec question, connection leakage
I've narrowed down the problem to AutoConnectionTracker. It's not completing, which isn't allowing the connections to be returned to the pool. https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 getResource() is throwing an IllegalStateException. The JavaDoc ( https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object-) states it should throw an ISE if a current transaction is not Active. The transaction is in the state ROLLED_BACK when AutoConnectionTracker tries to call getResource(). I think the Geronimo implementation ( https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203) maybe be a little too strict. The JTA Spec pdf doesn't offer exact hints of which statuses ( https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html) should be have getResource _not_ throw an ISE unfortunately. I was thinking of changing Geronimo's implementation to check for anything but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. The other way is to cast Transaction to the Geronimo implementation and use Geronimo specific APIs to get call getResource(). Do you guys have any preference which route I should take to fix? On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher wrote: > https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak > > Here's a project that reproduces the bug. This project intentionally > exceeds the transaction timeout (of 1s). Each invocation, the connection is > not returned to the pool and eventually you run out, causing your > application to freeze. > > > > On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher > wrote: > >> Hello Apache friends :) I have a question about the JTA and JMS/RA specs: >> >> If you borrow something from a RA, like a JMS Connection, and you're in >> XA Transaction, is it necessary to call connection.close()? It would seem >> JTA should be smart enough to know the connection is enrolled for 2 phase >> commit and should be smart enough to close it, but I'm not sure if that's >> part of the spec. >> >> In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with >> connectionFactory.createConnection(), and your code fails to call close() >> before the transaction completion, the connection leaks. (And >> unfortunately, calling close() after the transaction completes doesn't >> mitigate the problem). It took awhile for us to track this down. >> >> This becomes a huge problem if you're calling external services in your >> transaction. Let's say you have a reasonable transaction timeout of 30s >> set. You call three services, and they end up taking 15s a piece. Even if >> you're doing the right thing and you have connection.close() in a finally >> block, because your transaction isn't active when you call close, it leaks >> and it just gets "stuck" as an active connection, which eventually you hit >> the pool limit and your app freezes. >> >> On a separate note, we noticed if you open a connection outside of the >> scope of a transaction, then start a transaction, then create a session >> with session_transacted option, the session does not participate in the JTA >> (which seems out of spec). One most open the connection inside the >> transaction, AND open the session in the transaction, and close the >> connection in the transaction for everything to work. >> >> I'll get a reproducing project created, but I was curious if anyone knew >> offhand what the spec says. >> >> cheers, and thanks, >> -[the other] Jonathan >> >> -- >> Jonathan | exabr...@gmail.com >> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >> half full. >> Engineers, of course, understand the glass is twice as big as it needs to >> be. >> > > > -- > Jonathan | exabr...@gmail.com > Pessimists, see a jar as half empty. Optimists, in contrast, see it as > half full. > Engineers, of course, understand the glass is twice as big as it needs to > be. > -- Jonathan | exabr...@gmail.com Pessimists, see a jar as half empty. Optimists, in contrast, see it as half full. Engineers, of course, understand the glass is twice as big as it needs to be.
Re: JTA JMS Spec question, connection leakage
https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak Here's a project that reproduces the bug. This project intentionally exceeds the transaction timeout (of 1s). Each invocation, the connection is not returned to the pool and eventually you run out, causing your application to freeze. On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher wrote: > Hello Apache friends :) I have a question about the JTA and JMS/RA specs: > > If you borrow something from a RA, like a JMS Connection, and you're in XA > Transaction, is it necessary to call connection.close()? It would seem JTA > should be smart enough to know the connection is enrolled for 2 phase > commit and should be smart enough to close it, but I'm not sure if that's > part of the spec. > > In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with > connectionFactory.createConnection(), and your code fails to call close() > before the transaction completion, the connection leaks. (And > unfortunately, calling close() after the transaction completes doesn't > mitigate the problem). It took awhile for us to track this down. > > This becomes a huge problem if you're calling external services in your > transaction. Let's say you have a reasonable transaction timeout of 30s > set. You call three services, and they end up taking 15s a piece. Even if > you're doing the right thing and you have connection.close() in a finally > block, because your transaction isn't active when you call close, it leaks > and it just gets "stuck" as an active connection, which eventually you hit > the pool limit and your app freezes. > > On a separate note, we noticed if you open a connection outside of the > scope of a transaction, then start a transaction, then create a session > with session_transacted option, the session does not participate in the JTA > (which seems out of spec). One most open the connection inside the > transaction, AND open the session in the transaction, and close the > connection in the transaction for everything to work. > > I'll get a reproducing project created, but I was curious if anyone knew > offhand what the spec says. > > cheers, and thanks, > -[the other] Jonathan > > -- > Jonathan | exabr...@gmail.com > Pessimists, see a jar as half empty. Optimists, in contrast, see it as > half full. > Engineers, of course, understand the glass is twice as big as it needs to > be. > -- Jonathan | exabr...@gmail.com Pessimists, see a jar as half empty. Optimists, in contrast, see it as half full. Engineers, of course, understand the glass is twice as big as it needs to be.
JTA JMS Spec question, connection leakage
Hello Apache friends :) I have a question about the JTA and JMS/RA specs: If you borrow something from a RA, like a JMS Connection, and you're in XA Transaction, is it necessary to call connection.close()? It would seem JTA should be smart enough to know the connection is enrolled for 2 phase commit and should be smart enough to close it, but I'm not sure if that's part of the spec. In TomEE 7.0.6 we're noticing that if you borrow a JMS Connection with connectionFactory.createConnection(), and your code fails to call close() before the transaction completion, the connection leaks. (And unfortunately, calling close() after the transaction completes doesn't mitigate the problem). It took awhile for us to track this down. This becomes a huge problem if you're calling external services in your transaction. Let's say you have a reasonable transaction timeout of 30s set. You call three services, and they end up taking 15s a piece. Even if you're doing the right thing and you have connection.close() in a finally block, because your transaction isn't active when you call close, it leaks and it just gets "stuck" as an active connection, which eventually you hit the pool limit and your app freezes. On a separate note, we noticed if you open a connection outside of the scope of a transaction, then start a transaction, then create a session with session_transacted option, the session does not participate in the JTA (which seems out of spec). One most open the connection inside the transaction, AND open the session in the transaction, and close the connection in the transaction for everything to work. I'll get a reproducing project created, but I was curious if anyone knew offhand what the spec says. cheers, and thanks, -[the other] Jonathan -- Jonathan | exabr...@gmail.com Pessimists, see a jar as half empty. Optimists, in contrast, see it as half full. Engineers, of course, understand the glass is twice as big as it needs to be.
