[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559864#comment-16559864 ] Jeff Genender commented on AMQ-7015: Thanks CSL. I hope Gary can chime in above about a FILE parameter which I think would really help appease his concerns and this could be a cool enhancement for everyone. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559856#comment-16559856 ] Christopher L. Shannon commented on AMQ-7015: - No problem, I was just pointing out he wasn't -1 so we can just leave it as for the 5.15.5 release because you guys feel strongly about it. And I don't want to get into a big long thing about this so this is the last thing I will sayyes they are users and sure their opinion counts of course. But since I've been part of the community for the past 3-4 years it has been pointed out numerous times that the community is everyone and more than one vendor. So I agree their opinion counts but strongly disagree that based on that you can make a blanket statement and say there is "community support" without anyone else's opinion. (Although you could probably make the argument that there is lazy consensus since no one said they disagree) > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559833#comment-16559833 ] Jeff Genender commented on AMQ-7015: My apologies CLS, I read what Gary wrong. I changed it. Yes... that is community support. They are users. Call it what you want. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559827#comment-16559827 ] Jeff Genender commented on AMQ-7015: Lets do this... how about we add a FILE option to the attribute that looks for a file that can have "COMMIT" or "ROLLBACK" in it. If the file exists, it does what it needs and then deletes the file, so it is one time. By doing this allow you to have a permanent setting, a one time settings, or defualt to off. Gary, will that work? > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559829#comment-16559829 ] Christopher L. Shannon commented on AMQ-7015: - I was going to stay out of it but seriously Jeff? There is community support? There are 4 responses on that thread and all of the people work for you and the same company. That's not community support, that is a single vendor solving a problem for one customer they have. I'm not saying that I agree or disagree with the issue (i'm more or less +0 on this feature) but I take offense when you claim that's community support when you know it isn't. Furthermore, Gary specifically said that he won't -1 this issue. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559808#comment-16559808 ] Jeff Genender commented on AMQ-7015: I'm sorry Gary, that is not a technical reason, its a procedural reason. This does not affect the running broker and there are dozens of settings in ActiveMQ that changes the way the broker works. This does not alter the handling of XA transactions unless it is set, and it alters the setting just like the mbean does. So I disagree. The sad part is you are not offering other possibilities to have all of this. You are digging in your heals. There is now community support for this option: http://activemq.2283324.n4.nabble.com/Discuss-AMQ7015-tp4741551.html Unfortunately, unless you have a technical reason, your -1 doesn't stick and it goes in. If you want to bring this to the PMC, we can discuss this there. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559757#comment-16559757 ] Gary Tully commented on AMQ-7015: - unlike ignoreMissingJournalfiles/checkForCorruptJournalFiles leaving this on for a running broker is broken, technically! I can't add any more to the discussion. The mbean OOM is a bug. But i am not going to -1 this, do what you must but I would tackle this differently, possibly with some command line tool that flies through the journal. I know there is more work to doing that and it would not be a quick fix. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559734#comment-16559734 ] Jeff Genender commented on AMQ-7015: The mbean is fine for small numbers of records. If you have a large XA batch, then the mbean can actually cause a OOM due to the amount. So I disagree. We need a way to configure this inside the broker as a part of recovery. I do not see this any different than the ignoreMissingJournalfiles/checkForCorruptJournalFiles parameters. Perhaps the user wants the setting permanent simply due to large batch sizes. This would certainly be an enhancement for that. With the MBean *and* this parameter, you have the best of doing it all ways. Your objection appears to be non-technical in nature. Can you please provide a technical reason to not do this? Unless you have one, I do not see a reason to hold this back. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559472#comment-16559472 ] Gary Tully commented on AMQ-7015: - my issue is that you are using xml config to do an admin task that would be better handled with the admin api - the mbeans. This new flag only makes senses for a single broker restart, then needs to be unflipped and restarted. Running a broker with the bit set is broken, it is equivalent to xaRecovery=false but it is not implemented like that. if the issue is that using jmx is out of scope, then maybe we need to address that in some other way. But to my mind mbean.rollbackAll is the way to go. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558411#comment-16558411 ] Jeff Genender commented on AMQ-7015: The new code calls commit and rollback with 28819aea4aa981d33c710d9d5e26f3cb6e03c1de calls commit and rollback, with the store command. Is that disabled? From the code paths, it looks like it does a full commit or rollback and isn't disabled. I think what I am failing to understand is how that does not act nearly identically to the MBean? > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558405#comment-16558405 ] Gary Tully commented on AMQ-7015: - Having a mbean.rollbackAll operation leaves the broker still capable to do XA recovery for subsequent XA transactions. Having a kahadb flag effectively disables xa recovery. If that flag is set and recovery will not be used, then there is no need to do the work of persisting the prepare outcome. That feature, to my mind is xaRecovery=false. I don't know the use case but do you really intend to disable xa recovery fully? > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558378#comment-16558378 ] Jeff Genender commented on AMQ-7015: Hi Gary, I would love to have longer conversation to that we could discuss this more in-depth. I am a bit confused. The setting now acts nearly identical to the MBean, but works en-masse. It allows you to select comit/rollback/ nor nothing (default). The unit tests appear to confirm that it works whereby the commit adds the messages through, the rollback removes them. It seems to so the same as the MBean. Could you help explain why this new change does not work? > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[
https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558171#comment-16558171
]
Gary Tully commented on AMQ-7015:
-
[~jgenender] there may be cross wires here, my suggestion was to have an op on
the MBean to commit/rollback all as a convenience. In any event, is needs to be
an explicit operation to force a heuristic outcome. From an XA transaction
point of view the heuristic has got to be informed by the other XA resources in
the mix or the state of the distributed transaction as observed by the
transaction manager and its log.
I can't think of a use case that would know in advance if commit is appropriate
broker wide. Possibly rolling back is ok, but it will upset the distributed
transaction manager which will have to record the adverse outcome in some way.
however, In the analogy with persistence=false, xaRecovery=false may make
sense, and we can avoid sync storing a prepare record such that on first
recovery any pending xa transaction will rollback.
To achieve that, from a kahadb perspective, deciding to not store prepare
messages would suffice. The trace timers get in the way, but the index
processing does not care about the location and the prepare record is always
sync. With this change I think you would get what you are after, ie: no xa
recovery on restart.
{code:java}
diff --git
a/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/MessageDatabase.java
b/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/MessageDatabase.java
index 0e5c237d4..1016a9087 100644
---
a/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/MessageDatabase.java
+++
b/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/MessageDatabase.java
@@ -1117,16 +1117,22 @@ public abstract class MessageDatabase extends
ServiceSupport implements BrokerSe
* the JournalMessage is used to update the index just like it would be
done
* during a recovery process.
*/
+ boolean xaRecovery = true;
public Location store(JournalCommand data, boolean sync, IndexAware
before, Runnable after, Runnable onJournalStoreComplete) throws IOException {
try {
- ByteSequence sequence = toByteSequence(data);
- Location location;
+ ByteSequence sequence = null;
+ if (!xaRecovery || data.type() !=
KahaEntryType.KAHA_PREPARE_COMMAND) {
+ sequence = toByteSequence(data);
+ }
+ Location location = null;
checkpointLock.readLock().lock();
try {
long start = System.currentTimeMillis();
- location = onJournalStoreComplete == null ?
journal.write(sequence, sync) : journal.write(sequence, onJournalStoreComplete)
;
+ if (sequence != null) {
+ location = onJournalStoreComplete == null ?
journal.write(sequence, sync) : journal.write(sequence, onJournalStoreComplete);
+ }
long start2 = System.currentTimeMillis();
//Track the last async update so we know if we need to sync at
the next checkpoint
if (!sync && journal.isJournalDiskSyncPeriodic()) {{code}
> Startup performance improvement when log contains prepared transactions.
>
>
> Key: AMQ-7015
> URL: https://issues.apache.org/jira/browse/AMQ-7015
> Project: ActiveMQ
> Issue Type: New Feature
>Affects Versions: 5.15.4
>Reporter: Heath Kesler
>Assignee: Jeff Genender
>Priority: Minor
> Fix For: 5.16.0, 5.15.5
>
>
> I have a KahaDB that's performing a recovery on each startup.
> Digging deeper into it I've found that the issue is that the db.log contains
> prepared transactions.
> The MessageDatabase will discard those entries in memory, however it does not
> remove transaction info from those messages (I believe that's by design). So
> on each restart, the broker will find those entries and again discard them in
> memory.
> If I access the broker via JMX, I can go to the prepared XAs and execute a
> clear on them one by one.
> When i restart my broker, i don't have a recovery attempted again.
> Performing a remove operation for each message can be very time consuming, so
> i'd like to introduce an optional parameter to allow all prepared XAs to be
> removed on recovery.
> Please see my forth coming patch with unit test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[
https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556145#comment-16556145
]
Jeff Genender commented on AMQ-7015:
Hi Gary,
Your idea makes a lot of sense. Made the change to be
{{purgeRecoveredXATransactionStrategy}} and may be never/commit/rollback.
Thanks for your input and feedback!
> Startup performance improvement when log contains prepared transactions.
>
>
> Key: AMQ-7015
> URL: https://issues.apache.org/jira/browse/AMQ-7015
> Project: ActiveMQ
> Issue Type: New Feature
>Affects Versions: 5.15.4
>Reporter: Heath Kesler
>Assignee: Jeff Genender
>Priority: Minor
> Fix For: 5.16.0, 5.15.5
>
>
> I have a KahaDB that's performing a recovery on each startup.
> Digging deeper into it I've found that the issue is that the db.log contains
> prepared transactions.
> The MessageDatabase will discard those entries in memory, however it does not
> remove transaction info from those messages (I believe that's by design). So
> on each restart, the broker will find those entries and again discard them in
> memory.
> If I access the broker via JMX, I can go to the prepared XAs and execute a
> clear on them one by one.
> When i restart my broker, i don't have a recovery attempted again.
> Performing a remove operation for each message can be very time consuming, so
> i'd like to introduce an optional parameter to allow all prepared XAs to be
> removed on recovery.
> Please see my forth coming patch with unit test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556133#comment-16556133 ] ASF subversion and git services commented on AMQ-7015: -- Commit 28819aea4aa981d33c710d9d5e26f3cb6e03c1de in activemq's branch refs/heads/master from [~jgenender] [ https://git-wip-us.apache.org/repos/asf?p=activemq.git;h=28819ae ] AMQ-7015 - Changed attribute to purgeRecoveredXATransactionStrategy and allow NEVER, COMMIT, and ROLLBACK > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556131#comment-16556131 ] ASF subversion and git services commented on AMQ-7015: -- Commit 2a2e01df6a0a68ab5d74886e21416f4e8425bc0c in activemq's branch refs/heads/activemq-5.15.x from [~jgenender] [ https://git-wip-us.apache.org/repos/asf?p=activemq.git;h=2a2e01d ] AMQ-7015 - Changed attribute to purgeRecoveredXATransactionStrategy and allow NEVER, COMMIT, and ROLLBACK > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555882#comment-16555882 ] Jeff Genender commented on AMQ-7015: Hi Gary, Patch appears to be a stop gap way to purge all prepared transactions that are stuck in recovery. There is an mbean that allows full commit/rollback which is awesome. But for large amounts of XA stuck in prepare, the mbean can be unwieldy and users need a quick way to clean them out otherwise a slow recovery is kicked off on every restart. This is a switch that allows the XA messages to be purged from recovery in one shot. Its a switch and its off by default. Its a convenience setting. Let us know your thoughts on that. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Assignee: Jeff Genender >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552539#comment-16552539 ] Gary Tully commented on AMQ-7015: - this feature breaks the xa contract, recovery is part of xa, the TM will deliver the outcome, that is part of it's contract. The MBean is there to allow an override, xa heuristics. If it is too tedious to individually rollback each transaction, maybe add an option to the mbean to rollback/commit all pending transactions. What does it mean to use an XA transaction with purgeRecoveredXATransactions=true? I guess it is analogous to running a broker with persistence=false and sending a persistent message. However in that case, the feature is better enabled by not writing a prepare record at all. That will have the same effect, be more explicit and preform better. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > Fix For: 5.16.0, 5.15.5 > > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549639#comment-16549639 ] Jeff Genender commented on AMQ-7015: Great patch Heath. Thanks for your contribution! > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549638#comment-16549638 ] Jamie goodyear commented on AMQ-7015: - I've updated [http://activemq.apache.org/kahadb.html] to reference the new optional parameter. > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549637#comment-16549637 ] ASF subversion and git services commented on AMQ-7015: -- Commit 24b9ae2ed3c01ae9c67a94bfa50ca4037228f27b in activemq's branch refs/heads/master from hkesler [ https://git-wip-us.apache.org/repos/asf?p=activemq.git;h=24b9ae2 ] AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549631#comment-16549631 ] ASF subversion and git services commented on AMQ-7015: -- Commit ce7498c971b99e2515f07aab36418a1a0f19c03e in activemq's branch refs/heads/activemq-5.15.x from hkesler [ https://git-wip-us.apache.org/repos/asf?p=activemq.git;h=ce7498c ] AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549633#comment-16549633 ] ASF GitHub Bot commented on AMQ-7015: - Github user asfgit closed the pull request at: https://github.com/apache/activemq/pull/290 > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549632#comment-16549632 ] ASF subversion and git services commented on AMQ-7015: -- Commit 63779e2f78001ffbbb490e5b00bf83134be58756 in activemq's branch refs/heads/activemq-5.15.x from [~jgenender] [ https://git-wip-us.apache.org/repos/asf?p=activemq.git;h=63779e2 ] AMQ-7015 This closes #290 > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-7015) Startup performance improvement when log contains prepared transactions.
[ https://issues.apache.org/jira/browse/AMQ-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549626#comment-16549626 ] ASF GitHub Bot commented on AMQ-7015: - GitHub user heathkesler opened a pull request: https://github.com/apache/activemq/pull/290 AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery You can merge this pull request into a Git repository by running: $ git pull https://github.com/heathkesler/activemq AMQ-7015 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq/pull/290.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #290 commit ce7498c971b99e2515f07aab36418a1a0f19c03e Author: hkesler Date: 2018-07-19T17:53:04Z AMQ-7015 Added a purgeRecoveredXATransactions property on the KahaDB adaptor to purge prepared XA messages on recovery > Startup performance improvement when log contains prepared transactions. > > > Key: AMQ-7015 > URL: https://issues.apache.org/jira/browse/AMQ-7015 > Project: ActiveMQ > Issue Type: New Feature >Affects Versions: 5.15.4 >Reporter: Heath Kesler >Priority: Minor > > I have a KahaDB that's performing a recovery on each startup. > Digging deeper into it I've found that the issue is that the db.log contains > prepared transactions. > The MessageDatabase will discard those entries in memory, however it does not > remove transaction info from those messages (I believe that's by design). So > on each restart, the broker will find those entries and again discard them in > memory. > If I access the broker via JMX, I can go to the prepared XAs and execute a > clear on them one by one. > When i restart my broker, i don't have a recovery attempted again. > Performing a remove operation for each message can be very time consuming, so > i'd like to introduce an optional parameter to allow all prepared XAs to be > removed on recovery. > Please see my forth coming patch with unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
