Re: Default transaction timeout on screen widget
On Tue, 1 Oct 2019 at 19:06, Scott Gray wrote: > Hi Paul, > > I've never seen a transaction timeout actually cause locks to be released > ahead of time before the transaction finally attempts to commit, are you > saying that happens? OK, now we need to talk about context. The OFBiz transaction timeout uses javax.transaction.TransactionManager.setTransactionTimeout, so it depends on your JDBC driver, but I reckon that's what I'd want - the transaction should fail if it can't do the job within the time limit. Often the default timeout is infinite, so never happens, because what's a reasonable timeout depends on the work the application is doing. An end of financial year transaction might quite legitimately take a weekend to complete. InnoDB has an option innodb_rollback_on_timeout ( https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_rollback_on_timeout). By default it rolls back only the last statement. That seems very weird to me - a rollback should roll back the transaction, not any one statement. Postgres has statement_timeout and idle_in_transaction_session_timeout ( https://www.postgresql.org/docs/9.6/runtime-config-client.html) In my experience if you have a 60 second timeout but > your transaction doesn't try to commit for 15 minutes then the locks will > be held for the full 15 minutes (before finally rolling back when you > attempt to commit). > Then I'd agree with you - the timeout is only informing you your query was slow to execute, and there is no other benefit to the timeout. Might as well have some sort of monitoring for query execution times, and turn off timeouts. If the locks are going to stay on anyway, might as well commit the first transaction if it completes without error. > But no sooner did I ask this question about the relevance of timeouts than > I came across a good use case for them. If your transaction is wrapping an > API call and the intermediate gateways (like a proxy server or whatever) > have a timeout, then it can be better to have the transaction timeout > shorter than the gateway timeout. In this manner the client won't > encounter situations where they receive a gateway timeout error response > but the API call they made is actually successful on the server. > We are used to transactions spanning several changes within a database - the classic example is a transfer between two bank accounts. And transactions should give us the famous all-or-nothing ACID behaviour. Mix in anything else, like the API call, and it gets harder. Queuing services like JMS, MSMQ, ServiceBus, RabbitMQ, AMQP give the same transactional behaviour to sending and receiving messages. Posting a message to a queue is a pending operation that becomes real when you commit a transaction, so any other processing before or after must complete successfully. You can rollback if any error occurs, which undoes the posting of the message. Similarly, at the other end we pull a message off the queue and try to process it. In the event of any failure, a transaction is rolled back and the message is still safely there on the queue. In the absence of queuing, the best you can probably do is build in compensation - call the API, and in the event of a subsequent failure issue a second API call to undo the first. The analogy is like clearing a cheque instead of transferring between your own bank accounts - in the event of a failure, the second bank sends a decline message that undoes the effect of the deposit. Conceptually, there's a transaction but it's built by composing a sequence of smaller and simpler transactions rather than having an overarching one. When you are managing compensation, you have to implement an undo stack for yourself rather than relying on a transaction to do it for you. Cheers Paul Foxworthy -- Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 Australia Phone: +61 3 9585 6788 Web: http://www.coherentsoftware.com.au/ Email: i...@coherentsoftware.com.au
Re: Default transaction timeout on screen widget
Hi Paul, I've never seen a transaction timeout actually cause locks to be released ahead of time before the transaction finally attempts to commit, are you saying that happens? In my experience if you have a 60 second timeout but your transaction doesn't try to commit for 15 minutes then the locks will be held for the full 15 minutes (before finally rolling back when you attempt to commit). But no sooner did I ask this question about the relevance of timeouts than I came across a good use case for them. If your transaction is wrapping an API call and the intermediate gateways (like a proxy server or whatever) have a timeout, then it can be better to have the transaction timeout shorter than the gateway timeout. In this manner the client won't encounter situations where they receive a gateway timeout error response but the API call they made is actually successful on the server. Regards Scott On Thu, 26 Sep 2019 at 00:46, Paul Foxworthy wrote: > On Tue, 17 Sep 2019 at 18:45, Scott Gray > wrote: > > > Has anyone ever had a good experience with transaction timeouts? I feel > > like they achieve nothing useful in that: > > 1. They don't interrupt any work that is taking too long (they just > prevent > > it from being committed) > > 2. By the time the exception is thrown, all the work is already done and > > the result is that it just gets thrown away in exchange for an error > > > > Surely there's a positive side to them, and I've just never seen it? > > > > Hi Scott, > > Timeouts are not good for the transaction, they're good for competing > transactions. > > In many DBMSes, a transaction keeps things locked until commit, to > achieve isolation, the I in ACID. If an unusual error situation arises, or > somebody has designed a transaction (badly) to include a user's think time, > it's possible the transaction does not commit at all, blocking access to > data touched by the transaction for everyone else, indefinitely. > > A transaction timeout is an estimate of a reasonable amount of time within > which the transaction should complete. If it doesn't complete, the timeout > ensures it will roll back, so other transactions won't have to wait too > long. It's something like an operating system segfaulting a rogue process > in order to protect all the others. > > Cheers > > Paul Foxworthy > > -- > Coherent Software Australia Pty Ltd > PO Box 2773 > Cheltenham Vic 3192 > Australia > > Phone: +61 3 9585 6788 > Web: http://www.coherentsoftware.com.au/ > Email: i...@coherentsoftware.com.au >
Re: Default transaction timeout on screen widget
On Tue, 17 Sep 2019 at 18:45, Scott Gray wrote: > Has anyone ever had a good experience with transaction timeouts? I feel > like they achieve nothing useful in that: > 1. They don't interrupt any work that is taking too long (they just prevent > it from being committed) > 2. By the time the exception is thrown, all the work is already done and > the result is that it just gets thrown away in exchange for an error > > Surely there's a positive side to them, and I've just never seen it? > Hi Scott, Timeouts are not good for the transaction, they're good for competing transactions. In many DBMSes, a transaction keeps things locked until commit, to achieve isolation, the I in ACID. If an unusual error situation arises, or somebody has designed a transaction (badly) to include a user's think time, it's possible the transaction does not commit at all, blocking access to data touched by the transaction for everyone else, indefinitely. A transaction timeout is an estimate of a reasonable amount of time within which the transaction should complete. If it doesn't complete, the timeout ensures it will roll back, so other transactions won't have to wait too long. It's something like an operating system segfaulting a rogue process in order to protect all the others. Cheers Paul Foxworthy -- Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 Australia Phone: +61 3 9585 6788 Web: http://www.coherentsoftware.com.au/ Email: i...@coherentsoftware.com.au
Re: Default transaction timeout on screen widget
Hi Nicolas,
I have no objection but I don't think it is something I would ever use. I
would either fix the screen or increase the timeout for that specific
screen.
Regards
Scott
On Fri, 20 Sep 2019 at 20:20, Nicolas Malin
wrote:
> Hello Scott, thanks for your return,
>
> in line ...
>
> On 9/17/19 10:45 AM, Scott Gray wrote:
> > Has anyone ever had a good experience with transaction timeouts? I feel
> > like they achieve nothing useful in that:
> > 1. They don't interrupt any work that is taking too long (they just
> prevent
> > it from being committed)
> > 2. By the time the exception is thrown, all the work is already done and
> > the result is that it just gets thrown away in exchange for an error
> >
> > Surely there's a positive side to them, and I've just never seen it?
>
> Completely agree, it's also my life :) , transaction timeout it's just a
> pain to be sure that you waste your time.
>
> Currently the advantage that I saw, it's detect an unexpected too long
> service and help to spend enough time to optimize it or thinking about
> an other solution.
>
> > But Nicolas, in your case I think the solution lies elsewhere. I don't
> > think it's ever ok for a screen to take over a minute to load and tie up
> > the http connection all that time. Slow pages happen to all of us, but
> IMO
> > the solution always lies in making the screen faster, limiting the amount
> > of data that can be processed in a single request or by making the
> request
> > asynchronous.
>
> Sure, I'll join you there. It's generally my first step, currently I
> most concentrate on the second step, decrease error raise where there is
> no benefit.
>
> Do you have an objection if I implement the properties explained below
> with the current default value (60s)? Like this no change for general
> case, and for server with high load, free will for administrator site to
> adjust it.
>
> Cheers,
>
> Nicolas
>
> >
> > Regards
> > Scott
> >
> > On Wed, 11 Sep 2019 at 06:57, Nicolas Malin
> > wrote:
> >
> >> Hi all,
> >>
> >> Since I increase the sensibility of error message [1], we have on
> >> different screen that take some time to rendering an error throw due to
> >> transaction timeout.
> >>
> >> By default each screen is rendering with the default timeout (60s) that
> >> isn't enough when you have big data compilation or some external service
> >> latency.
> >>
> >> Of course it's possible to analyze each case with purpose to increase
> >> the screen velocity or set a transaction-timeout on screen definition,
> >> but as first easy step what do you think if we add a default transaction
> >> timeout for screen to 10 minutes with possibility to override by
> >> properties ?
> >>
> >> Example:
> >>
> >> *
> >>
> >>
> >>
> framework/widget/src/main/java/org/apache/ofbiz/widget/model/ModelScreen.java
> >> @@ -122,7 +123,7 @@ public class ModelScreen extends ModelWidget {
> >>// wrap the whole screen rendering in a transaction, should
> >> improve performance in querying and such
> >>Map parameters =
> >> UtilGenerics.cast(context.get("parameters"));
> >>boolean beganTransaction = false;
> >> -int transactionTimeout = -1;
> >> +int transactionTimeout =
> >> UtilProperties.getPropertyAsInteger("widget",
> >> "widget.transaction.timeout.default", 600);
> >>if (parameters != null) {
> >>String transactionTimeoutPar =
> >> parameters.get("TRANSACTION_TIMEOUT");
> >>if (transactionTimeoutPar != null) {
> >> @@ -152,12 +153,7 @@ public class ModelScreen extends ModelWidget {
> >>// If transaction timeout is present, use it to start the
> >> transaction
> >>// If transaction timeout is set to zero, no transaction
> >> is started
> >>if (useTransaction) {
> >> -if (transactionTimeout < 0) {
> >> -beganTransaction = TransactionUtil.begin();
> >> -}
> >> -if (transactionTimeout > 0) {
> >> -beganTransaction =
> >> TransactionUtil.begin(transactionTimeout);
> >> -}
> >> +beganTransaction =
> >> TransactionUtil.begin(transactionTimeout);
> >>}
> >>
> >>// render the screen, starting with the top-level section
> >>
> >> ***
> >>
> >> Any remarks ?
> >>
> >> In parallel i will investigate why the error message catch is so
> sensible.
> >>
> >> Nicolas
> >>
> >> [1] http://svn.apache.org/viewvc?view=revision=1856175
> >>
> >> --
> >> logoNrd
Re: Default transaction timeout on screen widget
Hello Scott, thanks for your return,
in line ...
On 9/17/19 10:45 AM, Scott Gray wrote:
Has anyone ever had a good experience with transaction timeouts? I feel
like they achieve nothing useful in that:
1. They don't interrupt any work that is taking too long (they just prevent
it from being committed)
2. By the time the exception is thrown, all the work is already done and
the result is that it just gets thrown away in exchange for an error
Surely there's a positive side to them, and I've just never seen it?
Completely agree, it's also my life :) , transaction timeout it's just a
pain to be sure that you waste your time.
Currently the advantage that I saw, it's detect an unexpected too long
service and help to spend enough time to optimize it or thinking about
an other solution.
But Nicolas, in your case I think the solution lies elsewhere. I don't
think it's ever ok for a screen to take over a minute to load and tie up
the http connection all that time. Slow pages happen to all of us, but IMO
the solution always lies in making the screen faster, limiting the amount
of data that can be processed in a single request or by making the request
asynchronous.
Sure, I'll join you there. It's generally my first step, currently I
most concentrate on the second step, decrease error raise where there is
no benefit.
Do you have an objection if I implement the properties explained below
with the current default value (60s)? Like this no change for general
case, and for server with high load, free will for administrator site to
adjust it.
Cheers,
Nicolas
Regards
Scott
On Wed, 11 Sep 2019 at 06:57, Nicolas Malin
wrote:
Hi all,
Since I increase the sensibility of error message [1], we have on
different screen that take some time to rendering an error throw due to
transaction timeout.
By default each screen is rendering with the default timeout (60s) that
isn't enough when you have big data compilation or some external service
latency.
Of course it's possible to analyze each case with purpose to increase
the screen velocity or set a transaction-timeout on screen definition,
but as first easy step what do you think if we add a default transaction
timeout for screen to 10 minutes with possibility to override by
properties ?
Example:
*
framework/widget/src/main/java/org/apache/ofbiz/widget/model/ModelScreen.java
@@ -122,7 +123,7 @@ public class ModelScreen extends ModelWidget {
// wrap the whole screen rendering in a transaction, should
improve performance in querying and such
Map parameters =
UtilGenerics.cast(context.get("parameters"));
boolean beganTransaction = false;
-int transactionTimeout = -1;
+int transactionTimeout =
UtilProperties.getPropertyAsInteger("widget",
"widget.transaction.timeout.default", 600);
if (parameters != null) {
String transactionTimeoutPar =
parameters.get("TRANSACTION_TIMEOUT");
if (transactionTimeoutPar != null) {
@@ -152,12 +153,7 @@ public class ModelScreen extends ModelWidget {
// If transaction timeout is present, use it to start the
transaction
// If transaction timeout is set to zero, no transaction
is started
if (useTransaction) {
-if (transactionTimeout < 0) {
-beganTransaction = TransactionUtil.begin();
-}
-if (transactionTimeout > 0) {
-beganTransaction =
TransactionUtil.begin(transactionTimeout);
-}
+beganTransaction =
TransactionUtil.begin(transactionTimeout);
}
// render the screen, starting with the top-level section
***
Any remarks ?
In parallel i will investigate why the error message catch is so sensible.
Nicolas
[1] http://svn.apache.org/viewvc?view=revision=1856175
--
logoNrd
Re: Default transaction timeout on screen widget
Same feeling here, i just see it as a smell to analyse, to detect bad
design, system failure (disk access...) or regression.
Regards
Gil
Le 20:45 - mardi 17 sept., Scott Gray a écrit :
> Has anyone ever had a good experience with transaction timeouts? I feel
> like they achieve nothing useful in that:
> 1. They don't interrupt any work that is taking too long (they just prevent
> it from being committed)
> 2. By the time the exception is thrown, all the work is already done and
> the result is that it just gets thrown away in exchange for an error
>
> Surely there's a positive side to them, and I've just never seen it?
>
> But Nicolas, in your case I think the solution lies elsewhere. I don't
> think it's ever ok for a screen to take over a minute to load and tie up
> the http connection all that time. Slow pages happen to all of us, but IMO
> the solution always lies in making the screen faster, limiting the amount
> of data that can be processed in a single request or by making the request
> asynchronous.
>
> Regards
> Scott
>
> On Wed, 11 Sep 2019 at 06:57, Nicolas Malin
> wrote:
>
> > Hi all,
> >
> > Since I increase the sensibility of error message [1], we have on
> > different screen that take some time to rendering an error throw due to
> > transaction timeout.
> >
> > By default each screen is rendering with the default timeout (60s) that
> > isn't enough when you have big data compilation or some external service
> > latency.
> >
> > Of course it's possible to analyze each case with purpose to increase
> > the screen velocity or set a transaction-timeout on screen definition,
> > but as first easy step what do you think if we add a default transaction
> > timeout for screen to 10 minutes with possibility to override by
> > properties ?
> >
> > Example:
> >
> > *
> >
> >
> > framework/widget/src/main/java/org/apache/ofbiz/widget/model/ModelScreen.java
> > @@ -122,7 +123,7 @@ public class ModelScreen extends ModelWidget {
> > // wrap the whole screen rendering in a transaction, should
> > improve performance in querying and such
> > Map parameters =
> > UtilGenerics.cast(context.get("parameters"));
> > boolean beganTransaction = false;
> > -int transactionTimeout = -1;
> > +int transactionTimeout =
> > UtilProperties.getPropertyAsInteger("widget",
> > "widget.transaction.timeout.default", 600);
> > if (parameters != null) {
> > String transactionTimeoutPar =
> > parameters.get("TRANSACTION_TIMEOUT");
> > if (transactionTimeoutPar != null) {
> > @@ -152,12 +153,7 @@ public class ModelScreen extends ModelWidget {
> > // If transaction timeout is present, use it to start the
> > transaction
> > // If transaction timeout is set to zero, no transaction
> > is started
> > if (useTransaction) {
> > -if (transactionTimeout < 0) {
> > -beganTransaction = TransactionUtil.begin();
> > -}
> > -if (transactionTimeout > 0) {
> > -beganTransaction =
> > TransactionUtil.begin(transactionTimeout);
> > -}
> > +beganTransaction =
> > TransactionUtil.begin(transactionTimeout);
> > }
> >
> > // render the screen, starting with the top-level section
> >
> > ***
> >
> > Any remarks ?
> >
> > In parallel i will investigate why the error message catch is so sensible.
> >
> > Nicolas
> >
> > [1] http://svn.apache.org/viewvc?view=revision=1856175
> >
> > --
> > logoNrd
Re: Default transaction timeout on screen widget
Has anyone ever had a good experience with transaction timeouts? I feel
like they achieve nothing useful in that:
1. They don't interrupt any work that is taking too long (they just prevent
it from being committed)
2. By the time the exception is thrown, all the work is already done and
the result is that it just gets thrown away in exchange for an error
Surely there's a positive side to them, and I've just never seen it?
But Nicolas, in your case I think the solution lies elsewhere. I don't
think it's ever ok for a screen to take over a minute to load and tie up
the http connection all that time. Slow pages happen to all of us, but IMO
the solution always lies in making the screen faster, limiting the amount
of data that can be processed in a single request or by making the request
asynchronous.
Regards
Scott
On Wed, 11 Sep 2019 at 06:57, Nicolas Malin
wrote:
> Hi all,
>
> Since I increase the sensibility of error message [1], we have on
> different screen that take some time to rendering an error throw due to
> transaction timeout.
>
> By default each screen is rendering with the default timeout (60s) that
> isn't enough when you have big data compilation or some external service
> latency.
>
> Of course it's possible to analyze each case with purpose to increase
> the screen velocity or set a transaction-timeout on screen definition,
> but as first easy step what do you think if we add a default transaction
> timeout for screen to 10 minutes with possibility to override by
> properties ?
>
> Example:
>
> *
>
>
> framework/widget/src/main/java/org/apache/ofbiz/widget/model/ModelScreen.java
> @@ -122,7 +123,7 @@ public class ModelScreen extends ModelWidget {
> // wrap the whole screen rendering in a transaction, should
> improve performance in querying and such
> Map parameters =
> UtilGenerics.cast(context.get("parameters"));
> boolean beganTransaction = false;
> -int transactionTimeout = -1;
> +int transactionTimeout =
> UtilProperties.getPropertyAsInteger("widget",
> "widget.transaction.timeout.default", 600);
> if (parameters != null) {
> String transactionTimeoutPar =
> parameters.get("TRANSACTION_TIMEOUT");
> if (transactionTimeoutPar != null) {
> @@ -152,12 +153,7 @@ public class ModelScreen extends ModelWidget {
> // If transaction timeout is present, use it to start the
> transaction
> // If transaction timeout is set to zero, no transaction
> is started
> if (useTransaction) {
> -if (transactionTimeout < 0) {
> -beganTransaction = TransactionUtil.begin();
> -}
> -if (transactionTimeout > 0) {
> -beganTransaction =
> TransactionUtil.begin(transactionTimeout);
> -}
> +beganTransaction =
> TransactionUtil.begin(transactionTimeout);
> }
>
> // render the screen, starting with the top-level section
>
> ***
>
> Any remarks ?
>
> In parallel i will investigate why the error message catch is so sensible.
>
> Nicolas
>
> [1] http://svn.apache.org/viewvc?view=revision=1856175
>
> --
> logoNrd
Default transaction timeout on screen widget
Hi all,
Since I increase the sensibility of error message [1], we have on
different screen that take some time to rendering an error throw due to
transaction timeout.
By default each screen is rendering with the default timeout (60s) that
isn't enough when you have big data compilation or some external service
latency.
Of course it's possible to analyze each case with purpose to increase
the screen velocity or set a transaction-timeout on screen definition,
but as first easy step what do you think if we add a default transaction
timeout for screen to 10 minutes with possibility to override by
properties ?
Example:
*
framework/widget/src/main/java/org/apache/ofbiz/widget/model/ModelScreen.java
@@ -122,7 +123,7 @@ public class ModelScreen extends ModelWidget {
// wrap the whole screen rendering in a transaction, should
improve performance in querying and such
Map parameters =
UtilGenerics.cast(context.get("parameters"));
boolean beganTransaction = false;
- int transactionTimeout = -1;
+ int transactionTimeout =
UtilProperties.getPropertyAsInteger("widget",
"widget.transaction.timeout.default", 600);
if (parameters != null) {
String transactionTimeoutPar =
parameters.get("TRANSACTION_TIMEOUT");
if (transactionTimeoutPar != null) {
@@ -152,12 +153,7 @@ public class ModelScreen extends ModelWidget {
// If transaction timeout is present, use it to start the
transaction
// If transaction timeout is set to zero, no transaction
is started
if (useTransaction) {
- if (transactionTimeout < 0) {
- beganTransaction = TransactionUtil.begin();
- }
- if (transactionTimeout > 0) {
- beganTransaction =
TransactionUtil.begin(transactionTimeout);
- }
+ beganTransaction =
TransactionUtil.begin(transactionTimeout);
}
// render the screen, starting with the top-level section
***
Any remarks ?
In parallel i will investigate why the error message catch is so sensible.
Nicolas
[1] http://svn.apache.org/viewvc?view=revision=1856175
--
logoNrd
