Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Anssi Kääriäinen
On 09/23/2013 12:27 AM, Aymeric Augustin wrote: On 22 sept. 2013, at 23:09, Shai Berger wrote: The code base I mentioned earlier relies mostly on TransactionMiddleware, but does have some explicit commit_on_success's etc. (it's Django 1.4 based, so no @atomic or anything

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Michael Manfre
On Sun, Sep 22, 2013 at 5:09 PM, Shai Berger wrote: > If this is agreed, then we should add an assertNoQueries, deprecate the > existing assertNumQueries, and add a new assertNumQueries that takes a > backend > specification, shouldn't we? This is regardless of what we do

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
On 22 sept. 2013, at 23:09, Shai Berger wrote: > The code base I mentioned earlier relies mostly on TransactionMiddleware, but > does have some explicit commit_on_success's etc. (it's Django 1.4 based, so > no > @atomic or anything like it). I'm genuinely interested in

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Shai Berger
On Sunday 22 September 2013 22:55:15 Aymeric Augustin wrote: > Hi Michael, > > On 22 sept. 2013, at 22:48, Michael Manfre wrote: > > On Sun, Sep 22, 2013 at 3:18 PM, Aymeric Augustin > > wrote: > > > > The extra queries for the savepoints

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Shai Berger
On Sunday 22 September 2013 12:59:02 Florian Apolloner wrote: > On Sunday, September 22, 2013 8:38:10 PM UTC+2, Shai Berger wrote: > > I would take Anssi's suggestion another step forward -- or backward, > > depends > > where you're looking from :-) -- stop marking transactions for rollback. > >

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
Hi Michael, On 22 sept. 2013, at 22:48, Michael Manfre wrote: > On Sun, Sep 22, 2013 at 3:18 PM, Aymeric Augustin > wrote: > > The extra queries for the savepoints will break all tests that use > assertNumQueries > — well, all except

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Michael Manfre
On Sun, Sep 22, 2013 at 3:18 PM, Aymeric Augustin < aymeric.augus...@polytechnique.org> wrote: > On 22 sept. 2013, at 20:38, Shai Berger wrote: > The extra queries for the savepoints will break all tests that use > assertNumQueries > — well, all except assertNumQueries(0). How

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Florian Apolloner
On Sunday, September 22, 2013 8:38:10 PM UTC+2, Shai Berger wrote: > > I would take Anssi's suggestion another step forward -- or backward, > depends > where you're looking from :-) -- stop marking transactions for rollback. > Make > save() and associates use savepoints, only on PostgreSQL,

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
On 22 sept. 2013, at 20:38, Shai Berger wrote: > I would take Anssi's suggestion another step forward -- or backward, depends > where you're looking from :-) -- stop marking transactions for rollback. Make > save() and associates use savepoints, only on PostgreSQL, so that,

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Shai Berger
On Sunday 22 September 2013 10:38:32 Anssi Kääriäinen wrote: > > I would go for an approach where all queries in marked-for-rollback block > are prevented. See https://github.com/akaariai/django/compare/ticket_21134 > > Note that this is the test case I am complaining about: > > r1 =

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Anssi Kääriäinen
On Sunday, September 22, 2013 9:30:06 PM UTC+3, Aymeric Augustin wrote: > > On 22 sept. 2013, at 19:38, Anssi Kääriäinen > > wrote: > > > There is no explicit use of savepoint=False. It is reasonable to expect > that the code succeeds on databases that allow this coding

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
On 22 sept. 2013, at 19:38, Anssi Kääriäinen wrote: > There is no explicit use of savepoint=False. It is reasonable to expect that > the code succeeds on databases that allow this coding pattern. Yes, you're right… I looked at your version of the patch: 1) I believe

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Anssi Kääriäinen
On Saturday, September 21, 2013 8:07:19 PM UTC+3, Aymeric Augustin wrote: > > On 21 sept. 2013, at 15:53, Richard Ward > wrote: > > I'm OK with trying to to raise exceptions when making a query and >> get_rollback returns True. I'm not sure it's doable; patch welcome ;-)

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Aymeric Augustin
On 21 sept. 2013, at 15:53, Richard Ward wrote: > I'm OK with trying to to raise exceptions when making a query and > get_rollback returns True. I'm not sure it's doable; patch welcome ;-) > > How about this: >

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Kääriäinen Anssi
For the performance part: a simple model.save() is about 50% more expensive with savepoints. This time is used in the database. In addition there are 3 network trips instead of one. This could add latency in some usecases. Original message Subject: Re: Is

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Aymeric Augustin
> Le 21 sept. 2013 à 15:53, Richard Ward a écrit : > > You say in your docs patch that savepoints are cheap Truth be said, I haven't run benchmarks. > so what is transaction.atomic(savepoint=False) for? is it just for > performance, or is more like an assertion that we

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-20 Thread Aymeric Augustin
On 20 sept. 2013, at 10:32, Richard Ward wrote: > I'll open a ticket about the docs later today. I just took a few minutes to write a patch: https://github.com/django/django/commit/4db2752e28668ed8826b770ef2ed26e8c1562db6 Let me know if you think it can be improved.

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-20 Thread Richard Ward
> > So you don't think that "one step of an atomic operation failing" is > reason to fail the atomic operation? Doesn't sound intuitive to me. > ... > The concept of an "atomic" operation, even outside the context of DBMSs, is > pretty much the consistent throughout CS. > > If you are

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-20 Thread Aymeric Augustin
Yes, I agree, that why I suggested filing a documentation ticket in my first answer. -- Aymeric. > Le 20 sept. 2013 à 01:32, Richard Ward a écrit : > > I think the docs could be improved -- You received this message because you are subscribed to the Google Groups

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-20 Thread Aymeric Augustin
On 19 sept. 2013, at 23:38, Anssi Kääriäinen wrote: > Here, another_obj.save() will succeed. But the transaction will be *always* > rolled back, silently. I don't see any good reason not to error out when > using a connection which is going to be rolled back in

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-20 Thread Aymeric Augustin
On 19 sept. 2013, at 22:47, Hannu Krosing wrote: > Actually you can use subtransactions in postgresql if you want to manage > failed queries yourself That's exactly what transaction.atomic does, on all supported databases. I'm sorry if I sound like I need a transactions 101

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Curtis Maloney
I feel compelled to comment here... On 20 September 2013 09:32, Richard Ward wrote: > > I don't think that what I was trying to do (continue using a transaction > after an insert failed) was too outlandish. I'm not demanding that Django > should allow me to do it (tho

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Richard Ward
Thanks for the explanations. I wasn't aware that particular difference between PostgreSQL and MySQL (I've not done much with PostgreSQL). Perhaps I could have been clearer in my original post: Ansii is right about silence... when I call my_func the transaction.atomic decorator rolls back the

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Marc Tamlyn
This is what nested atomics do. In effect savepoints are just creating another transaction inside the existing one (if my memories of Aymeric's talk are correct). On 19 Sep 2013 21:48, "Hannu Krosing" wrote: > On 09/19/2013 04:52 PM, Anssi Kääriäinen wrote: > > After some

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Anssi Kääriäinen
I think there is a case of talking past each other going on here. On Thursday, September 19, 2013 11:33:00 PM UTC+3, Aymeric Augustin wrote: > Nonetheless, in my opinion, the root cause of the perceived misbehavior is > that user code (not Django code) catches IntegrityError inside an atomic >

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Hannu Krosing
On 09/19/2013 04:52 PM, Anssi Kääriäinen wrote: > After some investigation it turns out that this isn't about > IntegrityErrors at all. Instead the problem is that inside @atomic block > Model.save() uses @atomic(savepoint=False) internally. And > @atomic(savepoint=False) forces the outer atomic

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
On 19 sept. 2013, at 16:52, Anssi Kääriäinen wrote: > After some investigation it turns out that this isn't about IntegrityErrors > at all. Instead the problem is that inside @atomic block > Model.save() uses @atomic(savepoint=False) internally. And >

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Anssi Kääriäinen
After some investigation it turns out that this isn't about IntegrityErrors at all. Instead the problem is that inside @atomic block Model.save() uses @atomic(savepoint=False) internally. And @atomic(savepoint=False) forces the outer atomic block to roll back if errors happen. If I recall

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
2013/9/19 Anssi Kääriäinen > There is a definite disparency about what the code does and what the docs > say: > """ > If the block of code is successfully completed, the changes are committed > to the database. If there is an exception, the changes are rolled back. > """

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Anssi Kääriäinen
There is a definite disparency about what the code does and what the docs say: """ If the block of code is successfully completed, the changes are committed to the database. If there is an exception, the changes are rolled back. """ If an exception is catched inside the block, then there should

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
2013/9/19 Richard Ward > So what is the problem here? I assume it is one of: > >1. 'innocent_looking_function' is badly written: it should not be >catching IntegrityErrors under any circumstances (though that seems like a >valid thing to do), it should

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Gert Van Gool
If the `innocent_looking_function` would use transactions. And thus handles the `IntegrityError by` issuing a rollback, just like `get_or_create` does ( https://github.com/django/django/blob/1.6b4/django/db/models/query.py#L360-L390 ). Do you see the same behaviour in `my_func`? -- Gert Mobile:

Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Richard Ward
We're building a site that needs to use transactions, and have been doing so against the 1.6 beta as 1.6 is nearly out and we thought it would be easier to use the new transaction api, but we came across an unexpected problem. Basically if you catch an IntegrityError and 'recover' from it,