Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Anssi Kääriäinen
On 09/23/2013 12:27 AM, Aymeric Augustin wrote: On 22 sept. 2013, at 23:09, Shai Berger wrote: The code base I mentioned earlier relies mostly on TransactionMiddleware, but does have some explicit commit_on_success's etc. (it's Django 1.4 based, so no @atomic or anything like it). I'm genuine

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Michael Manfre
On Sun, Sep 22, 2013 at 5:09 PM, Shai Berger wrote: > If this is agreed, then we should add an assertNoQueries, deprecate the > existing assertNumQueries, and add a new assertNumQueries that takes a > backend > specification, shouldn't we? This is regardless of what we do with @atomic. assertN

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
On 22 sept. 2013, at 23:09, Shai Berger wrote: > The code base I mentioned earlier relies mostly on TransactionMiddleware, but > does have some explicit commit_on_success's etc. (it's Django 1.4 based, so > no > @atomic or anything like it). I'm genuinely interested in making it as easy as po

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Shai Berger
On Sunday 22 September 2013 22:55:15 Aymeric Augustin wrote: > Hi Michael, > > On 22 sept. 2013, at 22:48, Michael Manfre wrote: > > On Sun, Sep 22, 2013 at 3:18 PM, Aymeric Augustin > > wrote: > > > > The extra queries for the savepoints will break all tests that use > > assertNumQueries — wel

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Shai Berger
On Sunday 22 September 2013 12:59:02 Florian Apolloner wrote: > On Sunday, September 22, 2013 8:38:10 PM UTC+2, Shai Berger wrote: > > I would take Anssi's suggestion another step forward -- or backward, > > depends > > where you're looking from :-) -- stop marking transactions for rollback. > > M

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
Hi Michael, On 22 sept. 2013, at 22:48, Michael Manfre wrote: > On Sun, Sep 22, 2013 at 3:18 PM, Aymeric Augustin > wrote: > > The extra queries for the savepoints will break all tests that use > assertNumQueries > — well, all except assertNumQueries(0). How do you propose to deal with that.

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Michael Manfre
On Sun, Sep 22, 2013 at 3:18 PM, Aymeric Augustin < aymeric.augus...@polytechnique.org> wrote: > On 22 sept. 2013, at 20:38, Shai Berger wrote: > The extra queries for the savepoints will break all tests that use > assertNumQueries > — well, all except assertNumQueries(0). How do you propose to d

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Florian Apolloner
On Sunday, September 22, 2013 8:38:10 PM UTC+2, Shai Berger wrote: > > I would take Anssi's suggestion another step forward -- or backward, > depends > where you're looking from :-) -- stop marking transactions for rollback. > Make > save() and associates use savepoints, only on PostgreSQL, so

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
On 22 sept. 2013, at 20:38, Shai Berger wrote: > I would take Anssi's suggestion another step forward -- or backward, depends > where you're looking from :-) -- stop marking transactions for rollback. Make > save() and associates use savepoints, only on PostgreSQL, so that, > everywhere, > o

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Shai Berger
On Sunday 22 September 2013 10:38:32 Anssi Kääriäinen wrote: > > I would go for an approach where all queries in marked-for-rollback block > are prevented. See https://github.com/akaariai/django/compare/ticket_21134 > > Note that this is the test case I am complaining about: > > r1 = Reporter.ob

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Anssi Kääriäinen
On Sunday, September 22, 2013 9:30:06 PM UTC+3, Aymeric Augustin wrote: > > On 22 sept. 2013, at 19:38, Anssi Kääriäinen > > > wrote: > > > There is no explicit use of savepoint=False. It is reasonable to expect > that the code succeeds on databases that allow this coding pattern. > > Yes, you

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Aymeric Augustin
On 22 sept. 2013, at 19:38, Anssi Kääriäinen wrote: > There is no explicit use of savepoint=False. It is reasonable to expect that > the code succeeds on databases that allow this coding pattern. Yes, you're right… I looked at your version of the patch: 1) I believe the low-level APIs (rollb

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-22 Thread Anssi Kääriäinen
On Saturday, September 21, 2013 8:07:19 PM UTC+3, Aymeric Augustin wrote: > > On 21 sept. 2013, at 15:53, Richard Ward > > wrote: > > I'm OK with trying to to raise exceptions when making a query and >> get_rollback returns True. I'm not sure it's doable; patch welcome ;-) > > > How about this

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Aymeric Augustin
On 21 sept. 2013, at 15:53, Richard Ward wrote: > I'm OK with trying to to raise exceptions when making a query and > get_rollback returns True. I'm not sure it's doable; patch welcome ;-) > > How about this: > https://github.com/RichardOfWard/django/commit/cb46c75db275db59b54511c090286255bd9

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Kääriäinen Anssi
tomic" in 1.6 supposed to work this way? From: Aymeric Augustin To: "django-developers@googlegroups.com" CC: Le 21 sept. 2013 à 15:53, Richard Ward mailto:daedalusf...@gmail.com>> a écrit : You say in your docs patch that savepoints are cheap Truth be said, I haven

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Aymeric Augustin
> Le 21 sept. 2013 à 15:53, Richard Ward a écrit : > > You say in your docs patch that savepoints are cheap Truth be said, I haven't run benchmarks. > so what is transaction.atomic(savepoint=False) for? is it just for > performance, or is more like an assertion that we are definitely in a > t

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Richard Ward
> > I'll open a ticket about the docs later today. https://code.djangoproject.com/ticket/21134 > Let me know if you think it can be improved. > That looks nice and explicit to me. I'm OK with trying to to raise exceptions when making a query and > get_rollback returns True. I'm not sure

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-21 Thread Richard Ward
Oh dear, I just posted using the wrong Google account! Short version: ticket here: https://code.djangoproject.com/ticket/21134 Possible patch for raising exceptions when using broken transactions here: https://github.com/RichardOfWard/django/commit/cb46c75db275db59b54511c090286255bd9cc46d There

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-20 Thread Aymeric Augustin
On 20 sept. 2013, at 10:32, Richard Ward wrote: > I'll open a ticket about the docs later today. I just took a few minutes to write a patch: https://github.com/django/django/commit/4db2752e28668ed8826b770ef2ed26e8c1562db6 Let me know if you think it can be improved. -- Aymeric. -- You re

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-20 Thread Richard Ward
> > So you don't think that "one step of an atomic operation failing" is > reason to fail the atomic operation? Doesn't sound intuitive to me. > ... > The concept of an "atomic" operation, even outside the context of DBMSs, is > pretty much the consistent throughout CS. > > If you are expect

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
Yes, I agree, that why I suggested filing a documentation ticket in my first answer. -- Aymeric. > Le 20 sept. 2013 à 01:32, Richard Ward a écrit : > > I think the docs could be improved -- You received this message because you are subscribed to the Google Groups "Django developers" group.

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
On 19 sept. 2013, at 23:38, Anssi Kääriäinen wrote: > Here, another_obj.save() will succeed. But the transaction will be *always* > rolled back, silently. I don't see any good reason not to error out when > using a connection which is going to be rolled back in always. As far as I can tell th

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
On 19 sept. 2013, at 22:47, Hannu Krosing wrote: > Actually you can use subtransactions in postgresql if you want to manage > failed queries yourself That's exactly what transaction.atomic does, on all supported databases. I'm sorry if I sound like I need a transactions 101 course; I actually s

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Curtis Maloney
I feel compelled to comment here... On 20 September 2013 09:32, Richard Ward wrote: > > I don't think that what I was trying to do (continue using a transaction > after an insert failed) was too outlandish. I'm not demanding that Django > should allow me to do it (tho that would be nice), but I

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Richard Ward
Thanks for the explanations. I wasn't aware that particular difference between PostgreSQL and MySQL (I've not done much with PostgreSQL). Perhaps I could have been clearer in my original post: Ansii is right about silence... when I call my_func the transaction.atomic decorator rolls back the tr

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Marc Tamlyn
This is what nested atomics do. In effect savepoints are just creating another transaction inside the existing one (if my memories of Aymeric's talk are correct). On 19 Sep 2013 21:48, "Hannu Krosing" wrote: > On 09/19/2013 04:52 PM, Anssi Kääriäinen wrote: > > After some investigation it turns o

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Anssi Kääriäinen
I think there is a case of talking past each other going on here. On Thursday, September 19, 2013 11:33:00 PM UTC+3, Aymeric Augustin wrote: > Nonetheless, in my opinion, the root cause of the perceived misbehavior is > that user code (not Django code) catches IntegrityError inside an atomic >

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Hannu Krosing
On 09/19/2013 04:52 PM, Anssi Kääriäinen wrote: > After some investigation it turns out that this isn't about > IntegrityErrors at all. Instead the problem is that inside @atomic block > Model.save() uses @atomic(savepoint=False) internally. And > @atomic(savepoint=False) forces the outer atomic bl

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
On 19 sept. 2013, at 16:52, Anssi Kääriäinen wrote: > After some investigation it turns out that this isn't about IntegrityErrors > at all. Instead the problem is that inside @atomic block > Model.save() uses @atomic(savepoint=False) internally. And > @atomic(savepoint=False) forces the outer a

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Anssi Kääriäinen
After some investigation it turns out that this isn't about IntegrityErrors at all. Instead the problem is that inside @atomic block Model.save() uses @atomic(savepoint=False) internally. And @atomic(savepoint=False) forces the outer atomic block to roll back if errors happen. If I recall corre

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
2013/9/19 Anssi Kääriäinen > There is a definite disparency about what the code does and what the docs > say: > """ > If the block of code is successfully completed, the changes are committed > to the database. If there is an exception, the changes are rolled back. > """ > > If an exception is ca

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Anssi Kääriäinen
There is a definite disparency about what the code does and what the docs say: """ If the block of code is successfully completed, the changes are committed to the database. If there is an exception, the changes are rolled back. """ If an exception is catched inside the block, then there should

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Aymeric Augustin
2013/9/19 Richard Ward > So what is the problem here? I assume it is one of: > >1. 'innocent_looking_function' is badly written: it should not be >catching IntegrityErrors under any circumstances (though that seems like a >valid thing to do), it should instead use something like get_

Re: Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Gert Van Gool
If the `innocent_looking_function` would use transactions. And thus handles the `IntegrityError by` issuing a rollback, just like `get_or_create` does ( https://github.com/django/django/blob/1.6b4/django/db/models/query.py#L360-L390 ). Do you see the same behaviour in `my_func`? -- Gert Mobile: +

Is "transaction.atomic" in 1.6 supposed to work this way?

2013-09-19 Thread Richard Ward
We're building a site that needs to use transactions, and have been doing so against the 1.6 beta as 1.6 is nearly out and we thought it would be easier to use the new transaction api, but we came across an unexpected problem. Basically if you catch an IntegrityError and 'recover' from it, the