Re: Django Oracle backend vs. numbers

[Shai Berger]

On Tuesday 25 September 2012, Ian Kelly wrote:
> On Tue, Sep 25, 2012 at 11:47 AM, Ian Kelly  wrote:
> > Thanks for putting together a pull request.  Please note the patch
> > will also need to update the documentation to state the new cx_Oracle
> > version requirement (was 4.3.1, now 5.0.1).
> 
> Also, there should probably be something about this in the 1.5 release
> notes.

Yep, forgot about documentation, thanks. PR updated.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Oracle backend vs. numbers

[Shai Berger]

On Wednesday 26 September 2012 20:26:11 Andre Terra wrote:
> On Mon, Sep 24, 2012 at 9:41 PM, Shai Berger  wrote:
> > While preparing the patch, I ran into the inspectdb regression test
> > suite; this suite includes, among other things, a model with a database
> > column named
> > 'prc(%) x', which breaks test database creation on Oracle on current
> > trunk. I
> > intend to include a fix for this (in a separate commit, of course) in the
> > first
> > part as well (doubling percent signs in quote_name()).
> 
> For the record, I believe the relevant ticket and discussion are located
> at:
> 
> https://code.djangoproject.com/ticket/18843
> 

No and maybe: I think it's the "other side" of the same issue. The ticket is 
about problems in inspection; the problem I ran into is in creating a table, 
where the column name is given in the model. My code does nothing to fix the 
inspetion (and indeed, there are failures in the test suite with it); but 
without some fix to the problem I solved, the test suite cannot be run at all.

It may be the problem Anssi refers to in his comment there, I'm not sure.

I can open a ticket for this if core devs like, but it feels like unnecessary 
procedure when it is included with a fix that has already been discussed.

Thanks,
Shai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Oracle backend vs. numbers

[Andre Terra]

On Mon, Sep 24, 2012 at 9:41 PM, Shai Berger  wrote:

> While preparing the patch, I ran into the inspectdb regression test suite;
> this suite includes, among other things, a model with a database column
> named
> 'prc(%) x', which breaks test database creation on Oracle on current
> trunk. I
> intend to include a fix for this (in a separate commit, of course) in the
> first
> part as well (doubling percent signs in quote_name()).
>

For the record, I believe the relevant ticket and discussion are located at:

https://code.djangoproject.com/ticket/18843


Cheers,
AT (airstrike)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DatabaseFeatures and supports_transactions

[maxi]

El martes, 25 de septiembre de 2012 21:01:39 UTC-3, Karen Tracey escribió:
>
> On Tue, Sep 25, 2012 at 11:24 AM, maxi  >wrote:
>
>> El martes, 25 de septiembre de 2012 09:05:47 UTC-3, Karen Tracey 
>> escribió: 
>>
>>> First, this method is used during testing, generally not during normal 
>>> operation. It's implemented as a cached property so regardless of how many 
>>> times the value is tested the underlying code which creates and drops a 
>>> table is only actually called once per database. Is it needed? We need to 
>>> know if the database actually supports transactions or if 
>>> commit/rollback/etc are simply no-ops. If you know of an alternative way of 
>>> determining this, that works across all databases, please share. I added 
>>> that code and I've never liked it but I don't know of another way to 
>>> accomplish what it needs to do.
>>>
>>>
>> No, I just answer because it caught my attention. Why not just trust in a 
>> True/False property ?
>> BTW, What do you mean with "commit/rollback/etc are simply no-ops" ?
>>
>>
> We have no single value we can set for this property for MySQL; whether 
> that DB supports transactions is dependent on the configuration of the DB 
> server. If the MySQL server is configured to use the MyISAM storage engine, 
> then all transaction methods such as commit and rollback simply do nothing 
> (are no-ops).
>  
>
>> I'm working on django-firebird backend implementation (now using the new 
>> firebird python-driver "fdb") and I need reimplement supports_transactions, 
>> because I need to commit any work before drop the table, otherwise, I fall 
>> into "Object in use" error.
>>
>
> That seems odd. What does "object in use" mean? The routine is currently 
> coded to:
>
> 1 - create table
> 2 - commit #1
> 3 - insert row into table
> 4 - rollback #3
> 5 - select count of rows in table
> 6 - read result of #5
> 7 - drop the table
>  
> You need to commit before #7? Doesn't seem like that should be necessary. 
> Can you elaborate on why Firebird thinks it is necessary?
>

Yes, I need to commit before 7 because the transaction is still active in 
that point.

cursor.execute('SELECT COUNT(X) FROM ROLLBACK_TEST')
count, = cursor.fetchone()
self.connection.commit()  # <-- I need add this.
cursor.execute('DROP TABLE ROLLBACK_TEST')
self.connection._commit()

cursor.execute start a new transaction, but cursor.fetchone() does not 
close it. Then I need a explicit commit, otherwise, drop table will fail 
because the table is still in use for the current active transaction.





-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/TSaDbzruc6gJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ORM refactoring, part 2

[Jonas Obrist]

This thread caught my interest (because of potential implications for 
django-hvad).

While the proposed changes break django-hvad, I've already written a simple 
patch for django-hvad, so this is a non-issue. (The problem was the cleanup 
of Query.where. master and 1.4 had lots of calls to start_subtree which 
added WhereNodes into Query.where.children, those children are now tuples 
which broke our code. However the tuples make a lot more sense).

Regarding the actual patch, I like the cleanup in add_q, it's a lot nicer 
to read now (although needs_having is a bit crazy).

On Saturday, August 25, 2012 9:35:02 PM UTC+2, Anssi Kääriäinen wrote:
>
> I have done some more ORM refactoring work. I thought it would be a 
> good idea to post a summary of what is going on. 
>
> First, I haven't committed the utils.tree refactoring patch I was 
> planning to commit [https://github.com/akaariai/django/commits/ 
> refactor_utils_tree]. The reason is that I now feel that add_filter() 
> and deeper levels of the Query class need some refactorings first. 
>
> There is some work going on in the refactoring of the deeper levels of 
> the ORM. I committed join promotion improvement patch today: [https:// 
> code.djangoproject.com/ticket/16715#comment:25]. I have more planned 
> improvements to commit: 
>
>
> * Remove "dupe avoidance" logic from the ORM (#18748). 
> This is removing some code I strongly believe isn't needed any more. 
> This is still pending Malcolm's review. 
>
> * Fix join promotion in disjunctive filters (#18854). 
> This one should make join promotion in disjunction cases cleaner and 
> more efficient (as in less OUTER JOINS). In addition, this will make 
> add_q/add_filter a bit easier to understand. The problem currently is 
> that the first ORed filter is added to the query with connector=AND, 
> the rest of the filters in the disjunction with connector=OR. This is 
> done for join promotion reasons. However, this is very confusing when 
> trying to work with add_q and add_filter, and doesn't actually work 
> that well (see the added tests in the ticket's patch). 
>
> * Remove "trim" argument from add_filter (#18816). 
> The trim argument is only needed for split_exclude (that is, pushing 
> negated m2m filters to subqueries). So, add_filter (and then also 
> trim_joins) needs to handle split_exclude's special case. Handling 
> this case inside split_exclude() turned out to be a little ugly, but 
> IMO less ugly than the trim flag. This also allows further cleanups in 
> the following item. 
>
> * A biggie: complete refactoring of setup_joins (#10790, a separate 
> ticket could be good for this). 
> Best described in the last post in the ticket [https:// 
> code.djangoproject.com/ticket/10790#comment:41]. The patch isn't 
> completely ready (there are still stale comments floating around, the 
> commit history is ugly). 
>
>
> Applying the above patches should make the ORM code easier to follow. 
> Further features like custom lookups should be easier to add. The 
> produced queries should be of higher quality (less joins, less left 
> joins). And, it should be easier to do cleanups in the ORM. 
>
> A note about extra_filters: The setup_joins() refactoring removes the 
> "extra_filters" system used for generic relations. The system adds new 
> filters to the query for any joins generated by generic relations. 
> However, these are pushed into the WHERE clause, and this doesn't work 
> nicely with LOUTER JOINS. The refactoring adds the extra condition 
> directly into the join's ON clause, thus producing correct left outer 
> join conditions. 
>
> The extra_filters is (as far as I know) private API, but I'd like to 
> know if this is heavily used by the community. If so, it is easy 
> enough to leave this API in place (while still fixing the generic 
> relations stuff). 
>
> I hope I can get reviews for the above tickets. Getting reviews from 
> people who do not know the ORM is valuable, too, as one of the goals 
> is to make the ORM easier to understand. As the author I can't easily 
> see if my attempts to make the code easier to follow actually improve 
> anything. 
>
> Even if I do not get any reviews, I think it is a good idea to push 
> these patches in. Thus far it has been hard to get reviews for larger 
> ORM patches, and I am afraid that the refactoring work will stall if I 
> have to wait for a full review for every patch. If you want to review 
> the patches, but don't have time just now, please make a note in the 
> tickets about this. There is no hurry. 
>
> If pushing these patches without reviews seems like a bad idea to you, 
> then please say so (preferably before I commit anything)... 
>
> I am sorry if I haven't worked on other patches I thought I had time 
> to work on. The core ORM refactorings are IMO really important to work 
> on, and thus they have bypassed some other items in my admittedly too 
> long TODO list. 
>
>  - Anssi 
>

-- 
You received this message because you 

Re: DatabaseFeatures and supports_transactions

[maxi]

El martes, 25 de septiembre de 2012 20:57:54 UTC-3, Russell Keith-Magee 
escribió:
>
> On Tue, Sep 25, 2012 at 11:24 PM, maxi  
> wrote: 
> > 
> > 
> > El martes, 25 de septiembre de 2012 09:05:47 UTC-3, Karen Tracey 
> escribió: 
> >> 
> >> On Mon, Sep 24, 2012 at 10:15 PM, maxi  wrote: 
> >>> 
> >>> Hi, 
> >>> 
> >>> DatabseFeatures class has a supports_transactions property for test if 
> >>> the db engine support transactions. 
> >>> supports_transactions implementation makes  some metadata alters to 
> check 
> >>> this behaivor. 
> >>> 
> >>> Now, is really needed to do this to check transaction support? 
> >>> I don't know exactly how many times this supports_transactions is 
> called 
> >>> but I think what it produce unnecessary alterations in database 
> structure, 
> >>> or I'm misinterpreting how it supposed works. 
> >>> 
> >> 
> > 
> > Hi Karen, 
> > 
> >> 
> >> First, this method is used during testing, generally not during normal 
> >> operation. It's implemented as a cached property so regardless of how 
> many 
> >> times the value is tested the underlying code which creates and drops a 
> >> table is only actually called once per database. Is it needed? We need 
> to 
> >> know if the database actually supports transactions or if 
> >> commit/rollback/etc are simply no-ops. If you know of an alternative 
> way of 
> >> determining this, that works across all databases, please share. I 
> added 
> >> that code and I've never liked it but I don't know of another way to 
> >> accomplish what it needs to do. 
> >> 
> > 
> > No, I just answer because it caught my attention. Why not just trust in 
> a 
> > True/False property ? 
>
> As I recall, the reason we can't just use a property is MySQL. There's 
> no reliable way to tell whether the database has been set up as 
> InnoDB, which supports transactions, or MyISAM, which doesn't. An 
> experimental approach allows us to be certain. 
>
> > BTW, What do you mean with "commit/rollback/etc are simply no-ops" ? 
>
> Exactly that. On databases that don't support transactions, 
> commit/rollback etc are no-ops. 
>
> > I'm working on django-firebird backend implementation (now using the new 
> > firebird python-driver "fdb") and I need reimplement 
> supports_transactions, 
> > because I need to commit any work before drop the table, otherwise, I 
> fall 
> > into "Object in use" error. 
>
> Are you saying that the current 'supports_transaction' test is leaving 
> data in an uncommitted state? From a quick eyeball, I don't see how -- 
> which bit is tripping up Firebird? Where do you need to add a commit? 
>

No, I mean if I do select * from TABLE and inmediately I try to drop the 
table whitout make a commit, the table object is still in use and then I 
gen an "Object in use" exception.

Then, I need todo 
  - select * from TABLE  <- fetch
  - commit work
  - drop TABLE

And the current implementation is:

cursor.execute('CREATE TABLE ROLLBACK_TEST (X INT)')
self.connection._commit()
cursor.execute('INSERT INTO ROLLBACK_TEST (X) VALUES (8)')
self.connection._rollback()
cursor.execute('SELECT COUNT(X) FROM ROLLBACK_TEST')
count, = cursor.fetchone()
# Commit is needed here
cursor.execute('DROP TABLE ROLLBACK_TEST')
self.connection._commit()


BTW, I was talking about it with the fdb main developer and he notes what 
there is a possible bug here about how Transaction must work.

So, if  support_transaction is the natural feature from my db engine I can 
just return True. Is Ok?


Best regards.
---
Maxi


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/H88VFE55n-YJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.