Hey Shai -- Generally sounds like a good idea, I don't see why this shouldn't go in. You might want to wait until someone with more Oracle experience chimes in, but I'm having a hard time seeing why a number -> str -> number dance would be a good idea.
I'm generally adverse to introducing more settings -- settings creep is a real problem -- so a couple of alternate ideas you might consider in producing a patch: 1. Just pick one correct behavior, probably for each type. My guess would be that ints and floats should get the fast behavior, and decimals the slow-but-correct one. If you're using a decimal field it's because you care about precision, so correctness makes more sense there IMO. 2. If you must introduce a setting, do it in DATABASES[OPTIONS] instead of a top-level ORACLE_FAST_NUMBERS or whatever. Jacob On Wed, Aug 29, 2012 at 7:14 AM, Shai Berger <s...@platonix.com> wrote: > Hi Django developers, > > I've been working with Oracle on a project with some performance requirements, > and we've noticed long ago that the Django Oracle backend performs > significantly worse than other backends; we've been able to profile it and see > that, indeed, a lot of time is spent in processing the returned rows, while > the Oracle DBMS itself is not breaking a sweat. > > This week, I found what is, I think, the main culprit: The Oracle backend > prefers accuracy to performance. Since Oracle numbers can have more > significant > digits than C doubles (=CPython floats), and the backend wishes to preserve > this accuracy, it makes cxOracle (the Python Oracle client library) return all > numbers as strings, and then converts the strings back to numbers -- ints, > floats or decimals -- according to the Oracle data type descriptions. > > To make things even slower, the conversion decisions are made anew for each > row; this is done in the function _rowfactory (currently at > https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L790). > The function itself is called less efficiently than possible (on rows returned > from cxOracle, instead of within cxOracle before the rows are returned). > > For our application -- as for many others, I guess -- precision of 38 digits, > on real numbers or integers, is not important. I made a modified backend which > returns numbers as numbers and the improvement in performance is very > significant. A row-factory function is still required to make sure strings are > returned as unicode, and creating such a function per query to only go over > the string columns may improve things even more, but I didn't even have to go > there yet. > > Can we put this into Django core? Perhaps with a Settings knob for people who > really need all the precision? > > Thanks, > Shai. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
