Re: MS SQL pyodbc backend update to trunk

[vcc]

Fixed, thanks.

- Original Message - 
From: "bob84123" <[EMAIL PROTECTED]>
To: "Django developers" <django-developers@googlegroups.com>
Sent: Wednesday, July 16, 2008 7:51 AM
Subject: Re: MS SQL pyodbc backend update to trunk



Thanks for that.  Turned out the problem I was having was to do with
slicing queries where no order_by was explicitly specified.  I've
uploaded a slightly modified patch to the same place.

On Jul 15, 6:19 pm, "Ramiro Morales" <[EMAIL PROTECTED]> wrote:
> On Tue, Jul 15, 2008 at 12:53 AM, bob84123 <[EMAIL PROTECTED]> wrote:
>
> > Yes I am.
>
> See patch attached to issue #2 I just 
> opened:http://code.google.com/p/django-pyodbc/issues/detail?id=2
>
> Regards,
>
> --
> Ramiro Morales

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[bob84123]

Hi all.  Sorry about the noise before regarding DateField/TimeField..
it wasn't an issue specific to this backend.  It's actually a django
problem - details and fix at http://code.djangoproject.com/ticket/7801

Thanks.

On Jul 16, 9:51 am, bob84123 <[EMAIL PROTECTED]> wrote:
> Thanks for that.  Turned out the problem I was having was to do with
> slicing queries where no order_by was explicitly specified.  I've
> uploaded a slightly modified patch to the same place.
>
> On Jul 15, 6:19 pm, "Ramiro Morales" <[EMAIL PROTECTED]> wrote:
>
> > On Tue, Jul 15, 2008 at 12:53 AM, bob84123 <[EMAIL PROTECTED]> wrote:
>
> > > Yes I am.
>
> > See patch attached to issue #2 I just 
> > opened:http://code.google.com/p/django-pyodbc/issues/detail?id=2
>
> > Regards,
>
> > --
> >  Ramiro Morales
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[bob84123]

Thanks for that.  Turned out the problem I was having was to do with
slicing queries where no order_by was explicitly specified.  I've
uploaded a slightly modified patch to the same place.

On Jul 15, 6:19 pm, "Ramiro Morales" <[EMAIL PROTECTED]> wrote:
> On Tue, Jul 15, 2008 at 12:53 AM, bob84123 <[EMAIL PROTECTED]> wrote:
>
> > Yes I am.
>
> See patch attached to issue #2 I just 
> opened:http://code.google.com/p/django-pyodbc/issues/detail?id=2
>
> Regards,
>
> --
>  Ramiro Morales
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[Ramiro Morales]

On Tue, Jul 15, 2008 at 12:53 AM, bob84123 <[EMAIL PROTECTED]> wrote:
>
> Yes I am.
>

See patch attached to issue #2 I just opened:
http://code.google.com/p/django-pyodbc/issues/detail?id=2

Regards,

-- 
 Ramiro Morales

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[bob84123]

Yes I am.

On Jul 15, 11:51 am, "Ramiro Morales" <[EMAIL PROTECTED]> wrote:
> On Mon, Jul 14, 2008 at 10:09 PM, bob84123 <[EMAIL PROTECTED]> wrote:
>
> > (It also seems order_by is recently broken, I'm going to have a look
> > into that now.)
>
> Are you using SQL server 2000?
>
> Regards,
>
> --
>  Ramiro Morales
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[Ramiro Morales]

On Mon, Jul 14, 2008 at 10:09 PM, bob84123 <[EMAIL PROTECTED]> wrote:

>
> (It also seems order_by is recently broken, I'm going to have a look
> into that now.)

Are you using SQL server 2000?

Regards,

-- 
 Ramiro Morales

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[bob84123]

I've fixed a couple of little issues in the backend:
- Settings.DATABASE_ODBC_DRIVER never got included in the connection
string so I put it in at the start;
- I changed the way parameters with default values are checked from
"if not settings.PARAM" to "if 'PARAM' not in dir(settings)", because
the original will crash if the parameter is not specified (and it
should be able to be unspecified since otherwise it really doesn't
have a default).

(It also seems order_by is recently broken, I'm going to have a look
into that now.)

Wei guangjing, could you include this patch in the backend?

Index: mssql/base.py
===
--- mssql/base.py   (revision 16)
+++ mssql/base.py   (working copy)
@@ -23,8 +23,11 @@
 import datetime
 import os

-if not settings.DATABASE_COLLATE:
-settings.DATABASE_COLLATE = 'Latin1_General_CI_AS'
+if 'DATABASE_COLLATE' not in dir(settings):
+  settings.DATABASE_COLLATE = 'Latin1_General_CI_AS'
+if 'DATABASE_ODBC_DRIVER' not in dir(settings):
+  settings.DATABASE_ODBC_DRIVER = "{Sql Server}"
+

 try:
 import pyodbc as Database
@@ -114,7 +117,7 @@
 else:
 host_str = settings.DATABASE_HOST

-odbc_string=""
+odbc_string="DRIVER=%s;" % settings.DATABASE_ODBC_DRIVER

 if hasattr(settings, "DATABASE_ODBC_DSN"):
 odbc_string += "DSN=%s;" % settings.DATABASE_ODBC_DSN



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[Leo Soto M.]

On Thu, Jul 10, 2008 at 5:25 AM, vcc <[EMAIL PROTECTED]> wrote:
>
> Thanks Malcolm tredinick, your greate Queryset-refactor work make
> write a backend very clear, thanks again!
> Yes, make a external database backend is better now, I clean the code
> and get it done! I just upload source code to django-pyodbc  code.google.com/p/django-pyodbc> code.google project, and I join this
> project to continue improve this backend, thanks, peter hausel!
>
> Now django core only need follow patch:
> Index: contrib/sessions/backends/db.py
> ===
> --- contrib/sessions/backends/db.py (revision 7671)
> +++ contrib/sessions/backends/db.py (working copy)
> @@ -13,9 +13,12 @@
>
> def load(self):
> try:
> +date_now = datetime.datetime.now()
> +if hasattr(date_now, 'microsecond'):
> +date_now = date_now.replace(microsecond=0)
> s = Session.objects.get(
> session_key = self.session_key,
> -expire_date__gt=datetime.datetime.now()
> +expire_date__gt=date_now
> )
> return self.decode(s.session_data)
> except (Session.DoesNotExist, SuspiciousOperation):

You may want to look at #7570 for this kind of situations. Basically,
I'm proposing to let backends take care of this problems (no
microseconds support in this case) to avoid polluting higher level
code.

> Index: test/utils.py
> ===
> --- test/utils.py   (revision 7671)
> +++ test/utils.py   (working copy)
> @@ -74,7 +74,10 @@
>  def _set_autocommit(connection):
> "Make sure a connection is in autocommit mode."
> if hasattr(connection.connection, "autocommit"):
> -connection.connection.autocommit(True)
> +if callable(connection.connection.autocommit):
> +connection.connection.autocommit(True)
> +else:
> +connection.connection.autocommit = True
> elif hasattr(connection.connection, "set_isolation_level"):
> connection.connection.set_isolation_level(0)

As this is also needed for JDBC backends, I filled #7751 for this one.

-- 
Leo Soto M.
http://blog.leosoto.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[vcc]

Thanks Malcolm tredinick, your greate Queryset-refactor work make
write a backend very clear, thanks again!
Yes, make a external database backend is better now, I clean the code
and get it done! I just upload source code to django-pyodbc  code.google project, and I join this
project to continue improve this backend, thanks, peter hausel!

Now django core only need follow patch:
Index: contrib/sessions/backends/db.py
===
--- contrib/sessions/backends/db.py (revision 7671)
+++ contrib/sessions/backends/db.py (working copy)
@@ -13,9 +13,12 @@

 def load(self):
 try:
+date_now = datetime.datetime.now()
+if hasattr(date_now, 'microsecond'):
+date_now = date_now.replace(microsecond=0)
 s = Session.objects.get(
 session_key = self.session_key,
-expire_date__gt=datetime.datetime.now()
+expire_date__gt=date_now
 )
 return self.decode(s.session_data)
 except (Session.DoesNotExist, SuspiciousOperation):

Index: test/utils.py
===
--- test/utils.py   (revision 7671)
+++ test/utils.py   (working copy)
@@ -74,7 +74,10 @@
 def _set_autocommit(connection):
 "Make sure a connection is in autocommit mode."
 if hasattr(connection.connection, "autocommit"):
-connection.connection.autocommit(True)
+if callable(connection.connection.autocommit):
+connection.connection.autocommit(True)
+else:
+connection.connection.autocommit = True
 elif hasattr(connection.connection, "set_isolation_level"):
 connection.connection.set_isolation_level(0)

For contrib/sessions/backends/db.py, we see that we have
supports_usecs feature in DatabaseFeatures, but in
.objects.get()'s query don't use it. Maybe use one function
like 'sql_date_now()' in db.connection is better to get right datetime
now?
code like this:
from djang.db import connection
s = Session.objects.get(
 session_key = self.session_key,
 expire_date__gt= connection.sql_date_now()
 )

Any idea?

Regards,

Wei guangjing

On 7月8日, 下午9时36分, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> On Tue, 2008-07-08 at 21:26 +0800, vcc wrote:
> > Thanks for Russ Magee, I see.
>
> > I just updated this backend, it's look like everything work fine for me. I 
> > have some projects working with this backend, I'll keep improve it. Please 
> > check the patch in the attachments.
>
> So the best thing to do with a patch like this, rather than keep posting
> it here (since, as Russell mentioned, it's not up for consideration of
> committing to core at the moment), is to make it a proper external
> database backend.
>
> That means something that people can install anywhere on their Python
> path and then just put it into the DATABASE_ENGINE setting. It's a bit
> hard to read the patch you've attached, but it seems like you've made
> some changes to core pieces of code (e.g. models/fields/__init__.py).
> You should try to work out is the feature that is really needed there
> and then open a ticket with a general solution that can be used by all
> backends (and addition to DatabaseOperations, for example). For example,
> think about whether it's really necessary to add a tablespace option to
> the Field class or whether that can be part of the field column name
> (e.g. as a tuple).
>
> Ultimately, there should be no changes required to core for this after a
> few tickets for missing features are opened and discussed. Remember that
> we our goal is not to support every single feature of every database
> backend, so don't go crazy with feature requests. We just support the
> common stuff. But that's the right way forwards from here.
>
> Regards,
> Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[bob84123]

Looks like this backend is going quite well.  For what it's worth, I
solved the other end of the datetime/date/time problem by patching
django.db.models.query.QuerySet.iterator.  I expect there's a better
solution than this but I thought I'd post this because it at least
works.

def iterator(self):
"""
An iterator over the results from applying this QuerySet to
the
database.
"""
fill_cache = self.query.select_related
if isinstance(fill_cache, dict):
requested = fill_cache
else:
requested = None
max_depth = self.query.max_depth
extra_select = self.query.extra_select.keys()
index_start = len(extra_select)
for row in self.query.results_iter():
if fill_cache:
obj, _ = get_cached_row(self.model, row, index_start,
max_depth, requested=requested)
else:
obj = self.model(*row[index_start:])

# Fix for databases that only have datetime fields
# (rather than date, time and datetime fields)
import django
import datetime
for f in self.model._meta.fields:
  val = obj.__getattribute__(f.name)
  if isinstance(f, django.db.models.fields.DateField) and
isinstance(val, datetime.datetime):
obj.__setattr__(f.name, datetime.date(val.year,
val.month, val.day))
  elif isinstance(f, django.db.models.fields.TimeField)
and isinstance(val, datetime.datetime):
obj.__setattr__(f.name, datetime.time(val.hour,
val.minute, val.second))
# End datetime/date/time fix

for i, k in enumerate(extra_select):
setattr(obj, k, row[i])
yield obj

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[Malcolm Tredinnick]

On Tue, 2008-07-08 at 21:26 +0800, vcc wrote:
> Thanks for Russ Magee, I see.
> 
> I just updated this backend, it's look like everything work fine for me. I 
> have some projects working with this backend, I'll keep improve it. Please 
> check the patch in the attachments.

So the best thing to do with a patch like this, rather than keep posting
it here (since, as Russell mentioned, it's not up for consideration of
committing to core at the moment), is to make it a proper external
database backend.

That means something that people can install anywhere on their Python
path and then just put it into the DATABASE_ENGINE setting. It's a bit
hard to read the patch you've attached, but it seems like you've made
some changes to core pieces of code (e.g. models/fields/__init__.py).
You should try to work out is the feature that is really needed there
and then open a ticket with a general solution that can be used by all
backends (and addition to DatabaseOperations, for example). For example,
think about whether it's really necessary to add a tablespace option to
the Field class or whether that can be part of the field column name 
(e.g. as a tuple).

Ultimately, there should be no changes required to core for this after a
few tickets for missing features are opened and discussed. Remember that
we our goal is not to support every single feature of every database
backend, so don't go crazy with feature requests. We just support the
common stuff. But that's the right way forwards from here.

Regards,
Malcolm



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[James Bennett]

On Tue, Jul 8, 2008 at 8:26 AM, vcc <[EMAIL PROTECTED]> wrote:
> I just updated this backend, it's look like everything work fine for me. I 
> have some projects working with this backend, I'll keep improve it. Please 
> check the patch in the attachments.

As Russ, and the many previous threads have pointed out: the thing to
do right now is:

1. Put the code up somewhere public -- somewhere that is *not*
djangoproject.com's ticket tracker or a mailing-list thread -- and
advertise it to people.
2. Respond to bug reports and suggestions from people who use the code.
3. Repeat.

Once there's an established history of good code and willingness to
maintain it, then -- and only then -- should it be considered for
inclusion in Django itself. Remember: database backends don't have to
be distributed as part of Django, and can live anywhere on the Python
path, so it's easy to do this.


-- 
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[bob84123]

I'd like to thank you for this work.  I tried it and it *mostly* works
on my Windows Server 2003 machine with MS SQL Server 2000, with a few
hacks:

- Changed the last line in creation.py to (this was just to make it
work in SQL 2000 *for me*, clearly it shouldn't stay like this)
   DATA_TYPES['TextField'] = 'nvarchar(1000) %(db_collation)s'

- Inserted this line before the Database.connect call in
DatabaseWrapper.cursor in base.py (because otherwise I was getting
errors about a driver not being specified)  Apparently this is the
default according to the comments at the top of this file but it
doesn't get put in the command.  Again this is probably a bug that
should be fixed properly rather than applying my hack.
   odbc_string = 'DRIVER={SQL Server};' + odbc_string

- Changed CursorWrapper.execute in base.py to:

def execute(self, sql, params=()):
import datetime

sql = self.format_sql(sql)
# convert from unicode to utf-8 for pyodbc
ps = []
for p in params:
  if isinstance(p, unicode):
ps.append(unicode(p).encode('utf-8'))
  elif isinstance(p, datetime.date):
ps.append(datetime.datetime(p.year, p.month, p.day))
  elif isinstance(p, datetime.time):
ps.append(datetime.datetime(1970, 1, 1, p.hour, p.minute,
p.second))
  else:
ps.append(p)

params = tuple(ps)
return self.cursor.execute(sql, params)

To handle the fact that SQL Server doesn't have specific date or time
types.

- I still haven't managed to get the reverse of this working; I always
get datetime objects out of the database even for date fields and time
fields.  Does anyone see any way to fix this?  I had a play around
with get_cached_row in django.db.models.query.py but didn't really
have any luck.

I'd really like to see a SQL Server backend stay reasonably up to date
with django and I'm prepared to help with any work needed to achieve
this.

Thanks

On Jul 2, 11:48 pm, "vcc" <[EMAIL PROTECTED]> wrote:
> I port MS SQL pyodbc backend (django-pyodbc) to newforms-admin r7671, tested 
> on ubuntu 8.04 and with SQL Server 2005.
> I found sql server need convert  boolean value to integer (BooleanField), so 
> need add a new feature like 'needs_bool_to_integer' to DatabaseFeatures, 
> deafult to False.
>
> Since this MS SQL backend work both on Linux and Windows, so we may consider 
> have MS SQL backend back in Django internal.
>
> Please see patch in attachment.
>
> Best regards,
>
> Wei Guangjing
> _
>
>  django-pyodbc-r7671.patch
> 39KDownload
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MS SQL pyodbc backend update to trunk

[Russell Keith-Magee]

On Wed, Jul 2, 2008 at 9:48 PM, vcc <[EMAIL PROTECTED]> wrote:
> I port MS SQL pyodbc backend (django-pyodbc) to newforms-admin r7671, tested 
> on ubuntu 8.04 and with SQL Server 2005.
> I found sql server need convert  boolean value to integer (BooleanField), so 
> need add a new feature like 'needs_bool_to_integer' to DatabaseFeatures, 
> deafult to False.
>
> Since this MS SQL backend work both on Linux and Windows, so we may consider 
> have MS SQL backend back in Django internal.
>
> Please see patch in attachment.

Please see the many, many threads about MS-SQL support which discuss
at length Django's policy on adding new database backends.

We have repeatedly said that we will only consider a database backend
for inclusion in trunk when there is a demonstrated long term
commitment to supporting the patch.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

MS SQL pyodbc backend update to trunk

[vcc]

I port MS SQL pyodbc backend (django-pyodbc) to newforms-admin r7671, tested on 
ubuntu 8.04 and with SQL Server 2005. 
I found sql server need convert  boolean value to integer (BooleanField), so 
need add a new feature like 'needs_bool_to_integer' to DatabaseFeatures, 
deafult to False.

Since this MS SQL backend work both on Linux and Windows, so we may consider 
have MS SQL backend back in Django internal.

Please see patch in attachment.

Best regards,

Wei Guangjing
_

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



django-pyodbc-r7671.patch
Description: Binary data