Re: GeoDjango and Postgresql

[Justin Bronn]

On Nov 9, 6:25 am, Paul McMillan  wrote:
> I opened #14527 a while ago, but it hasn't gotten much 
> attention.http://code.djangoproject.com/ticket/14527
>
> It would fix the docs which claim that PostGIS doesn't support PostgreSQL 8.4.
>
> Can I get some GeoDjango users to chime in on this matter? I'd like to
> see this closed one way or the other.
>
> -Paul

As I told you in #django-dev that documentation is specific to the
GeoDjango windows installer _only_ -- hence why it's in the "Windows
XP" section:

http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#windows-xp

GeoDjango supports PostgreSQL 8.1 and above, as indicated at the top
of the documentation:

http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#spatial-database

The ticket will be addressed when the windows installer is either
updated (#14133) or discontinued due to lack of resources.

-Justin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Possible thread-safety regression for sitemaps in r13980

[Justin Bronn]

After r13980, GIS sitemaps were broken, and when attempting to fix I
had questions about the underlying implementation of the changeset:

http://code.djangoproject.com/changeset/13980/django/trunk/django/contrib/sitemaps/views.py

In the loop iterating over the `sitemaps` argument (L12), the `site`
value is a global Sitemap (or subclass thereof) instance typically
defined in a user module.  While adding the missing `site.request =
request` assignment in the GIS sitemaps code enables tests to pass --
this solution appears thread unsafe because we are assigning, on a per-
request basis to the sitemap view, the `request` attribute to a module-
level global.  What if two different search engines request the
sitemap index at the same time?  Each would be clobbering over the
`request` attribute for the other, producing inconsistent sitemaps.

First, I'd like more eyes to confirm/deny my interpretation here.
Second, if this is confirmed, how should we solve it -- the situation
is compounded by the fact this changeset fixes 7 tickets.  Luke, is
there anyway to decouple this from fixes for the other tickets?

-Justin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: importing fixtures to postgres fails to set sequences correctly

[Justin Bronn]

> > there seems to be a problem with django postgres backend, when
> > importing data from fixtures. Data are imported correctly, but the
> > sequences for primary keys are set incorrecly on models that have
> > generic.GenericRelation field. See example:

I have similar code, and was griping in #django-dev yesterday about
the regression.  I came up with an almost identical fix (using
`m2m_db_table`), so I've accepted the ticket.

> Of related interest would be to check if this issue existed previously
> (i.e., before the recent PostgreSQL sequence changes); if the issue
> didn't exist in Django 1.2 final, then checking the diff between the
> old and new implementations may also shed some light on potential
> solutions.

It's definitely a side-effect of r13328 -- I have production code that
works fine on 1.2.X, but crashes on trunk without the patch when
restoring fixtures.  My only question is whether there was an original
reason that `db_table` was used or if it was simply a copy & paste
oversight from the previous loop.

-Justin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Thanks for all the hard work

[Justin Bronn]

> Thank you very much for all the hard work that you've put in to
> Django.  It makes a big difference to web developers to have a
> framework that is so well made, so well thought out and so well
> documented.  It's a pleasure working with Django, and I'm looking
> forward to working with it for the foreseeable future.

Much appreciated.  I'm, for one, happy to hear a user's gratitude
rather than another pony request.  This is food for the core
developer's soul, and thank _you_ for motivating us.

Best Regards,
-Justin

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Oracle/GIS Testers Needed

[Justin Bronn]

> For GIS I'm fairly certain that's broken, however I'm not quite setup
> for running the GIS tests yet, therefore if you're set up for it and
> can run the tests please just reply with your tracebacks here
> (preferably with them on a pastebin like paste.pocoo.org though :))
> and I'll try to sort it out.

Yes, multi-db is seriously broken with GIS on all backends (it'd be
pointless to list all the errors here).  Basically, the massive shift
of `Query` internals to `SQLCompiler` is the root of the problem --
there needs to be a corresponding `GeoSQLCompiler` with `GeoQuerySet`
and `GeoQuery` wired to use it accordingly.  I'll try to look at it in
more detail this weekend and see if I can sketch out a better idea of
what needs to be done, but so far it's looking "non-trivial."

-Justin

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=.

Re: new Open Layers map widget for use in/out of admin

[Justin Bronn]

Charlie,

> I would be quite appreciative of any feedback or testing.  My dream,
> like anyone else's, is to get it into trunk eventually. :)  That will
> require more work, such as whatever is necessary to hook the widget into
> forms.ModelForm subclasses, and better documentation and tests.  If
> anyone has specific suggestions for how to get this trunk-worthy, I
> would love to hear them.  In the mean time, it's implemented as a
> portable app called "olwidget".

My bad for not seeing this a month ago -- I wish I had known sooner
[1] as way the map widget is done in the admin is very hackish and I'm
planning on generalizing it anyway for 1.2 to fix some fundamental
defects (e.g., #9806).  I'll review the code and see if I can work up
a patch putting this functionality in `django.contrib.gis.forms`.

> Under the hood, it uses a standalone javascript WKT <-> OL map
> translator that strips some of the magic of the current
> contrib.gis.admin implementation, and a django app with some pretty
> light-weight django classes to integrate with forms and admin.
>

While I haven't reviewed in detail, this sounds like a much better
approach than the haphazard way it's done currently.  I look forward
to reviewing, thanks for writing this.

Regards,
-Justin

[1] There's also the GeoDjango mailing list: 
http://groups.google.com/group/geodjango.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Dallas 1.1 sprint - dates?

[Justin Bronn]

On Apr 3, 9:50 am, Jeremy Dunck  wrote:
> Gary, Justin?
>

The 17th option is better for me, but at this point it's still not
looking like I'll be able to make it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GeometryField doesn't honor the required attribute

[Justin Bronn]

> I hope I made myself clearer this time. Opinions please? If this
> should be fixed, and I seriously think it must, I'll open a ticket and
> attach a patch.

I see what you're talking about now, and I agree it's a bug.  Please
open up a ticket and we'll consider the design decisions there.
Thanks for the clarification.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GeometryField doesn't honor the required attribute

[Justin Bronn]

On Mar 25, 12:20 pm, Fidel Ramos  wrote:
> My proposed solution would be to call Field's clean() instead of
> checking self.null.

Isn't the checking of `self.null` that you've identified as the
problem already in the `GeometryField.clean` method?  Are you
proposing a recursive call?

I agree that GeometryField is ripe to be improved -- but I'm unclear
on exactly what is the problem and the proposed solution here.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Reminder: Django 1.1 beta this week means feature freeze

[Justin Bronn]

> GIS is a bit of a special case; Justin Bronn is the maintainer there,
> and he and the rest of the GeoDjango contributors develop somewhat
> independently of the rest of Django.
>
> That said, they don't get any special exemptions in terms of timeline,
> so GeoDjango feature freeze is this weekend, too.
>
> /me looks meaningfully at Justin.

All of my features have been announced:

http://code.djangoproject.com/wiki/GeoDjango1.1

Realistically some of these features will not be in trunk by the end
of the weekend.  Fortunately the majority of them are already complete
in the GeoDjango 1.1 development mercurial, but are waiting more
review/testing, e.g., Aryeh won't be able to review the latest #9877
patch (a non-trivial enhancement) until the end of Sunday.   I'm going
to commit them anyway between now and PyCon, unless I'm forced to
sacrifice them at the altar of deadline technicalities.  I'd really
hate to tell those folks waiting for SpatiaLite support they'll have
to wait till 1.2 because I didn't prematurely commit the tickets by
midnight on Sunday.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Filtering with respect to a related objects field

[Justin Bronn]

> Is there a way that default managers can be taken into
> consideration when filtering through related fields?  I don't know
> exactly how this is supposed to work, is this the desired behavior?
>

It's not a bug, it's a feature :) Specifically, it's an artifact from
the 'fix' in #7666 -- fortunately, there's a way simple way to
override, but it'll require a simple sublcass of CurrentSiteManager:

class MySiteManager(CurrentSiteManager):
use_for_related_fields = True

That should be it.

Regards,
-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #7210 - F() Query Expressions

[Justin Bronn]

> The main issue I flagged to Russell in some code review notes was
> whether the code needed to be able to handle get_placeholder() output,
> instead of hard-coding "%s" everywhere for parameters. If there were
> particular column comparisons that required a more massaged approach to
> getting the data out of parameters in some cases and if we had to feed
> the placeholder functions to the Evaluator or something similar.
>
> Maybe we don't -- at least, not right now and not for GIS -- which is
> fine, too.

Good thing you made me second guess myself.  I thought that only the
reference to column was needed, but that was the simple case and I
neglected the situation where one geo column could be in a different
projection than the one referred to in the F expression.  Thus,
transformation SQL could actually needed (what `get_placeholder` does,
but on INSERT and UPDATE), and I've since updated to a newer patch and
added some tests.  That was a nice break, now back to studying...

-Justin

> Regards,
> Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #7210 - F() Query Expressions

[Justin Bronn]

> 2) F() expressions have not been updated to work with GIS fields.
> However, based on some initial discussions with Justin, it should be
> possible to make these changes without affecting any public API. If
> anyone with access and experience with contrib.GIS cares to help out
> on this front, assistance would be gratefully accepted; otherwise,
> we'll just have to wait until Justin surfaces from his Bar exams.

Currently, nothing is broken and all tests pass -- only that a
TypeError is raised when one tries to use F() on a geographic field.
I've got a preliminary patch that fixes F() expressions for GIS
fields:

http://code.djangoproject.com/ticket/10159

Hopefully some people can bang on it and create improvements for the
patch, including maybe a test case or two.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #3566 Aggregations: Call for testing

[Justin Bronn]

> Refactoring Query.convert_values() into the backend operations would
> remove some code duplication, and would also serve to keep all the
> data coercion/typecasting code in a common location rather than
> spreading some of it into the Query class. However, there are two
> things that prevented me from doing this refactor.
> ...
> Secondly, contrib.GIS has a convert_values() function that invokes
> super(GeoQuery, self).convert_values() in the case of an Oracle
> spatial backend. Justin - is there any particular reason to call the
> GeoQuery base class specifically, rather than invoking
> self.connection.ops.convert_values()?

Yes, because your subset implementation of `convert_values` does not
convert LOB values (which GeoQuery needs).  Specifically, geometries
are wrapped in a function that returns a CLOB, and the WKT string is
obtained from a `.read()` call on the CLOB.

I'm -1 on moving conversion operations from the Query class to the
backend.  However, I'd be +1 if Query.convert_values is kept  -- even
if it's just a wrapper around self.connection.ops.convert_values.  If
the conversion functions are _only_ available at the backend level,
then it would mean I would have to create distinct spatial database
backends every time I want to subclass the convert_values() method.  I
don't want to have to create distinct database backends just so I can
override a single method (DATABASE_ENGINE='gis_postgresql_psycopg2'
looks pretty awful to me).

> Is GeoQuery likely to have a base class other than OracleQuery in the
> Oracle case?

No; unless, of course, the backend requires a custom Query class like
Oracle does -- which would mean any future backends without LIMIT/
OFFSET and spatial support would also subclass from the backend's
Query class.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Aggregation branch fails with GIS models.

[Justin Bronn]

> I'll be looking into this, this week.
> Good news is: So far so good (But I haven't tried to mix geo aggregates with
> new aggregates yet).

OK, in the GeoDjango 1.1 mercurial [1], I've merged in Russell's
aggregation branch and modified the aggregate geographic methods to
internally use the new aggregation API.  Thus, there's now a
corresponding geographic aggregates module that contains Extent,
MakeLine, and Union.

I had to patch the aggregation branch a bit to make it amenable to
subclassing.  Specifically, I made the `aggregates` module an
attribute of Query and `normalize` a Query method so I could
substitute in the geographic aggregates module and be able to convert
geographic aggregate values.  These patches are attached to #3566
(queryset_modular_aggregates.diff).

Regards,
-Justin

[1] http://geodjango.org/hg/gis-1.1/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Aggregation branch fails with GIS models.

[Justin Bronn]

Ariel,

> Is it a gis issue or an aggregation one?

As I indicated in private correspondence, this was a matter of
updating GeoQuery.get_columns to reflect Russell's aggregation
changes.  I've attached to #3566 a patch for Russell's git repo that
fixes the problem (geoquery_aggregation_fix.diff).  For posterity,
I'll repeat my previous message:

While the gis tests pass, I'm not sure of the consequences when mixing
aggregate functionality w/geo functionality -- if you could play
around and note what works and what doesn't it'd be great.

I'm sure lots of GeoQuerySet methods may be simplified through the use
of Russell's API (obvious examples include my own geo-aggregates).  If
you're ambitious, take a look into this if you can; otherwise I'll get
around to deeper inspection of the source the some day.

Best Regards,
-Justin

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Dropping Python 2.3 compatibility for Django 1.1

[Justin Bronn]

+1.  Eliminates a ton of compatibility code, e.g., no more carrying
around a three thousand line Decimal implementation.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: ANN: Django 1.0.1 released

[Justin Bronn]

> This:http://code.djangoproject.com/ticket/9613seems to be of the
> same vein.  I haven't confirmed it however.

Yeah I confirmed it.  I went through the contrib apps one-by-one to
see if anything else was missing.  In addition to auth's missing
files, the templates for formtools are omitted.

I've got a patch on #9613; what other places should we be looking for
missing dirs?

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: ANN: Django 1.0.1 released

[Justin Bronn]

> I didn't know MANIFEST.in needed data directories listedI know from bug
> fixing in setup.py that it generates a list of data files to pass into setup
> -- I wonder what that is used for?  I guess it's for install but not
> distribution tarfile building?

Neither did I -- and those files _are_ in the `data_files` list used
by setup.py :)  From what I gather, if the files aren't in MANIFEST.in
and aren't Python source files then they'll be pruned. [1]

> I don't think a 2nd tarball with the 1.0.1 name is an option -- the whole
> point of having a label like 1.0.1 is so you know what you've got.  Two
> different 1.0.1 files with different contents defeats the purpose of that.

Agreed.  My only worry was that there would be a significant period of
time between 1.0.1 and 1.0.2 (like the time between 1.0 and 1.0.1).
If the timeline is for next week (like Jacob suggested) rather than
two months from now, it's a moot point and I'm +1 for fixing this
issue in a 1.0.2 release.

-Justin

[1] http://www.python.org/doc/2.5.2/dist/manifest.html
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: ANN: Django 1.0.1 released

[Justin Bronn]

> Thanks, I eventually figured this out on my own -- it was the
> problem.  Fixed in r9473 and r9474.
>

I wish I had known about MANIFEST.in sooner, is there any way we re-
tag and re-release 1.0.1 with the missing files? Or am I forced to
instruct GeoDjango users to avoid easy_install until 1.0.2?  If the
latter is the only option, I propose we set up a 1.0.2 timeline ASAP.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: ANN: Django 1.0.1 released

[Justin Bronn]

> The reason is that MANIFEST.in doesn't tell setuptools to include those
> directories in the distribution.

Thanks, I eventually figured this out on my own -- it was the
problem.  Fixed in r9473 and r9474.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: ANN: Django 1.0.1 released

[Justin Bronn]

> (and obviously I meant 'setup.py sdist')
>

I don't know how setup.py bundles a tarball, but it's doing it wrong
-- GeoDjango is broken in the 1.0.1 release tarball.  In particular,
at least the following directories were completely _omitted_ from this
release:

http://code.djangoproject.com/browser/django/branches/releases/1.0.X/django/contrib/gis/templates
http://code.djangoproject.com/browser/django/branches/releases/1.0.X/django/contrib/gis/tests/geoapp/sql

While the missing test data is OK, not having the GIS template
directory breaks a lot of functionality, including the geographic
admin and geo sitemaps.   I ran `python setup.py sdist` and watched it
skip over the directories myself -- those with superior setuptools
skills can hopefully inform me as to how this can happen.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Google maps subclass of GeoModelAdmin

[Justin Bronn]

> I suppose I can log both a ticket and place it on djangosnippets.org.
> Thanks

This has come up a few times in #geodjango, and I've already written
an example on how to use Google's base layer in the admin.  I
distilled it to a few snippets:

http://www.djangosnippets.org/snippets/1144/

It's actually a simple example of how to customize the geographic
admin; I'm not sure about including it SVN.

> > [1]http://code.google.com/support/bin/answer.py?answer=55141=10945

Thanks for finding this informative FAQ entry, I had not seen it
before.

Regards,
-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Oracle testing wanted

[Justin Bronn]

> Oracle is free for non production use.

Sort of -- there are some limitations, e.g., you aren't allowed to
develop GPL-licensed using their 'free' license.

> but you did want to know why none of the core devs
> have Oracle installed :)

I have the dev version installed on a Windows VM, and the official
Oracle maintainers (Ian Kelly and Matt Boersma) have their own
commercial installation.  We drink the Oracle kool-aid so the other
devs don't have to.

Regards,
-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Pickling QuerySets

[Justin Bronn]

> Am I missing where this is discussed, or is it an oversight, or is there
> a reason not to mention it?  I'll patch the docs if you'd like...

Ned,

Malcolm documented this behavior after qs-rf merged.   However, this
section may have slipped through the cracks with the documentation
refactor.  You can see what was written here:

http://code.djangoproject.com/browser/django/trunk/docs/db-api.txt?rev=8478#L398

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Oracle IntregrityError and get_or_create test case

[Justin Bronn]

Matt,

FYI, r8471 fixed my problems and all Oracle GeoDjango tests pass
again.  Thanks.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: You've broken Oracle

[Justin Bronn]

> Item.objects.dates('created', 'day')[0]
> DatabaseError: ORA-00923: FROM keyword not found where expected

That's the exact error that's giving me problems -- I think it's one
of the same issue as I'm having.  It's because of the `col.rsplit('.',
1)[1]` logic in Oracle's `as_sql` when doing offsets.  In other words
when the column is wrapped in a function (like the datetime SQL is)
then there will be a parenthesis after the quote still in the column
name.

> Did GeoDjango break in r8426 then, as Oracle in general did?  That's
> where extra_select changed.  I can see how our new code in r8445
> breaks on your function column.

Yes, GeoDjango broke in 8426, but a simple fix was committed in 8431
that fixed it PostGIS and MySQL -- basically I just needed to make
compatible for the data structure changes.

I came to the same conclusion on the pickling problems being caused by
the fact that OracleQuery doesn't exist at the module level, but
rather is returned from a function call dynamically.

-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: You've broken Oracle

[Justin Bronn]

> Rather than flag "row_number()" as an extra_select parameter (and then
> try to clean up after it later), Oracle now just uses the default
> set_limits and clear_limits methods and does all extra query munging
> in its as_sql() method.  And instead of doing an outer SELECT *, we
> SELECT specific columns (or aliases) by name and ignore our
> row_number() column, which is only used in the WHERE clause.
>
> This seems to fix most of the failures.  I'll check it in soon when
> I'm sure it's not causing any new problems.
>

This approach breaks slicing for Oracle in GeoDjango.  Specifically,
getting rid of the `SELECT *` and manually specifying the columns has
introduced side effects.  The following example will show the problems
-- here is a simple geographic model:

class City(models.Model):
name = models.CharField(max_length=30)
point = models.PointField()
objects = models.GeoManager()

In 8425, here's the SQL that `City.objects.all()[0]` would produce:

SELECT * FROM (SELECT (ROW_NUMBER() OVER (ORDER BY "GEO_CITY"."ID" ))
AS "RN", "GEO_CITY"."ID", "GEO_CITY"."NAME",
SDO_UTIL.TO_WKTGEOMETRY("GEO_CITY"."POINT") FROM "GEO_CITY") WHERE rn
> 0 AND rn <= 1

As you can see, I need to wrap the geometry column with a function
call that converts the column into an acceptable format.  With the
changes in r8445, this is the broken SQL that is produced:

SELECT "ID", "NAME", "POINT") FROM (SELECT ROW_NUMBER() OVER (ORDER BY
"GEO_CITY"."ID") rn, "GEO_CITY"."ID", "GEO_CITY"."NAME",
SDO_UTIL.TO_WKTGEOMETRY("GEO_CITY"."POINT") FROM "GEO_CITY") WHERE rn
> 0 AND rn <= 1

There are a few problems here.  The first is that the `col.rsplit('.',
1)[1]` logic is too brittle to handle a column wrapped in a stored
procedure.  But the greater issue is that to fix this in GeoQuery is
that I need to make `get_columns` aware if it's being called for an
outer select or the inner select, i.e., I'll have to set an alias on
the inner select while not putting any function wrapping on the outer
select.  I currently don't know how to do this other than adding extra
keyword parameters in my overridden implementation that may be used by
an overridden `as_sql` used only for Oracle just for this purpose.
Needless to say, this isn't trivial -- unless there's other ideas on a
different approach I may have to drop Oracle support for GeoDjango in
1.0.  I'm hesitant to be rewiring internals this close to the deadline
instead of focusing on documentation.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #7666: Default managers should not restrict access to single related objects

[Justin Bronn]

> I forgot to note before, I've added a patch to #7666 which uses James'
> suggestion of a pristine QuerySet.

Using a "pristine" QuerySet is a bad assumption to make, especially
for those who are using custom managers to properly generate SQL for
non-standard columns, e.g., geometries.  The patch committed in r8011
has the unintended side-effect of breaking related geographic queries
for MySQL and Oracle because it bypasses the default manager
(GeoManager) and thus does not know how to properly select geometry
columns.

One possible solution would be look for the presence of an attribute
on the default manager so that it used instead of QuerySet.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GIS: support for Points (markers) in Google Maps

[Justin Bronn]

> I could clean this up as a patch to the GIS branch, but there seems to
> be some confusion as to whether such functionality should be in the
> core product.
>
> What is the right way forward here?
>

File a ticket with your patch -- I'm +1 because it doesn't make sense
to support GPolygon and GPolyline but not GMarker (I haven't had time
to implement myself yet).

For the record, I think the confusion is localized to the extent we
should have an OpenLayers API; it has so much functionality it may be
counter-productive to try and box users into an extremely small subset
of its features.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django releases

[Justin Bronn]

> That aside, now that QSRF is  
> getting a real fleshing-out and all these reports are trickling in, I  
> think it would be a bad idea to stamp a version right now until either  
> someone can step up and fill Malcolm's shoes as a queryset maintainer,  
> or he becomes available once again from real life.

There's no way I can fill Malcolm's shoes, but I've been intimately
working with the qsrf code since November so I know it pretty well.
I'll be glad to take a look at some of these tickets and see what I
can do.  Feel free to contact me off-list and/or on IRC if you wanna
poke my brain about this stuff.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django releases

[Justin Bronn]

For the record, I'm -1 on releasing 1.0 without NFA.

As I've discussed with Jacob and Adrian, GeoDjango will be merged with
trunk at some point in the future.  In my opinion, there's nothing
preventing it from being included in the 1.0 release (I'm currently
addressing the biggest hurdle, documentation, by porting over the wiki
contents to ReST [1]).  One of my best features, the geographic admin,
depends on NFA [2]; thus, I'm willing to devote significant
development time to getting NFA merged.

A Django 1.0 with newforms-admin + gis is a compelling release -- it
would put all those serving haterade in their proper place.

[1] http://geodjango.org/hg/gis-docs
[2] http://geodjango.org/hg/gis-newforms
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Interaction between [Multiple]ModelChoiceField and its widget

[Justin Bronn]

> Also, when the number of choices is big, none of the provided widgets
> are very usable, so I have for instance created a TreeSelect widget
> which groups choices into categories (columns in the table) and
> displays them in a tree whose branches are collapsible (see an example
> at [1]).  But in order to do that, the widget needs to know its
> field's queryset, not just its choices.
>
> But as far as I can see, there is no builtin way, when you change the
> queryset property of a [Multiple]ModelChoiceField, for this change to
> propagate automatically to its widget.  

I totally agree.  A common use case I've encountered when using the
admin (specifically, I'm using newforms-admin) is the ability to
override the Model[Multiple]Choice field to only display the related
objects associated with a given model in the admin.  Otherwise, the
browser buckles under the pressure of having to populate a 
element with thousands of possibilities.  I believe such a
customization hook would benefit newforms-admin, and as customization
hooks are currently being discussed in a separate thread [1] perhaps
it should be mentioned again there.

Now before someone comes saying "use raw_id_fields", they are
insufficient because they are, well, too "raw" -- forcing a client to
input integer ids is not my idea of user friendly. I had to come up
with my own form fields, widgets, and a `ModelAdmin` subclass to
accomplish the customization of the queryset displayed to the user
[2].  Much of the code would be unnecessary if it was possible to
easily override the queryset.

[1] Simon Willison, "newforms-admin customisation hook suggestions",
http://groups.google.com/group/django-developers/browse_thread/thread/53eaa25074ff4369
[2] http://geodjango.org/hg/limited_related/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: PostgreSQL 8.3 and casting

[Justin Bronn]

> I see that the Oracle backend already includes
> OracleQueryconvert_values which does conversion into the backend based
> on the the passed value and the field type.
>
> Perhaps we need to add that for PostgreSQL now?  It's pretty onerous
> to do the casting at the app level.
>

The `convert_values` routine is only used on values coming _from_ the
database (via `resolve_columns`), and not on values going _to_ the
database.  I'm with Malcolm on not supporting this feature of
automatically casting to string because the database is doing the
"right thing" here. [1]

However, I believe there are still parts of Django that depend on this
behavior -- for instance  adding a search field to a related field in
the admin is one of these situations (I need to provide specifics, but
don't have it on hand at the moment).

[1] See 
http://groups.google.com/group/django-developers/browse_frm/thread/371e8743fffd3d0c.
See also: http://code.djangoproject.com/ticket/6523.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Maybe DEBUG=True should only record the last N SQL queries?

[Justin Bronn]

> What do people think of having the debug SQL log limited to only
> storing the last N queries, with N set to something sensible like 200
> by default? This behavior could be controlled by a setting so if
> people want to log everything they can:
>
> DEBUG_SQL_LOG_LIMIT = 200 # Set to 0 to disable logging, set to None
> for unlimited logging
>

+1.

Except that I'd still like `DEBUG=False` to disable logging without
setting the log limit to 0, but I don't think you were proposing this
behavior.

Even when debugging, I rarely have to go back more than a few queries
anyway -- I've been bitten by this more times than I can count ("Why
is this sucking RAM? Crap, forgot to turn DEBUG off again...").

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Changeset 7427

[Justin Bronn]

> I bet Oracle is broken in qs-rf.

Yeah, Malcolm opened up #7036 two days ago in response to errors
reported by Ian Kelly.  I've attached a patch.

The problem had to do with `resolve_columns` not taking into account
that extra select SQL is in a different position due to changes in
#7340.  The extra select SQL is needed for the hack that simulates
LIMIT and OFFSET under Oracle.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Public spec is needed for writing ORM adapters

[Justin Bronn]

Alex, Antonio, & Leon:

IBM should understand our frustration when a spec exists (in
`django.db.backends`) but, because it is in source code form, your
particular IBM department cannot look at it.  Thus, IBM is attempting
to shift the burden onto our community to produce onerous technical
documentation to _your_ requirements.  This is the source of friction
you are experiencing on this list.

Moreover, there are actually several legitimate reasons for the lack
of a spec: (i) the database internals are being refactored, and (ii)
each RDBMS is different and has its own quirks that are dealt with in
its respective backend.  For example, Oracle doesn't provide LIKE and
OFFSET capabilities, so those have to be worked around with a custom
QuerySet.  Producing a heavy-weight public-domain spec will bind us to
a model that won't allow us to adapt to the different variations of
SQL dialects -- and I'm sure that DB2 and Informix will have their
fair share of quirks.

How was this problem solved with other OSS projects that IBM worked
with -- e.g, was this process been recorded for other OSS projects
(like Django) to examine? Was this on the SQLAlchemy mailing lists, in
private discussions with their developers, and/or all of the above?
Was this somehow less of an issue becaus SQLAlchemy and its
documentation are provided under the MIT license?

Due to the scarcity of the core developer's time don't expect this
documentation to appear absent some incentive.  My last experience
trying to work with IBM's commercial databases was very painful[1],
and I personally would not invest myself again unless I was paid to.
If IBM is truly dedicated to having a Django database backend consider
transferring this to a department that can look at OSS code, or be
willing to hire someone to produce the documentation to your own
standards.  Other risk-reduction measures, such as indemnification
insurance, are also available for your corporate customers.

Regards,
-Justin

[1] "Informix", django-users, 
http://groups.google.com/group/django-users/browse_thread/thread/3f0f772e84befaa1
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: New field lookups

[Justin Bronn]

Julian,

The django-developers mailing list for discussing the development of
Django itself; django-users is the appropriate forum for this type of
question.

There are no lookups for hour, minute, second because this
functionality may be accomplished by utilizing a `range` lookup with
Python `datetime` objects (which may be specified down to the hour,
minute, and second).

See: http://www.djangoproject.com/documentation/db-api/#range

-Justin

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Future-proofing django.template (and elsewhere) against circular imports

[Justin Bronn]

> I'm *way* off base at the moment (and stuck gradually walking towards
> HEAD, fixing my patch in increments; this is where git definitely has an
> advantage over hg, but I'm using the latter).

I've found hg-svn to be particularly useful in helping me pull changes
from the various SVN Django branches -- if you aren't already using
it, it's definitely worth a try.

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: deprecated Model.__init__(*args)

[Justin Bronn]

I'm +1 for deprecating positional arguments from __init__().

> Relying on the order of fields in the model definition is asking for a
> heaping load of fail. Hence my desire to see it go away.

While I agree this argument applies for having positional args in
__init__(), those instantiating using a fromargs/fromtuple method (as
Ivan suggested) are more likely to be cognizant of the "heaping load
of fail" implications.

Are there other reasons that such a class method should be excluded?
Advanced users such as Ivan and David would benefit from its utility
and have given some valid use cases.  If it really is as simple as
Ivan has posted and doesn't cause implementation/maintenance headaches
in other portions of the code, then why not?

-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Extending Django in a production-oriented way. Is it possible?

[Justin Bronn]

> 2. '%s' placeholders are hardcoded into Django. Existing solutions
> (like in sqlite backend) introduce the unneeded overhead in each
> database query. There are many other small things hardcoded into
> Django with Postgres in mind. Other backends has workarounds with
> added overhead and it makes them weaker than de facto default postgres
> backend.

In the GeoDjango branch, we had the need to override the '%s'
placeholder for our Oracle and MySQL spatial backends.  I accomplished
this by adding a `get_placeholder` routine to the base Field class,
which is then overloaded as needed. [Malcolm: this is one the features
I'll specify in response to your inquiry].

See: 
http://code.djangoproject.com/browser/django/branches/gis/django/db/models/base.py
(lines 233 and 245).

> 3. I really need to look at queryset-refactor and use this branch as
> the base because it may solve the issues with custom table creation/
> deletion

Yes you really should; Malcolm has done a superb job with this
branch.  In my personal experience, my private merges with queryset-
refactor have immensely reduced the amount of required custom code.
Moreover, it is _much_ easier to subclass ORM components for specific
purposes.

In addition, as Malcolm noted, we also had to deal with custom SQL at
table creation time.  Thus, I believe your stated problems are hardly
intractable -- even without queryset-refactor -- as GeoDjango has been
doing many of your requested customizations for several months now.

Best Regards,
-Justin

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Bug #1443 / Dates prior to 1900 Potential Patch

[Justin Bronn]

I added your patch to the ticket.  It's much easier to submit
attachments, etc. when you register:

http://www.djangoproject.com/accounts/register/

Regards,
-Justin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GeoDjango--no "django_session_id_seq" with pgsql?

[Justin Bronn]

Matt,

First, a question of this sort is better suited to django-users, as
django-dev django-dev is for discussion of the development of Django
itself.

Second, from what I can tell db_dump.py is not part of the official
trunk -- thus I'm not familiar with it.  My priority is to have
compatibility with the features already within Django before I worry
about third-party scripts.

Finally, `django_session_id_seq` tells me that the error is related to
the sessions framework, django.contrib.sessions (which is included in
INSTALLED_APPS by default).  In the code for the Session model, the
primary key is manually specified for the `session_key` field --
therefore no AutoField is created for the Session model.  My guess is
the bug lies with the script's handling of models that use primary
keys outside the implicit AutoField -- most likely this is not related
to GeoDjango.

-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GeoDjango: ST_Transform broken (no SRID on first arg)?

[Justin Bronn]

Matt,

I'm having difficulty reproducing this error, as the objects I create
from the admin save ok.

Can you provide the code for your model and/or a more verbose
traceback?  Are you using psycopg2?  Have you modified any parts of
the code, especially any GEOS components?

I ask because the GEOS geometries include the the __conform__() and
getquoted() routines which are used to provide the '%s' expansions for
psycopg2.  For example, "POINT(5 23)" would become
"ST_GeomFromText('POINT(5 23)', 4326)" -- and the call you posted is
missing any ST_GeomFromText() references.  While we used to use the
PostGIS canonical EWKT (e.g., "SRID=4326;POINT(5 23)"), I decided to
use the ST_GeomFromText() function because it should be agnostic to
spatial database backends, rather than exclusive to PostGIS.

-Justin



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GeoDjango installation instructions correction?

[Justin Bronn]

> If I am understanding things correctly, GeoDjango uses ST_* functions in
> PostgreSQL, but these functions weren't added until 1.2.1.

I started developing GeoDjango when PostGIS 1.2.1 was the latest.  I
believe PostGIS began adding the ST_* routines in 1.2.1, and in 1.3.1
they deprecated any non-ST_* routines (e.g., in versions after 1.3.1
Transform(), GeomFromText(), etc. are completely replaced with their
ST_* counterparts).

I was aiming for compatibility with 1.1.0 and above, and the spatial
ORM automatically prefixes the lookup types with "ST_" if you're
running 1.2.1+.   However, I see that I was mistaken in some of my
assumptions.  Specifically, I thought that ST_GeomFromText and
ST_Transform were available in 1.1.0 -- which is apparently not the
case (I relied on comments in the SVN version that stated these
functions were introduced in 1.1.0).  GeoDjango has hard-coded 'ST_'
references in the following locations:

http://code.djangoproject.com/browser/django/branches/gis/django/contrib/gis/db/backend/postgis/field.py#L107
http://code.djangoproject.com/browser/django/branches/gis/django/contrib/gis/geos/base.py#L204

If you change these lines to delete the 'ST_' prefix, then it should
work.  I'll try and patch this issue -- thanks for posting!

-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: query.py refactoring by Malcolm: status?

[Justin Bronn]

I'm interested in the status too, and I'm willing to help out w/any
implementation.  I've got a few feature requests for this refactor;
specifically, the possibility of mainlining several of the
modifications I've made in the GIS branch that assist in constructing
spatial SQL.

-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Ticket #399 (Bigint field object needed) status

[Justin Bronn]

+1 to bigint support.

I ran into this problem recently when implementing IP geolocation
models.  I know I'm not the only one that has or will experience these
problems (remember when slashdot crashed after 16,777,216 comments?).

>From what I understand, the largest hurdle to having large integer
support is that different db backends have different sizes for
bigint.  Are there any other issues holding this back?

I'll be happy to implement tests for this patch; however, the only
integer tests I found were in tests/regression_tests/
serializers_regress.  Is it preferred to implement the tests there, or
somewhere else?

-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: db2 backend

[Justin Bronn]

I have no experience with DB2, but I'm interested in a backend for
Django. Specifically, my interest stems from DB2's spatial database
offering ("Spatial Extender") that is free and feature complete (OGC
compliant).  Oracle and ArcSDE have high licensing fees, and MySQL's
spatial support is extremely limited (and will be so for the
foreseeable future) -- thus leaving DB2 as the next closest candidate
for spatial support.

I was, however, turned off by the state of pydb2 -- it looked like the
project hasn't been touched in years.  This is not a priority for
GeoDjango at the moment, but it would be cool to play around with a
spatial database other than PostGIS.

-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: lazy-geometries/planning

[Justin Bronn]

> For instance, the lazy geometries patch in 4322 is way off
> track now we have support for ogr geometries.

The lazy geometries patch in 4322 is only a month old, and ten days
ago I incorporated portions of your patch in r5448.  Not exactly what
I would consider "way off track" -- at least from my experience with
open source projects.  Regardless, I'll concede that GDAL feature
additions have been prioritized over your patch for the reasons
enumerated below.

The addition of GDAL support was, I believe, more important for
GeoDjango than lazy geometries.  To begin with, all lazy-geometry
functionality can be accomplished with existing extra instance methods
(e.g. "geomodel.objects.get_geom_geos().convex_hull").  Further, my
priorities have been aligned with what's on the wiki -- the phase 2
components with the highest priority are distance queries and GIS
dataset utilities (i.e., importing from SHP files directly into
GeoDjango models).  The addition of native GDAL support is aligned
with these priorities.

As those familiar with GIS systems are aware, GIS data comes in a vast
multitude of formats (e.g. ESRI Shapefiles, MapInfo files, TIGER
files, etc.).  To complicate matters, each dataset will use different
spatial reference systems -- rarely do I see a SHP file in WGS84 (for
non-GIS folks this is the system for lat/lon coordinates you get from
GPS).  Fortunately, the GDAL library has an _excellent_ library for
both reading and transforming spatial data sets in these conditions.
While this may have been possible through the use of PostGIS SQL
statements such as "INSERT ... transform("POLYGON((... ))", 4326)," I
wanted to have this functionality accessible to the user, independent
of the database.

Distance queries present similar problems since the calculated
distance depends on the spatial reference system.  For example,
distances calculated with coordinates in WGS84 are in units of degrees
-- hardly of use to someone looking for a store within 2km of their
home.  Transforming to a projected coordinate system, with units in
meters (or miles), is more appropriate.  PostGIS transformations only
accept an EPSG code (integer) associated with the spatial reference
system; giving the user access to a SpatialReference object that can
take multiple inputs (proj strings, EPSG codes, Well-Known Text, etc.)
that can then be passed to future distance routines seemed like a
better solution.

> but it helps if i can see a little way into the future
> about what other changes are planned to happen, and who is working on what.

There has been little "formal" division of duties on the GIS branch --
the only thing I can think of is that Jeremy handles the syncs to
trunk.  Please be patient with our progress -- like other devs, we
have commitments outside of Django that may make commits and ticket
fixes sporadic.

Regards,
-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007: Check Constraints Update

[Justin Bronn]

Thejaswi,

> Problem:
> The check constraints got added successfully (after a syncdb) but the
> problem is when I go to the admin page and to the model's page in it,
> I get an error ... This error vanishes when I comment the check fields
> in the models.py. I don't understand why the Check (pseudo) field is
> getting (trying to get) rendered in the Admin section.

I think what's going on here is that since you decided to implement
your constraint checks as a field, django tries to retrieve the check
field on queries.  Specifically, take a look at line 477 in
_get_sql_clause() from django/db/models/query.py:
http://code.djangoproject.com/browser/django/trunk/django/db/models/query.py#L477

The select statement is being constructed for every field in the the
model, as it does by default.  However, when the query is run the
database cannot find that column and throws an error.

The model thinks check is a valid field because it is within the
_meta.fields attribute of the model.  The check field gets added to
the _meta.fields model attribute when the contribute_to_class() method
(inherited from the Field base class) is called:
http://code.djangoproject.com/browser/django/trunk/django/db/models/fields/__init__.py#L142

You could override the contribute_to_class() method in your Check
class definition to prevent this from happening, but then the SQL
would not be executed from management.py (since it increments over
_meta.fields).  I'm at a loss for a solution right now, but I think
that's your problem.  One idea is to place the check field in
different attribute in _meta (see django/db/models/options.py).

Regards,
-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Custom field _post_create_sql()

[Justin Bronn]

> In PostGIS, the AddGeometryColumn() function needs to be run separately
> after the create table. It adds the geometry field to the table, adds
> its definition (projection, geometry type, dimensions) to the
> 'geometry_columns' table, creates a number of constraints, etc.

Thanks, Rob, for the excellent summary; below, I'll give a bit more
detail.

> GIS guys, any particular reasons for the approach you took? -- laziness?
> economy? "it works, dammit!"? technical reason against the above?

Malcolm, the fact that AddGeometryColumn() must be called _after_ the
CREATE TABLE statement is not a choice of the GIS branch.  As I stated
before, geometry column creation with the AddGeometryColumn() stored
procedure is part of the OpenGIS standard. [1]

When figuring out where to place the check for the _post_create_sql()
function, I did not randomly choose a function in management.py.  My
goals were to (1) accomplish geometry column creation in the
aforementioned manner, (2) minimize patches outside of the contrib
directory, and (3) use the existing API to the fullest extent
possible.  I believe this solution, a three-line patch to an existing
post-initialization SQL function, satisfies these criteria.

-Justin

[1] See Open GIS Consortium, Inc., OpenGIS Simple Feature
Specification For SQL Rev. 1.1, Document 99-049 (May 5, 1999), at 2-38
(s. 2.3.8, Geometry Values and Spatial Reference Systems, pg. 40),
available at http://www.opengis.org/docs/99-049.pdf.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007: Implementing Check Constraints

[Justin Bronn]

>I am a participant in the Summer of Code 2007.

Welcome to the development list!

>For my project,all that I have to do is actually append the sql
>statements for the table (containing the check statements) when syncdb
>is called. My problem is a typical one that "There is more than one
>way of doing it"

We encountered a similar problem on the GIS branch.  Specifically, the
OpenGIS specification requires that compliant spatial databases add
geometry columns by calling the stored procedure AddGeometryColumn()
after the table definition.

The way we solved it was a bit different from your proposals.  We
patched the management.py function get_custom_sql_for_model() to look
for a function on the field called "_post_create_sql."  In our Field
subclass, GeometryField, we define a _post_create_sql method that
returns the needed SQL (a string).  The field-generated SQL hitches a
ride on the output of the initial SQL data function that is already
called anyway.

Patch to management.py is the first shown:
http://code.djangoproject.com/changeset/4674

Look at GeometryField to see how _post_create_sql() works:
http://code.djangoproject.com/browser/django/branches/gis/django/contrib/gis/db/models/fields/__init__.py

Regards,
-Justin


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Type coercion in Django

[Justin Bronn]

> How do you handle data input of GIS types? What format is used during
> assignment?

During assignment, PostGIS returns in a format called HEXEWKB
(hereinafter "HEX"), one of its 'canonical forms'.  Essentially it's a
hexadecimal string of the binary format (WKB, an OGC standard).  For
input you can input in either HEX or WKT (another OGC standard) since
PostGIS accepts both.

> Are you using to_python at all in the GIS branch?

Not yet, like you said it's used with the forms and serialization,
both of which I haven't gotten to yet.  I tried it out before thinking
it did coercion and discovered it had no effect as to what was
returned from the database.

> Is there a use case for being able to get back data of more than one?
> type, or is it possible to get a WKT or KML representation from a
> GEOSGeometry object?

>From GEOSGeometry it is simple to retrieve WKT and HEX, e.g., 'g.wkt'
or 'g.hex'.  However, KML is implemented by PostGIS and can only be
retrieved if you do "SELECT AsKML(the_geom)" on the database end.
Further, GML (ancestor to KML, another OGC standard) and SVG can also
be retrieved with the AsGML() and AsSVG(), respectively.

> My idea for the internal representation of a model field would be a
> lazily-instantiated GEOSGeometry object, which would fill Justin's
> requirements. That would also allow conversion to WKT/GML/KML/etc via
> the GEOS library. And if you assigned WKT to a field it'd convert that
> to a GEOSGeometry object.

+1 for lazy-instantiated model fields

> I haven't done any benchmarking. Justin?
Nothing scientific here, but I ran through the 150 state house
legislature districts of Texas (4MB in SHP file) in the following
manner

geoms = [h.get_poly_geos() for h in House.objects.all()]

That completed in 1.38 seconds.  The database query itself took .21,
making the overhead to be around 1.17s, significantly more than the
database.  But not too bad for processing a set of 'real-world'
geometries covering a very large area.   Definitely fast enough for
lazy-instantiation.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---