Re: App Engine support

[Russell Keith-Magee]

On Sun, Aug 30, 2009 at 8:52 PM, Waldemar Kornewald wrote:
>
> On Aug 28, 1:49 am, Russell Keith-Magee 
> wrote:
>> To the extent that I'm in a position to provide design guidance and
>> feedback from the perspective of the Django Core, put me on this list
>> too. Time permitting, I might be able to contribute some code, too.
>
> Awesome. Could you please provide some feedback on Andi's latest patch
> (or commit it if it's good enough)?
> http://code.djangoproject.com/ticket/10355
>
> Thanks a lot!

I've already given Andi a review of his first draft; his second draft
is on my list of things to look at.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App Engine support

[Russell Keith-Magee]

On Mon, Aug 31, 2009 at 2:48 AM, sjtirtha wrote:
>
> Hi,
>
> I'm starting using CouchDB with Python.
> Django db models always work with Queries. I can understand this,
> because SQL is a query language.
> However, the non sql DBs do not always have the concept of queries.
> For example, in CouchDB all CRUD operation is process via HTTP
> request.
> Create operation: PUT request
> Read operation   : GET request
> Update operation: POST request
> Delete operation: POST request (I'm not sure about this, CMIIW)
>
> So using the class or function name Query to access non sql DB will
> not suitable.
> How about to make it more general by using term Request or Operation?

Why, what a lovely bikeshed.

Seriously, of all the problems we have to deal with, the _name_ of the
Query class isn't one of them.

> I looked into django/db/models/query.py, the insert_query function
> implementation is very specific for SQL.

Yes. That's why we're embarking on a project to make it - and all the
other SQL specific parts of the generic interface - non-SQL specific.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App Engine support

[Russell Keith-Magee]

On Mon, Aug 31, 2009 at 2:54 AM, sjtirtha wrote:
>
> Hi,
>
> From this point of view, I would propose, to make a new Model that
> supports non SQL DB.
> So developer can decide whether they want to store the object in SQL
> or in non SQL DB.

There are two problems here:
 1. You want Django to be able to talk to non-SQL backends
 2. You want to be able to talk to two different backends in the same project.

Point 1 is the task of the non-SQL backend work.

Point 2 is the task of the multi-database support work that Alex
Gaynor has been working on for the GSoC.

There's no need to conflate the two problems. The multi-db interface
that Alex has been developing will allow one database to be Postgres
and a second to be MySQL; once there is support for non-SQL backends,
it will be trivial to make the second database CouchDB, or any other
supported backend.

Yours
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App Engine support

[sjtirtha]

Hi,

I'm starting using CouchDB with Python.
Django db models always work with Queries. I can understand this,
because SQL is a query language.
However, the non sql DBs do not always have the concept of queries.
For example, in CouchDB all CRUD operation is process via HTTP
request.
Create operation: PUT request
Read operation   : GET request
Update operation: POST request
Delete operation: POST request (I'm not sure about this, CMIIW)

So using the class or function name Query to access non sql DB will
not suitable.
How about to make it more general by using term Request or Operation?

I looked into django/db/models/query.py, the insert_query function
implementation is very specific for SQL.
How about to make the Manager class to decide which function should it
calls, if non SQL DB is set?
For example instead of directly forwarding the query in _insert method
of Manager class to insert_query, we call create_operation function,
if a Non SQL DB is set.

Regards,
Steve

Regards,
Steve



R

On Aug 28, 12:01 am, Waldemar Kornewald  wrote:
> On Aug 27, 11:54 pm, Waldemar Kornewald  wrote:
>
> > We also have a few other potential contributors on our list, but right
> > now they're too busy or not ready for contributing, yet:
> > * Thomas Bohmbach (from Giftag)
> > * Curtis Thompson (from Giftag)
> > * Joe Tyson (maybe you know him? :)
> > * Malcolm Tredinnick (core Django developer)
>
> > A few others have shouted, but I got no further reaction, so I don't
> > mention them here (please correct me and say something if I mistakenly
> > forgot you).
>
> I knew I forgot someone right after having hit "Send". Sorry, was
> focusing too much on App Engine. :(
>
> Mitch Garnaat (creator of boto) wants to help with the non-relational
> backends refactoring and implement SimpleDB support. I haven't seen
> any code from him, yet, but he added some info to the documentation.
> His view on SimpleDB is really important for getting the non-
> relational code generic enough. I hope we could get more input from
> other DB users (e.g.,CouchDB, MongoDB, Hadoop, ...).
>
> Bye,
> Waldemar

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App Engine support

[sjtirtha]

Hi,

I have another requirement for non sql DB support in Django, which I
think will be very applicable.
Most of the current applications do not use 100% non sql DB for the
data storage. Most of them still mix SQL and non SQL DB as data
storage.
Because both of them should not compete against one other, but they
should complete each other.
For example Facebook uses both SQL and non SQL DB.

>From this point of view, I would propose, to make a new Model that
supports non SQL DB.
So developer can decide whether they want to store the object in SQL
or in non SQL DB.

Regards,
Steve

On Aug 28, 12:01 am, Waldemar Kornewald  wrote:
> On Aug 27, 11:54 pm, Waldemar Kornewald  wrote:
>
> > We also have a few other potential contributors on our list, but right
> > now they're too busy or not ready for contributing, yet:
> > * Thomas Bohmbach (from Giftag)
> > * Curtis Thompson (from Giftag)
> > * Joe Tyson (maybe you know him? :)
> > * Malcolm Tredinnick (core Django developer)
>
> > A few others have shouted, but I got no further reaction, so I don't
> > mention them here (please correct me and say something if I mistakenly
> > forgot you).
>
> I knew I forgot someone right after having hit "Send". Sorry, was
> focusing too much on App Engine. :(
>
> Mitch Garnaat (creator of boto) wants to help with the non-relational
> backends refactoring and implement SimpleDB support. I haven't seen
> any code from him, yet, but he added some info to the documentation.
> His view on SimpleDB is really important for getting the non-
> relational code generic enough. I hope we could get more input from
> other DB users (e.g.,CouchDB, MongoDB, Hadoop, ...).
>
> Bye,
> Waldemar
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: CSRF proposal and patch ready for review

[Russell Keith-Magee]

On Mon, Aug 3, 2009 at 11:25 PM, Luke Plant wrote:
>
> Hi all,
>
> Some big changes to the CSRF protection nearly got in to Django 1.1,
> but didn't.  Since then, more work has been done, overhauling the
> whole thing really.  There has been a huge amount of discussion on
> mailing lists and tickets, so I've put together what I consider to be
> the conclusions, and my proposal for a way forward, on this page:
>
>  http://code.djangoproject.com/wiki/CsrfProtection
>
> That page has almost everything you need to know, but just to
> highlight some things:
>
>  * The proposal is implemented in the lp-csrf_rework branch of
>   this repo:
>
>   http://bitbucket.org/spookylukey/django-trunk-lukeplant/
>
>   The difference from trunk is regularly copied to patches on ticket
>   #9977, if you want simple diffs.
>
>  * The proposal (at the bottom of that page) needs review and
>   consensus.
>
>  * Most of the implementation has been pretty well tested, but of
>   course could do with review, especially as it is security related.
>
>  * The strict referer checking for HTTPS has not been tested at all
>   in any live situations.  (I don't have a live HTTPS site handy
>   for doing such tests).  It's possible that special attention also
>   needs to be paid to HTTPS on non-default ports. Obviously these
>   things need to be addressed before this could go into trunk.
>
>  * The tutorial has not been updated.  Fixing it is not trivial.
>   Since we are including the CsrfViewMiddleware in
>   project_template/settings.py, POST forms, such as the one in
>   tutorial 4, require {% load csrf %} and {% csrf_token %}, and
>   corresponding view functions (created in tutorial 3) require
>   RequestContext.  I don't know what to do about this.  Fixing
>   it is ugly, not fixing it is worse.  One option is to rewrite the
>   view function from tutorial 3 in tutorial 4, combining the two
>   views in one -- they have a fair amount of redundant code
>   anyway -- and add RequestContext at that point.
>
> I'm away for 3 weeks from this Wednesday, so I won't be able to
> respond to questions during that time.  I imagine for most queries
> about the patch, 'Glenn' from ticket #9977, who has done a lot of the
> work on this (many thanks Glenn!), will be able to answer most things.
> Hopefully he's on the list!
>
> It would be best for discussion to take place on this list (or on
> ticket #9977), the wiki page is just for gathering conclusions.

I've had a quick look at the patch, and found a few minor cosmetic
things. I've also done a lot of reading of the archives to understand
why the patch is the way it is. A comprehensive teardown of the patch
will take a bit longer, but before I do that teardown, I think there's
a bigger issue that we need to address.

Are we really sure we want to do this?

Jacob flagged this back on April 1, and as far as I can make out, his
criticism is still valid. The problem is summed up by the following
question:

Are you absolutely sure that you have correctly updated all the
templates that Django ships?

You say that the update process is a simple search and replace, and
I'm sure you've been very thorough, but every form you've accidentally
missed (or overlooked because it renders the form in a strange way)
will be a regression that throws 403 errors.

Every single Django user will need to undergo the same code and
template update process. Your 'search and replace' estimate also
discounts the the engineering cost of testing and validation that we
will be imposing on every single user.

Another consequence - the external community of apps will also be
suddenly split into two - those that have been updated for CSRF
support, and those that haven't. You won't be able to use a v1.1
Django-compatible app on a Django v1.2 install without disabling CSRF
support.

One option you've mentioned is to install the CSRFResponseMiddleware
as a transition solution. This will hide the problem for v1.2, but it
will just come back to bite use when we complete the deprecation
cycle. At that point, any form that hasn't been updated will stop
working. At which point, we're back to the engineering overhead of the
update.

And all of this is dancing around the fact that the tag insertion
process is just messy. Every form template needs to import the CSRF
module. Every view with a form _must_ use a RequestContext. The
upgrade path for existing apps involves a lot of little changes. Yes,
you've documented the transition, but every extra step in a transition
document is one more thing that can (and will) be misinterpreted or
misunderstood by somebody. There's just too many moving parts involved
here for this to be a painless transition for everyone.

I completely agree that CSRF protection is important, and it's a
crying shame that the CSRF middleware is off by default. As Simon
mentioned early on, this probably means that the CSRF middleware isn't
getting used anywhere near as often as it should.

I also 

Re: App Engine support

[Alex Gaynor]

On Sun, Aug 30, 2009 at 11:06 AM, Waldemar
Kornewald wrote:
>
> On Aug 30, 3:50 pm, Andi Albrecht 
> wrote:
>> No need to hurry. I'll keep this in sync with the trunk - should be
>> really unproblematic :)
>
> I'm not really doing this just for fun, so I'd rather get it done
> faster. ;)
>
>> Regarding the App Engine backend: I have some minimal code here for
>> the database backend. Currently it does nothing useful but it provides
>> at least the backed so that it is importable in the shell.
>> Andy, are you working on this too? Maybe you've got more on your desk
>> than just a database backend stub...
>
> Nice. If you already have commit access you could push it into the
> backend repo. Please create a separate package within the repo because
> we'll have multiple backends in there (email, DB, maybe sessions,
> maybe memcache). BTW, did you have a look at the DB backend in app-
> engine-patch? That already provides a lot of important features like
> creating a test DB, managing different DB files per appid, supporting
> remote DB access, etc.
>
> In general, it's always a good idea to first look at the app-engine-
> patch source (including the patched Django repository), so you don't
> duplicate code unnecessarily.
>
>> BTW, what would be a good name for this backend? "appengine", or is
>> "bigtable" more appropriate?
>
> I'd say "appengine" because we're not really using BigTable directly
> (the App Engine datastore is built on top of BigTable, but they're not
> the same).
>
> Bye,
> Waldemar Kornewald
> >
>

I'd be -1 on having a seperate, restricted API.  There is, IMO, no
reason to have more than one API, any sort of restricted system should
come in the form of documentation saying what the minimum
functionality needed for a database backend to support Django's
various features + the backend throwing exceptions for unsupported
operations.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App Engine support

[Waldemar Kornewald]

On Aug 30, 3:50 pm, Andi Albrecht 
wrote:
> No need to hurry. I'll keep this in sync with the trunk - should be
> really unproblematic :)

I'm not really doing this just for fun, so I'd rather get it done
faster. ;)

> Regarding the App Engine backend: I have some minimal code here for
> the database backend. Currently it does nothing useful but it provides
> at least the backed so that it is importable in the shell.
> Andy, are you working on this too? Maybe you've got more on your desk
> than just a database backend stub...

Nice. If you already have commit access you could push it into the
backend repo. Please create a separate package within the repo because
we'll have multiple backends in there (email, DB, maybe sessions,
maybe memcache). BTW, did you have a look at the DB backend in app-
engine-patch? That already provides a lot of important features like
creating a test DB, managing different DB files per appid, supporting
remote DB access, etc.

In general, it's always a good idea to first look at the app-engine-
patch source (including the patched Django repository), so you don't
duplicate code unnecessarily.

> BTW, what would be a good name for this backend? "appengine", or is
> "bigtable" more appropriate?

I'd say "appengine" because we're not really using BigTable directly
(the App Engine datastore is built on top of BigTable, but they're not
the same).

Bye,
Waldemar Kornewald
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App Engine support

[Andi Albrecht]

On Sun, Aug 30, 2009 at 2:52 PM, Waldemar Kornewald wrote:
>
> On Aug 28, 1:49 am, Russell Keith-Magee 
> wrote:
>> To the extent that I'm in a position to provide design guidance and
>> feedback from the perspective of the Django Core, put me on this list
>> too. Time permitting, I might be able to contribute some code, too.
>
> Awesome. Could you please provide some feedback on Andi's latest patch
> (or commit it if it's good enough)?
> http://code.djangoproject.com/ticket/10355

No need to hurry. I'll keep this in sync with the trunk - should be
really unproblematic :)

Regarding the App Engine backend: I have some minimal code here for
the database backend. Currently it does nothing useful but it provides
at least the backed so that it is importable in the shell.
Andy, are you working on this too? Maybe you've got more on your desk
than just a database backend stub...

BTW, what would be a good name for this backend? "appengine", or is
"bigtable" more appropriate?

Andi

>
> Thanks a lot!
>
> Bye,
> Waldemar
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: I want to know how to traversal all the urls in a django project?

[Karen Tracey]

Please don't crosspost to both django-users and django-developers. Pick the
right list and post there, in this case -users.

Karen

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App Engine support

[Waldemar Kornewald]

On Aug 29, 8:34 pm, Mike Malone  wrote:
> > Do you also intend to contribute some code? You could team up with
> > Mitch on the SimpleDB backend, for example (at least, you seemed to be
> > most interested in that).
>
> Sure.

Great! :)

> [...] I'm afraid that
> emulating the SQL join syntax in general for all backends would be
> overpromising. One of the reasons some of these new datastores are so
> useful is because they force you to rethink your data and store it in
> a way that makes it efficient to retrieve. You lose this benefit if
> you hide the efficient interface exposed by the datastore behind a
> layer that does a bunch of inefficient stuff.

Indeed, JOINs can often be evil, but they are still needed and
actively used even on non-relational DBs. They're just written by
hand. You definitely can use them in a scalable and efficient way *if*
you know what you're doing. A dumbed down API - just to prevent noobs
from doing stupid things - doesn't make sense. Instead, the emulation
code should warn the developer when something gets emulated and might
potentially be inefficient.

Bye,
Waldemar Kornewald
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---