#29752 Adding a ALLOWED_HOSTS_IGNORABLE_URLS setting

[Jonas H]

Hi,

I've started a discussion on https://code.djangoproject.com/ticket/29752 to 
add a new ALLOWED_HOSTS_IGNORABLE_URLS setting.

The setting can become handy if you can't control the Host header sent to 
your application but still want to accept the request. An example of this 
is health checks made by AWS ECS/Fargate – google "django allowed_hosts 
aws" and find 16,000 results with tips how to work around the problem.

I'd like to discuss the addition on this list as per Tim's triage.

Jonas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/302133e2-5b3f-4524-869a-a0122de6998e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Runtime error

[Jonas H]

Not sure if this is what's meant by the first post, but I got very confused 
the other day by the ./runtests.py script:

If you run ./runtests.py ../tests/foo, it crashes with all sorts of "X 
doesn't declare explicit app label" exceptions.

If you run ./runtests.py foo, it works.

Not sure why I even used the first invocation, though.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5b042224-99c3-4ab3-9cc1-a195d3dd3693%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Templates: __html__ method support

[Jonas H]

Proposal: Support the __html__ method as an alternative/addition to the 
__str__ for turning objects into strings in the template layer.

If this has been discussed before, please point me to it; I couldn't find 
anything with the search function.

Some custom classes may have, in addition to a __str__ representation, a 
natural representation that is better suited for HTML output. Example:

class Money:
def __init__(self, amount, currency):
self.amount = amount
self.currency = currency

def __str__(self):
return '%s %s' % (self.currency, self.amount)

def __html__(self):
# Always show amount and currency on same line
return '%s\xa0%s' % (self.currency, self.amount)

`conditional_escape` and friends already consider the __html__ method, and 
this works out well:

>>> str(Money(1, '$'))
'$ 1'
>>> conditional_escape(Money(1, '$'))
'$\xa01'

In templates however it doesn't work that way because variables are always 
turned into strings before stuffing them into `conditional_escape` 
(see 
https://github.com/django/django/blob/98706bb35e7de0e445cc336f669919047bf46b75/django/template/base.py#L977).
 
My suggestion is to change the behaviour of that function so that it works 
as follows:

- Given I write {{ foo }}
- Does foo have a __html__ method? If yes, return `foo.__html__()`
- Otherwise, return `conditional_escape(str(foo))`

Do think that's a good idea?

Jonas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ea088de2-e538-4808-a7fd-8726929e2b91%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Possible bug transaction rollback or SQLite backend

[Jonas H.]

> On 03 Dec 2015, at 17:04, Carl Meyer  wrote:
> 
>> I use Python 3.5, SQLite 3.9.2 and I can reproduce the crash on Django
>>> =1.8.4,<1.9.
> 
> This might be useful information to add to
> https://code.djangoproject.com/ticket/24080, if it's not already present
> there.
> 
> Does the version range you give imply that the issue goes away in Django
> 1.9, or just that you've never tried it there?

So, having ported my application to Django 1.9, I can say that the error is 
gone in Django 1.9. It happens in Django 1.8.X, X>=4.

I can’t test on Django 1.8.X, X<4, because the application needs #25160 fixed. 
I can’t test at all on Django < 1.8.

Another observation is that the error only appears *without* migrations, i.e. 
if the “migrations” directory doesn’t exist. It works for 1.8.X, X>=4 if proper 
migrations exists.

Also, the error only happens when `raise SkipTest` isn’t used correctly, as 
shown earlier in this thread.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/C0FD1C0A-C9F4-4586-8C9B-97D9CE91721F%40lophus.org.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Possible bug transaction rollback or SQLite backend

[Jonas H]

Tim,

thanks for the pointer. I'm not sure if my issue is related. I've found a 
minimal example for another (same?) 
issue: https://code.djangoproject.com/ticket/25860

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/19ad9d9d-1ea0-4a9a-a280-3b02ff3b2320%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Possible bug transaction rollback or SQLite backend

[Jonas H]

Hi there,

there seems to be something wrong with either the SQLite backend, the 
SQLite library itself or Django's transaction handling. Whenever I try to 
run the test suite of a specific project of mine 
(https://github.com/jonashaag/fahrtkostenrechner), using 
`fahrtkostenrechner/manage.py test events`, it shows the following behavior:

* On Linux, it doesn't crash and shows actual test failures.
* On OSX, with SQLite's in-memory database as test database, a segfault (!) 
occurs. (I'm not familiar with the lower-level Python SQLite bindings, but 
I guess a segfault should *never* happen in the first place?!)
* On OSX, with a on-disk test database, a 
"django.db.utils.ProgrammingError: Cannot operate on a closed database." 
(caused by `_cursor()`: 
https://github.com/django/django/blob/4d0f8831a7498ab3b1ebcf4cafa2ee234503e4f8/django/db/backends/base/base.py#L206)
 
occurs.

It looks to me that the cause of this is some unfinished transaction 
rollback, but I'm not sure, so I'm asking for guidance here. In the file 
linked above in L193, is it legitimate for `needs_rollback` to  be True? In 
my debugging I found `connection != None` and `needs_rollback=True` 
whenever `_cursor()` fails in L206, so I figured maybe there might be an 
outstanding rollback for some reason?

I use Python 3.5, SQLite 3.9.2 and I can reproduce the crash on Django 
>=1.8.4,<1.9.

Jonas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/39e18dc4-7e7d-4e99-a56c-e0e133ff5938%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Django 1.9 AppRegistryNotReady

[Jonas H]

Hello all,

I'm not sure if it's expected behaviour,  so I thought I'd rather report 
this here: I've had issues importing multiple (well[ish]-known) 3rd party 
apps with Django 1.9 that import fine on Django 1.8. The error I'm getting 
is AppRegistryNotReady, caused by some (indirect) import to some model in 
the 3rd party app's __init__.py file. I know that importing models before 
the app is fully loaded has been officially deprecated in 1.7, but it seems 
to have worked well anyways since then.

One example of such a 3rd party app 
is https://github.com/chrisglass/django_polymorphic.

So maybe this should be fixed, if possible, for Django 1.9; or deliberately 
not fixed to force people adhere the "don't import models in __init__" 
policy.

Jonas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3309082d-385a-485d-8c57-7f5bfa740506%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django-nonrel patches

[Jonas H.]

On 06/26/2012 11:07 PM, Adam "Cezar" Jenkins wrote:

Last I heard Jonas wanted to do a new release of mongodb-engine. He might
have meant to say he was not maintaining the project anymore, but he's not
the only one with commit access.


The main thing is that I've lost interest in maintaining this huge 
project on my own -- and to be honest, there haven't been too many 
code/documentation contributions since the GitHub migration.


I tried to do a new MongoDB Engine release for a few times but every 
time something got in my way (like Travis not working, University, ...). 
Not too much support from other people here either.


I'm not blaming anyone here. It's just the current state of things.


In terms of the fundamental "MongoDB vs. ORM" debate, here are my 2 
cents: It's okay if we only get to work 70% of the ORM and apps -- 
that's already a huge benefit for people working with MongoDB and 
Python.  Most apps don't use JOINs or could get rid of them with only a 
handful of simple changes.


Cheers :-)
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: gsoc proposal, dynamic list form field

[Jonas H.]

tl;dr: http://gsoc-demo.herokuapp.com/addrecipe (the "Ingredients" forms)

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting multiline tags

[Jonas H.]

On 02/24/2012 10:01 AM, Stephan Jaensch wrote:

This thread contains 6 people expressing support for this change, and 2 against 
(a BDFL, a core developer) -- and you can add me to the -0 list. There are over 
6000 subscribers to django-developers. I put it to you that the vast majority 
of people haven't expressed an opinion -- and many of those haven't expressed 
an opinion because they're happy with (or indifferent to) the status quo, and a 
BDFL has already indicated that the status quo is his preferred option.


You asked for it, so here is my +1.


+1 from me too.

Maybe someone could invent an official feature voting tool? :-)

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Issue in sending email

[Jonas H.]

Please post questions about using Django on django-users, not here. The
topic of this list is the development of Django itself.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-nonrel patches

[Jonas H.]

On Sun 08 Jan 2012 05:39:02 PM CET, Emil Stenström wrote:



On Thursday, December 8, 2011 10:31:39 PM UTC+1, jo...@lophus.org wrote:


1.) I didn't write the code, I'm just submitting the patches in their
current state


Jonas: Do you have information on who wrote the code that you submitted? I
guess a good way forward would be to find those people, and have them
contribute a brief explanation of what problem their patch was trying to
solve. For a "solution" to be accepted, you need to know the problem.



Waldemar Kornewald wrote most of the code but he's pretty busy these 
days.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #16630: Support for HTML5 input types

[Jonas H.]

On 12/28/2011 05:15 AM, poswald wrote:

I have a concern regarding this.. A few times I have tried to
integrate django-floppyforms which takes a similar approach to what
the html5 widgets offer (input types are specified by default) and it
often causes pain. The reason is that while browsers say they support
html5 input types sometimes that support is lacking or very badly
implemented. As an example, if you say  the only officially supported date format is an
RFC-3339 'full-date' format (-MM-DD) according to the spec:

http://dev.w3.org/html5/markup/input.date.html#input.date.attrs.value
http://tools.ietf.org/html/rfc3339

This means that you cannot have any other format of date string in
that form field.

Now, this ticket 16630 doesn't change the date field specifically but
it does change the number field. (Is there a similar ticket for
changing the date field?) I've run into a similar problem with the
type='number' that this ticket does change. The problem I ran into was
that forms cannot easily use the THOUSANDS_SEPARATOR because it is not
a valid number. It has to be a 'float' to be valid. This means
technically you need to use the text widget for that.


The reason why I decided to not include the 'date' type in the patch was 
indeed the lack of browser support for non-anglophone input formats.


Seems like this needs to be extended to numbers as well.

--
http://jonas.lophus.org | http://django-mongodb.org

Want to become a Django MongoDB Engine/Django-nonrel core developer? 
Get in touch: django-non-relatio...@googlegroups.com


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-nonrel patches

[Jonas H.]

On 12/08/2011 11:39 PM, Jacob Kaplan-Moss wrote:

On Thu, Dec 8, 2011 at 3:31 PM, Jonas H.<jo...@lophus.org>  wrote:

2.) I don't like wasting my time, so I won't write extensive documentation
before the patch gets accepted


I'm sorry, but that's just not how we do things around here.
Documentation isn't an afterthought; is as important -- more even --
then the code itself. In order for a patch to be considered "done", it
needs documentation.

If you don't feel the same that's fine, but you'll need to connect
with someone who doesn't see documentation as a waste of time.
Otherwise with a million patches to review I'm going to prioritize the
ones that have documentation, and yours'll get lost.


I don't think documentation is a waste of time and tests aren't either.

I just don't like to write docs and tests in the first of thirteen 
iterations. I don't think the patches are ready for checking yet; what 
I'm currently looking for is general feedback on the approach taken by 
the modifications.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-nonrel patches

[Jonas H.]

On 12/07/2011 09:53 PM, Aymeric Augustin wrote:

I just came across your patches while I was triaging the "unreviewed" queue. 
Unfortunately, I found it difficult to assess them, since they don't include tests or 
docs.

Generally, all changes should come with tests. If you're adding new features — for 
instance "Support for non-integer primary keys" sounds an interesting one — 
those should be documented too.

That would help people who aren't familiar with django-nonrel (like me) to 
review your patches, and in particular to understand their consequences.


So there are multiple reasons why these patches don't include tests nor 
docs:


1.) I didn't write the code, I'm just submitting the patches in their 
current state


2.) I don't like wasting my time, so I won't write extensive 
documentation before the patch gets accepted


3.) Many changes can't be tested in a useful manner using unittest. For 
instance the non-integer primary key patch should really be tested using 
integration tests -- but that requires complex mocking which takes a lot 
of time. Again I'd like to not waste my time. I'm willing to write tests 
and docs once the patch is accepted.


To make things clear, I don't refuse to do *informal* documentation 
(like in this thread or on the ticket tracker) because that doesn't take 
so much time and brain power :-)


So if you need help understanding the patches, shoot!

Jonas

--
http://jonas.lophus.org | http://django-mongodb.org

Want to become a Django MongoDB Engine/Django-nonrel core developer? 
Get in touch: django-non-relatio...@googlegroups.com


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Django-nonrel patches

[Jonas H.]

Hello list!

On Jacob's suggestion in this thread [1] back in April, I split the diff 
between Django trunk and Django-nonrel into logically separated patches.


I uploaded most of them to the ticket tracker (a few things are still 
missing but these are the most important changes):


https://code.djangoproject.com/ticket/17335
https://code.djangoproject.com/ticket/17336
https://code.djangoproject.com/ticket/17337
https://code.djangoproject.com/ticket/17338
https://code.djangoproject.com/ticket/17339
https://code.djangoproject.com/ticket/17340
https://code.djangoproject.com/ticket/14881 (not exactly a new, has been 
around for a while)


Happy reviewing! :-)

Jonas

[1] 
http://groups.google.com/group/django-developers/browse_thread/thread/2c52d074b5a0f614/8610a64348e6f216?


--
http://jonas.lophus.org | http://django-mongodb.org

Want to become a Django MongoDB Engine/Django-nonrel core developer? 
Get in touch: jo...@lophus.org


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 and you

[Jonas H.]

On 11/07/2011 12:00 PM, richard.prosser wrote:

I may be willing to support this. I have 18 month's of Python
experience (mostly 2.4) and I have been out of work for a long time
due to illness, so a project like this may look good on my CV!

I have never been involved in an OS project before, though I am aware
of the guidelines.

At present the only machine I have available is a Windows Vista
laptop :-( Are there development/test servers available?


You don't need a Linux/Unix since Python/Django works on Windows.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [NoSQL] Sub-object queries / refactoring JOIN syntax into model fields

[Jonas H.]

bump

On 09/28/2011 12:52 AM, Jonas H. wrote:

Hallöchen,

some non-relational databases (e.g. MongoDB) have support for
arbitrarily nested objects. To make queries that "reach" into these
sub-objects, the Django-nonrel developers find it appealing to use JOIN
syntax. For instance, if you had this person in your database

{'name': 'Bob', 'address': {'city': 'NY', 'street': 'Wall Street 42'}}

you could find Bob using these queries:

Person.objects.filter(name='Bob')
Person.objects.filter(address__city='NY')
Person.objects.filter(address__street__startswith='Wall')
...

Similarly, sub-objects may be stored in a list, like so:

{
'votes': [
{'voter': 'Bob', 'vote': 42},
{'voter': 'Ann', 'vote': 3.14}}
]
}

Vote.objects.filter(votes__vote__gt=2)
...


These sub-object queries are essential for non-relational databases to
be really useful so this is an important feature.

What's the core team's opinion on this topic -- is there any chance to
get something like that into Django at all? (Maybe you think two
meanings for one syntax cause too much confusion)

Secondly, how could this be implemented? I thought about refactoring
JOIN syntax handling into the model fields (as little logic as required;
refactoring the actual hardcore JOIN generation code seems like an
impossible task for anyone but the original author)... any other ideas?

So far,
Jonas


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ORM refactoring

[Jonas H.]

On 10/10/2011 04:59 PM, Anssi Kääriäinen wrote:

In the perfect world you could have a SQLA backend, or qs.as_sqla()
method. That would be neat, yes. QuerySet chaining could be hard to
implement, though (or does SQLA have support for something like
that?). But in my opinion this is another argument for refactoring the
ORM. When the ORM has better separation of concerns and better
abstractions between compiler and sql/query.py it will be easier to
write different compilers - be it a compiler for Cassandra or SQLA.


What's required to make different database backends easy are correct 
abstractions -- currently the ORM backend abstraction is pretty messed 
up, at least it's not written with non-SQL backends in mind.


/Everything/ SQL related should be moved into a base SQL backend. The 
code in django.db.models should be completely query language/database 
agnostic. Currently, SQL generation is scattered over multiple places 
and abstraction layers, django.db.models.sql.* and django.db.backends etc.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re:

[Jonas H.]

On 10/04/2011 05:51 PM, Ted Gruenloh wrote:

The django online documentation mentions that the SQL equivalent for  
__icontains is something like:

 SELECT ... WHERE headline ILIKE '%Lennon%';
However, for postgresql - one of the dbs that actually supports ILIKE - I 
noticed __icontains was actually performing something similar to:
 SELECT ... WHERE LOWER(headline) LIKE LOWER('%Lennon%');

The ILIKE is obviously much faster [...]


https://code.djangoproject.com/ticket/3575

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [NoSQL] Sub-object queries / refactoring JOIN syntax into model fields

[Jonas H.]

On 09/28/2011 11:47 AM, Florian Apolloner wrote:



On Wednesday, September 28, 2011 12:52:51 AM UTC+2, jo...@lophus.org wrote:


What's the core team's opinion on this topic -- is there any chance to

Secondly, how could this be implemented?


Did you search the mailing list about this topic at all yet? I am not sure,
but I think Alex implemented something like that in the NoSQL branch.


No, there's no such thing in Alex' GSoC branch.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

[NoSQL] Sub-object queries / refactoring JOIN syntax into model fields

[Jonas H.]

Hallöchen,

some non-relational databases (e.g. MongoDB) have support for 
arbitrarily nested objects. To make queries that "reach" into these 
sub-objects, the Django-nonrel developers find it appealing to use JOIN 
syntax. For instance, if you had this person in your database


  {'name': 'Bob', 'address': {'city': 'NY', 'street': 'Wall Street 42'}}

you could find Bob using these queries:

  Person.objects.filter(name='Bob')
  Person.objects.filter(address__city='NY')
  Person.objects.filter(address__street__startswith='Wall')
  ...

Similarly, sub-objects may be stored in a list, like so:

  {
'votes': [
  {'voter': 'Bob', 'vote': 42},
  {'voter': 'Ann', 'vote': 3.14}}
]
  }

  Vote.objects.filter(votes__vote__gt=2)
  ...


These sub-object queries are essential for non-relational databases to 
be really useful so this is an important feature.


What's the core team's opinion on this topic -- is there any chance to 
get something like that into Django at all? (Maybe you think two 
meanings for one syntax cause too much confusion)


Secondly, how could this be implemented? I thought about refactoring 
JOIN syntax handling into the model fields (as little logic as required; 
refactoring the actual hardcore JOIN generation code seems like an 
impossible task for anyone but the original author)... any other ideas?


So far,
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: new feature idea: even and odd forloop iteration variables

[Jonas H.]

On 09/19/2011 12:58 PM, skazhy wrote:

Hi all!

I've been working with Django for a while now, but today I decided to
make a first contribution to Django that would make templates less
ugly by adding these two values to loop_dict (in django/template/
defaulttags.py):

loop_dict['even'] = (i %2 == 0)
loop_dict['odd'] = (i %2 != 0)

Using {% if forloop.even %} would be more clear and readable than {%
if forloop|divisibleby:"2" %}.
As I am new to working *on* Django I don't know the history of this
idea (if there is one) and the correct way of sending patches, all
comments (and corrections!) are appreciated!


-Karlis Lauva /skazhy/



https://docs.djangoproject.com/en/dev/ref/templates/builtins/#std:templatetag-cycle

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: "universal" view decorators

[Jonas H.]

On 09/15/2011 11:27 PM, Donald Stufft wrote:

tl;dr; Using Mixins to add in functionality to a CBV makes way more sense then
using a decorator which is essentially going to be doing the same thing as a
mixing would, it just makes finding what is going on harder, makes customizing
the decorator harder and/or uglier, and goes against the way functionality is
currently added to a CBV.


That sounds a lot better to me.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixture loading using bulk_create

[Jonas H.]

On 09/12/2011 12:15 AM, Anssi Kääriäinen wrote:

The feature could be useful if there are users loading big fixture
files regularly. Otherwise it complicates fixture loading for little
gain.


Maybe we could simply add an option to the loaddata command -- so that 
if someone really needs tons of fixtures for their tests it's possible 
to profit from bulk insertions by manually invoking loaddata from their 
test code. And the implementation is quite simple:


http://paste.pocoo.org/show/474602/ (doesn't cover all edge-cases yet)

I did some benchmarking with this code and it speeds up fixture loading 
*a lot*: http://www.chartgo.com/get.do?id=bdfe6af778 (chunksize=0 does 
not use `bulk_create` but `save`, and the speedups seen for chunksize=1 
is because `bulk_create` is used, thus avoiding `save` overhead)


Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal for a new templatetag definition syntax

[Jonas H.]

On 09/12/2011 11:49 AM, Cal Leeming [Simplicity Media Ltd] wrote:

I may have misunderstood this thread slightly but, should templatetags
follow the same structure as *args, **kwargs??

i.e.

*args
{% create_title "hello world" 800 800 %}

**kwargs
{% create_title title="helloworld" width=800 height=800 %}

mixture
{% create_title "helloworld" width=800 height=800 %}

Apologies if I have completely missed the point of this thread.

Cal


I guess it's not the "point of this thread" but rather Evgeny's design 
proposal. Alex' intention seems to be discussion on a new syntax to 
define template tags.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal for a new templatetag definition syntax

[Jonas H.]

On 09/12/2011 10:47 AM, lucky wrote:

Hello, Alex,

I agree with Wim Feijen. The ability to implement a arbitrary syntax
for tag parameters is too abstract problem and this causes an increase
in the complexity of the solution. Tag delelopers are forced to define
a grammar and deal with lexing and parsing. End users must to learn
and remember a custom syntax for each of tags. [...]

We should not be afraid of the python-like grammar for tag
paramenters. There is not principal difference beetween custom {%
mytag src limit 1 offset 10 %} and pythonic {% mytag src limit=1
offset=10 %}. The restrict syntax makes the solution simpler.


+1

Template tags should be forced to have a simple syntax. Users should not 
have to look up the syntax each time.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Fixture loading using bulk_create

[Jonas H.]

I started hacking the loaddata command to make use of the shiny new 
`bulk_create` code -- however, it seems that fixture loading (at least 
in its current incarnation) is incompatible to bulk inserts, for this 
reasons:


1. It's possible to have model objects overridden by fixtures.
   e.g. in modeltests/fixtures/fixtures/fixture1.json, there's a
   `fixtures.article` with pk=3, and in fixture2.json there's another
   pk=3 article which overrides that from fixture1.

   Of course this is not compatible with forced inserts.

2. pre_save/post_save signals are not sent.
   It seems like it's not documented anywhere that signals are sent
   during fixture loading but I think that for backwards compatibility
   reasons this behaviour should not be changed.

Any ideas on these issues?

Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Suppressed template errors in admin

[Jonas H.]

On 08/24/2011 05:44 PM, Ole Laursen wrote:

Hi!

I have a project where setting TEMPLATE_STRING_IF_INVALID has been
invaluable in finding problems (for various reasons). The caveat
mentioned in the docs don't kick in here because no templates are
inherited from elsewhere. Except the admin which unfortunately breaks
down in some places.

I realize there's been a big discussion before about the merits of
error suppression. No need to repeat that.

But is it really not possible to fix admin in the few places it's a
problem? Those I've seen so far have one-line fixes where you add an
empty variable to the context. Seems harmless to me. I realize
cluttering the template itself with a {% if var %} is ugly. No need to
go there.

I don't mind writing a patch for the cases that bother me, but won't
do it unless somebody is willing to apply it?


Ole




+1

This little ugly hack is incredibly useful during development:

# Hack
class RaiseMissingVariable(object):
def __mod__(self, val):
raise AttributeError("Missing %r" % val)

def __contains__(self, what):
return True
TEMPLATE_STRING_IF_INVALID = RaiseMissingVariable()

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Consistent exception handling in templates.

[Jonas H.]

On 07/08/2011 05:24 PM, Jonathan Slenders wrote:

My feeling is that we should silence TypeError, AttributeError and
KeyError in _resolve_lookup, but only when they occur one frame deep.


I agree with what you propose but AFAIK it's not easy to solve without 
doing frame analysis (I'd say that feature is not worth doing such 
hacks).  That's because of the way `hasattr` is implemented:


  def hasattr(obj, attr):
  try:
  getattr(obj, attr)
  return True
  except AttributeError:
  return False

This code directly mirrors the code found in CPython and implies that 
/hasattr "executes" properties/.


So there's no way to figure out whether an attribute exists without 
executing the code of properties.


Jonas

--
Django + MongoDB: http://django-mongodb.org

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Form Rendering API Proposal

[Jonas H.]

On 06/23/2011 02:11 PM, Harro wrote:

- Will the as_* methods on forms be deprecated? They seem to be a nice
shorter version then the new way to do it.


I'd rather provide a shorter version of {% form %} for built-in layouts:

{% form foobar 'table' %}
as shorthand for
{% form foobar 'forms/layouts/table.html' %}

This behaviour could even be extended to automagic template selection, 
so if you do {% form ... 'foo' %} Django searches for a 'foo.html' 
template in "project/current_app/templates/current_app/forms/layouts/", 
then "project/templates/forms/layouts/", falling back to 
"django/forms/templates/forms/layouts/" if none of those directories 
contains the desired file.  Your proposal doesn't tell anything about 
where form templates are searched, is something similar to this intended?


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Reshaping the "Contributing to Django" documentation

[Jonas H.]

"[...] try and duplicate it. If you can duplicate it [...]"


I'd stick with "reproduce" here because "duplicate" may be confused with 
"duplicate tickets".


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

[Jonas H.]

On 05/17/2011 04:48 PM, Jeremy Dunck wrote:

1) Use file modification timestamps for all model and test related files.
Advantages: simple, works.
Disadvantages: Triggers cache invalidation for changes not related to models
or tests


I think this is a pretty big win, even though it's not theoretically optimal.


Only for "does-it-still-work" sort of tests. Not for test-driven 
development, because your models and tests change all the time.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

[Jonas H.]

On 05/17/2011 01:55 PM, Ned Batchelder wrote:

On 5/16/2011 11:18 PM, Erik Rose wrote:

How about caching the test databases? The database state could be
cached after model setup (which takes some time if you've got lots of
them) + initial data fixture setup, and after the setup for each test
case (fixtures + setUp() method).

So, in the best case, no database setup is required at all to run
tests -- which encourages test driven development :-)

So that would be 11 separate DBs for our tests, and you'd just switch
between them? Interesting idea.


Erik: yep


I'd been thinking recently about this as well: when you consider all the
test runs, they're very repetitive. Every time the tests are run, they
go through the same set of steps: a) create database, b) install
fixtures, c) run tests. Steps a, b, and c take too long. Step c is what
we're really interested in, and almost always, steps a and b have the
same outcome as the last time we ran them. We all know what to do if an
operation takes too long and usually is the same as last time: cache its
outcome. The outcome in this case is the state of the database. Caching
it could be as simple as making a copy of the database after the
fixtures are installed, then using that copy to run tests.

The complications are: 1) in any interesting test suite, there isn't a
single outcome of a+b, because different tests will have different
fixtures and perhaps even different models, so a number of copies will
have to be captured. 2) As with any caching scheme, invalidation is
important and tricky. In the normal course of development, how will
these cached copies of the database be invalidated and recreated?
Perhaps this isn't so bad, it's roughly analogous to writing migrations,
which we know how to deal with.


Invalidation is what I'm unsure about too -- multiple ideas came to my 
mind, all involving some sort of Great Hash(tm):


1) Use file modification timestamps for all model and test related files.
Advantages: simple, works.
Disadvantages: Triggers cache invalidation for changes not related to 
models or tests


2) #1 but do hash the model definitions (at Python level)
Advantages: no cache invalidation on non-model changes.
Disadvantages: tricky, triggers cache invalidation for changes not 
related to tests


3) Hash the SQL generated for setup/fixtures. (step in right before the 
SQL is sent to the database)

Advantages: No false-positives, simple
Disadvantages: Does not eliminate the need for SQL generation and 
fixture parsing + model creation, so this might not be the "highest of 
highs" ;-)


Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

[Jonas H.]

On 05/14/2011 04:26 PM, Jonas H. wrote:

I believe there's no generalized way of creating databases in Django
now, so that would have to be added.


s/creating/copying/

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

[Jonas H.]

How about caching the test databases? The database state could be cached 
after model setup (which takes some time if you've got lots of them) + 
initial data fixture setup, and after the setup for each test case 
(fixtures + setUp() method).


So, in the best case, no database setup is required at all to run tests 
-- which encourages test driven development :-)


I believe there's no generalized way of creating databases in Django 
now, so that would have to be added.


I'd love to hack on that :-)

Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Field lookup types and backwards incompatibility

[Jonas H.]

On 05/11/2011 03:17 AM, Chris Beaven wrote:

3. Change lookups to be uppercase (start the process of deprecating the
current lowercase ones)



uagh, .filter(foo_CONTAINS=bar)?

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: NoSQL support

[Jonas H.]

On 04/28/2011 08:59 AM, Russell Keith-Magee wrote:

To be completely frank, from my perspective, the code is an unknown
quantity at this point. It *might* be fine -- but it might not, on
anything from a scale from "needs minor work" to "needs to be
rebuilt". I simply don't know, and any process that will lead to me
knowing requires me to spend a non-trivial amount of time reviewing
the code and it's branch.


For anyone who's interested, here's the complete diff of Django-nonrel 
against Django 1.3: http://paste.pocoo.org/show/379546/


I think all those changes could fit into ~10 concrete Trac tickets.

(That doesn't mean discussions won't consume a lot of time for everybody 
who's involved -- I just wanted give people an idea about kind and 
quantity of the code changes.)


Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [5-for-1] Review request

[Jonas H.]

On 04/25/2011 05:33 PM, Jacob Kaplan-Moss wrote:

On Mon, Apr 25, 2011 at 10:28 AM, Jonas H.<jo...@lophus.org>  wrote:

I'd like you to look into http://code.djangoproject.com/ticket/15696.


Done. Unfortunately, the result's probably not what you wanted. If
you'd like to re-use your review wuffie I'd happily take a look at
something else instead; just let me know.


That's nice, then please have a look at 
http://code.djangoproject.com/ticket/15799.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

[5-for-1] Review request

[Jonas H.]

Here's what I reviewed:

http://code.djangoproject.com/ticket/15783
http://code.djangoproject.com/ticket/15774
http://code.djangoproject.com/ticket/15801
http://code.djangoproject.com/ticket/15896
http://code.djangoproject.com/ticket/15895

I'd like you to look into http://code.djangoproject.com/ticket/15696.

Thanks!
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Right way to do formfield-less fields?

[Jonas H.]

On 04/10/2011 03:14 PM, Jonas H. wrote:

Hello!

What's the preferred way to do a custom model field that has no
corresponding formfield? Intuitively, I'd say raise a
NotImplementedError, but that would make the whole model unusable in the
admin even if that field is excluded from being displayed.

That's because excluding a field from the admin display does not mean
`Field.formfield(...)` is left alone, a call into `formfield` happens
during validation checks (`check_formfield` in admin/validation.py).

So, are fields without a formfield possible already? If yes, how to do
it? If not, would you accept a patch that makes the `ModelAdmin.exclude`
attribute validation not call into `Field.formfield`?

Cheers,
Jonas


*push*

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Right way to do formfield-less fields?

[Jonas H.]

On 04/10/2011 03:51 PM, Karen Tracey wrote:

Please ask questions about using Django on django-users. This list is for
discussing the development of Django itself.


Karen, thanks for the hint, but I'm perfectly aware of that. Actually, 
this *is* a question about developing Django.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Right way to do formfield-less fields?

[Jonas H.]

Hello!

What's the preferred way to do a custom model field that has no 
corresponding formfield? Intuitively, I'd say raise a 
NotImplementedError, but that would make the whole model unusable in the 
admin even if that field is excluded from being displayed.


That's because excluding a field from the admin display does not mean 
`Field.formfield(...)` is left alone, a call into `formfield` happens 
during validation checks (`check_formfield` in admin/validation.py).


So, are fields without a formfield possible already? If yes, how to do 
it? If not, would you accept a patch that makes the `ModelAdmin.exclude` 
attribute validation not call into `Field.formfield`?


Cheers,
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Forms Parent ,Sibling reference

[Jonas H.]

On 01/10/2011 06:39 PM, Daniel Corbe wrote:

Hi,
How to get reference of one field form from another field form when
both are in same form?



wrong list, head over to django-users

--
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal for inclusion of two additional default template tags.

[Jonas H.]

On 01/07/2011 05:11 PM, Jonathan S wrote:

** And a {% decorate %} template tag:
http://paste.pocoo.org/show/316593/


+1


{% macro %} avoids the usage of external template files. I think it's
not worth the overhead of using an external file, when someone has to
repeat small parts of code in a template.


+1, but did you actually do some benchmarking on this?

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Need help w/ sql.where.WhereNode tree

[Jonas H.]

On 12/03/2010 03:06 PM, Jonas H. wrote:

On 12/03/2010 10:04 AM, Waldemar Kornewald wrote:

You probably don't want to use those aliases. Instead, you should use
a separate alias namespace for
embedded object filters. Otherwise you'll have to deal with the
complex JOIN code in the backend unnecessary. Aliases for embedded
objects could just be simple integers that get incremented for each
field independently with each new filter() call that mentions the
respective field.


Well that's where I'm stuck. I have no idea about how I could know in
the `add_filter` method, called from `add_q` for every filter, whether
the filters already added to the where clause come from the same
`.filter` call or from another.


So I did something I'd call "code guessing" and finally got a working 
solution, although I'm not sure it breaks anything... could someone tell 
me whether it does?


diff -r a632a02c9ff2 django/db/models/sql/query.py
[snip]
@@ -1022,6 +1023,13 @@
  field, target, opts, join_list, last, extra_filters = 
self.setup_joins(

  parts, opts, alias, True, allow_many, can_reuse=can_reuse,
  negate=negate, process_extras=process_extras)
+ except PseudoJoin, e:
+ join = (alias, 'dummy', 'dummy', e.field.attname)
+ alias = self.join(join, reuse=can_reuse, always_create=True)
+ self.unref_alias(alias)
+ can_reuse.add(alias)
+ e.field.handle_join(self, parts, alias, lookup_type, value)
+ return
  except MultiJoin, e:
  self.split_exclude(filter_expr, LOOKUP_SEP.join(parts[:e.level]),
  can_reuse)
@@ -1333,6 +1341,8 @@
  self.update_dupe_avoidance(dupe_opts, dupe_col, alias)

  if pos != len(names) - 1:
+ if hasattr(field, 'handle_join'):
+ raise PseudoJoin(field)
  if pos == len(names) - 2:
  raise FieldError("Join on field %r not permitted. Did you 
misspell %r for the lookup type?" % (name, names[pos + 1]))

  else:

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Purpose of constant_time_compare?

[Jonas H.]

Hello out there,

what is the point of `django.utils.crypto.constant_time_compare`? I 
understand it takes O(n) time no matter what input it is feeded with, 
but of what avail is it?


Can the time spent in *one single string comparison* really make such a 
huge difference?


Confused,
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Need help w/ sql.where.WhereNode tree

[Jonas H.]

On 12/03/2010 10:04 AM, Waldemar Kornewald wrote:

You probably don't want to use those aliases. Instead, you should use
a separate alias namespace for
embedded object filters. Otherwise you'll have to deal with the
complex JOIN code in the backend unnecessary. Aliases for embedded
objects could just be simple integers that get incremented for each
field independently with each new filter() call that mentions the
respective field.


Well that's where I'm stuck. I have no idea about how I could know in 
the `add_filter` method, called from `add_q` for every filter, whether 
the filters already added to the where clause come from the same 
`.filter` call or from another.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DatabaseError swallowed?

[Jonas H.]

On 12/02/2010 04:27 PM, Dan Fairs wrote:

I was wondering whether the bug still exists in Python 2.x because I take it 
only AttributeError and TypeError should be ignored while calling __len__ in 
list() -- if that's the case, the bug is definitely still present.


True - I'll give it a try in 2.7 when I have a moment to grab and build it. 
That doesn't help us that much though, as 2.6 (which I'm running on, 2.6.1 to 
be precise) is a supported version.


I just checked, it doesn't exist in 2.7...


Anyway, I propose to call len() on those iterators, if possible, before calling 
list() because otherwise all bugs in backends will be swallowed.


That makes sense. I'd considered proposing storing any exception raised in 
__len__ on an instance variable to be re-raised later; but your solution is 
better, as you'd get better tracebacks.

Should I raise a bug for this behaviour? Working up a test and patch should be 
straightforward for this one, if we agree it's a bug that needs fixing (even if 
it's a workaround for the underlying Python issue).


... so even though it's not a Python bug but a Python "feature" I find 
it rather annyoing that list(x) swallows all TypeErrors in x.__len__ -- 
think about all the cases in that TypeErrors are raised! Wrong number of 
arguments, unsupported objects passed as argument e.g. to str/dict/list 
methods, ...


Regarding the fact that the calls to list() in QuerySet.__len__ is the 
central point to backend invocation I'm strongly for special-handling 
this case.


Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Need help w/ sql.where.WhereNode tree

[Jonas H.]

On 12/01/2010 08:04 AM, Waldemar Kornewald wrote:

In the end JOINs will be a rather complicated solution. I thought
that's why you wanted to extend the ORM directly to handle embedded
fields in a special way. That should be easier and cleaner.


That's what I'm up to, but I have to know how to reverse engineer 
.filter calls from the where tree, anyway.


The changes currently involved are (in django/db/models/sql/query.py):

Line 1022:
 try:
 ... = self.setup_joins(...)
 except MultiJoin, e:
 ...
+ except PseudoJoin, e:
+ e.field.handle_join(self, parts, alias, lookup_type, value)
+ return

Line 1339:
 if pos != len(names) - 1:
+ if hasattr(field, 'handle_join'):
+ raise PseudoJoin(field)
 if pos == len(names) - 2:
 ...

The problem now of course is that the alias tables are not changed in 
any way so the same alias is reused in each call to 'handle_join'.


I couldn't figure out what exactly I have to with that load of alias and 
join tables - I found the code way too complicated, in particular 
because if I got it right it's not unusual that aliases are "freed" 
every now and then and so on.


So this is where I'm asking the Django devs to give me an advice (or, 
even better, a copy-and-paste-ready solution :-) on how to correctly 
decide which alias to use.  (I already figured out how to make the alias 
be different for each 'add_filter' call, but obviously that's not what I 
need).


Thanks!
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DatabaseError swallowed?

[Jonas H.]

Hi Dan,

On 12/02/2010 03:43 PM, Dan Fairs wrote:

My gut feeling is that this boils down to this vastly simplified demonstration 
of how list() works:


class Foo(object):

...   def __len__(self):
... print 'len called'
... raise ValueError
...   def __iter__(self):
... return iter([1,2,3])
...

a = Foo()
list(a)

len called
[1, 2, 3]

Here, you can see that when converting to a list, Python calls __len__, and if 
that raises an exception, discards it and goes on to call __iter__.

So - my hypothesis (unproved, as I could benefit from someone with deeper ORM 
knowledge) is that the call to list() in my original test case calls 
QuerySet.__len__(), which ends up raising a DatabaseError (caused by an 
underlying database lock, the behaviour I'm actually testing for). Python's 
subsequent call to QuerySet.__iter__() succeeds, but ends up returning an empty 
iterator due to some pre-existing state *handwaving here*.

It's the handwaving bit I'm not sure about :). Does that hypothesis sound 
plausible? It seems to be borne out by the snippet below, where I've removed 
the underlying table:


I ran into this issue, too. My workaround was to place a

if hasattr(self._iter, '__len__'):
len(self._iter)

and the same for self.generator in QuerySet.__len__.

Btw, there's a bug report for this from 2009: 
http://bugs.python.org/issue1242657


I was wondering whether the bug still exists in Python 2.x because I 
take it only AttributeError and TypeError should be ignored while 
calling __len__ in list() -- if that's the case, the bug is definitely 
still present.


Anyway, I propose to call len() on those iterators, if possible, before 
calling list() because otherwise all bugs in backends will be swallowed.


Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Need help w/ sql.where.WhereNode tree

[Jonas H.]

Hello List!

I'm working on queries on embedded objects for Django-nonrel (more 
precisely, djangotoolbox) that will use JOIN-like syntax.


For this, I need to know how to distinguish
  .filter(spam__op=eggs, foo__op=bar)
from
  .filter(spam__op=eggs).filter(foo__op=bar)

in the .where tree used in django.db.models.sql.query.Query.

My first idea was to guess the original query expression from the where 
tree's structure, but I suspect that will work out for all kinds of 
queries (using Q objects in particular).


So -- is it possible? How do the SQL backends decide when to do one JOIN 
with two SELECTs or two JOINS with one SELECT each?


Thanks!
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Non-SQL database cache backend

[Jonas H.]

On 11/02/2010 01:09 AM, Russell Keith-Magee wrote:

   - The base class refactoring makes sense, although what you're
proposing isn't really just a 'BaseDB backend', but a 'backend that
has to implement culling'. The reason I can say this is that the logic
you have factored out is duplicated in the filesystem and local memory
backends (but not the memcache backend).


I have to admit I only looked at the database backend :-)


   - Regarding the test case for culling -- is there any reason that
the test case can't just set self.cache, rather than adding the
argument to the test method?


Of course not, I just didn't think of that.


Lastly, rather than handling this as two patches, I'm happy for this
to be submitted as a single patch under a "refactoring for the benefit
of non-db cache backends" moniker.


Here you go: http://code.djangoproject.com/ticket/14596

Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Non-SQL database cache backend

[Jonas H.]

On 10/29/2010 03:37 PM, Russell Keith-Magee wrote:

Writing a MongoDB (or any other non-SQL) cache backend shouldn't be a
particularly onerous task; after all, the queries are simple, and the
.The SQL backend is only 150 lines of code; a backend for any noSQL
backend should run to about the same line count.


Alright, here you go -- 100 lines :-)

http://github.com/django-mongodb-engine/mongodb-cache/blob/master/django_mongodb_cache/backend.py

Basically it's a 1:1 copy of the SQL backend with a somewhat different 
culling and serialization behaviour. Culling: Deletes entities sorted by 
expiration time (ascending).  Serialization: Uses pickle only for 
objects that can't be serialized as BSON.


I have attached two patches that I would like you to apply.

The first is purely cosmetic: It moves the 
`cache.backends.db.CacheClass.__init__` method in a 
`BaseDatabaseCacheClass` super class. Without that, my backend would 
have to derive from the SQL implementation class, which is confusing and 
not OOP-ish :-) (or re-implement all the dirty pseudo-model stuff, which 
isn't either)


The second patch adds a test case for cull_frequency == 0.

The two tests that check for not calling the model's 'default' callback 
more than once don't pass ('expensive_calculation' is called five 
times), and I don't know why.
Could you try to give any pointers how to fix this or where I should 
start debugging this issue?


I'm looking forward to any kind of feedback!

Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

diff -r d578c187d733 tests/regressiontests/cache/tests.py
--- a/tests/regressiontests/cache/tests.py  Sat Oct 23 11:31:45 2010 +0200
+++ b/tests/regressiontests/cache/tests.py  Mon Nov 01 15:12:00 2010 +0100
@@ -353,17 +353,18 @@
 self.assertEqual(self.cache.get('key3'), 'sausage')
 self.assertEqual(self.cache.get('key4'), 'lobster bisque')
 
-def perform_cull_test(self, initial_count, final_count):
+def perform_cull_test(self, initial_count, final_count, cache=None):
 """This is implemented as a utility method, because only some of the 
backends
 implement culling. The culling algorithm also varies slightly, so the 
final
 number of entries will vary between backends"""
+cache = cache or self.cache
 # Create initial cache key entries. This will overflow the cache, 
causing a cull
 for i in range(1, initial_count):
-self.cache.set('cull%d' % i, 'value', 1000)
+cache.set('cull%d' % i, 'value', 1000)
 count = 0
 # Count how many keys are left in the cache.
 for i in range(1, initial_count):
-if self.cache.has_key('cull%d' % i):
+if cache.has_key('cull%d' % i):
 count = count + 1
 self.assertEqual(count, final_count)
 
@@ -410,6 +411,10 @@
 def test_cull(self):
 self.perform_cull_test(50, 29)
 
+def test_zero_cull_frequency(self):
+cache = get_cache('db://%s?max_entries=30_frequency=0' % 
self._table_name)
+self.perform_cull_test(50, 18, cache=cache)
+
 class LocMemCacheTests(unittest.TestCase, BaseCacheTests):
 def setUp(self):
 self.cache = get_cache('locmem://?max_entries=30')
diff -r d578c187d733 django/core/cache/backends/db.py
--- a/django/core/cache/backends/db.py  Sat Oct 23 11:31:45 2010 +0200
+++ b/django/core/cache/backends/db.py  Sun Oct 31 18:57:55 2010 +0100
@@ -25,7 +25,7 @@
 self.managed = True
 self.proxy = False
 
-class CacheClass(BaseCache):
+class BaseDatabaseCacheClass(BaseCache):
 def __init__(self, table, params):
 BaseCache.__init__(self, params)
 self._table = table
@@ -45,6 +45,7 @@
 except (ValueError, TypeError):
 self._cull_frequency = 3
 
+class CacheClass(BaseDatabaseCacheClass):
 def get(self, key, default=None):
 self.validate_key(key)
 db = router.db_for_read(self.cache_model_class)

Non-SQL database cache backend

[Jonas H.]

Hi folks,

upon the recommendation on the django-nonrel mailing list, I'll re-post 
my original question here:


-- Begin original message --
I just tried to use MongoDB as cache backend (only for development ;-) 
but it failed because apparently the Django guys do hand-written SQL in 
the database cache backend, probably for the sake of performance.


Of course I could write a MongoDB cache backend myself -- which probably 
wouldn't be /that/ hard -- but isn't database abstraction the job for 
Django's database framework? [...]

-- End original message --

How do think about that? I'm sure a very thin abstraction layer would 
suffice. Or, if possible (I'm not /that/ deep into the Django ORM, so I 
can't tell), some small part of the ORM's query abstraction could be 
used so the database backend would not require a certain type of database?


Cheers,
Jonas

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.