Re: Resetting cache gracefully at the end of every test?

[Jeremy Dunck]

On Fri, Jul 1, 2011 at 12:27 PM, Jim Dalton  wrote:
> Awesome feedback, thanks.
>
> On Jul 1, 2011, at 10:01 AM, Jeremy Dunck wrote:
>
...
>> If I were to do this, I wouldn't have cache shared between tests or
>> environments, so I'd be tempted to use flush.
>
> I'm not totally sure if I'm following you on this point?

What I mean is, the scenario you described doesn't affect me, because
flushing rather than deleting specific keys would be fine to me.  I
guess, generally, I meant to point out that presuming your scenario
applies to the community is a bad point to start from.  More to the
point, a solution for core should be either an extension point, or a
solution that all can agree is good commonly.

>> Rather than "test_" and bookkeeping on key names, the key prefix was
>> the hostname + timestamp from the beginning of the run, so that test
>> keys would never collide?
>
> What I like about this idea is it avoids the bookkeeping and monkey-patching 
> nonsense. What I don't like about it is that it feels like you could 
> potentially be adding a lot of junk to your cache. A test run might end up 
> hitting it with many values, each unique. A requirement I'm sort of working 
> with in my mind here is that the cache returns to the "pristine" state it was 
> in prior to the test run.
>

That's true about collecting garbage.  It would matter for filesystem
cache, I guess.  I don't much care about that, since cache for me is
either memcache (which dumps LRU entries) or dummy.  As far as
returning to "pristine", hmm.  If that's the case, I think you're
arguing for setUp and tearDown to do that setup.

>> Or, what if we just had hooks called from setUp and tearDown, either
>> NotImplemented methods, or signals?
>>
>> Book-keeping with monkey-patched methods sounds fiddly to me; consider
>> the case where celery pokes keys in, but our bookkeeping doesn't know
>> about it?
>
...
> I"m not quite sure what you mean about celery poking keys in. Maybe you can 
> elaborate a bit? To be clear I'm not sure we could keep the actual underling 
> cache entirely pristine if you were using it in different ways from different 
> systems; the goal would really be to ensure calls using the cache backend API 
> are rolled back with every test.

What I meant to point out is that the test process, which is doing the
book-keeping, may not have visibility to all of the key cleanup that's
needed; a common use case of that is that celery (an out-of-process
task queue commonly used with Django) does some work and pushes some
data into cache in response to an async.  I'm arguing that if you want
a clean workspace, how you get there depends on both how keys are
generated and what the cache backend is.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Resetting cache gracefully at the end of every test?

[Jim Dalton]

Awesome feedback, thanks.

On Jul 1, 2011, at 10:01 AM, Jeremy Dunck wrote:

> Well, I think you forgot all the other low-level cache options, like
> .incr, .add, etc.?

Yep, forgot those so they would conceivably need to be tracked.

> 
> If I were to do this, I wouldn't have cache shared between tests or
> environments, so I'd be tempted to use flush.

I'm not totally sure if I'm following you on this point?

> 
> Rather than "test_" and bookkeeping on key names, the key prefix was
> the hostname + timestamp from the beginning of the run, so that test
> keys would never collide?

What I like about this idea is it avoids the bookkeeping and monkey-patching 
nonsense. What I don't like about it is that it feels like you could 
potentially be adding a lot of junk to your cache. A test run might end up 
hitting it with many values, each unique. A requirement I'm sort of working 
with in my mind here is that the cache returns to the "pristine" state it was 
in prior to the test run.

> 
> Or, what if we just had hooks called from setUp and tearDown, either
> NotImplemented methods, or signals?
> 
> Book-keeping with monkey-patched methods sounds fiddly to me; consider
> the case where celery pokes keys in, but our bookkeeping doesn't know
> about it?

I know what you mean about it being fiddly, but I'm kind of envisioning 
something not all that different from what goes on with Template.render() 
settings.EMAIL_BACKEND, or the database backends during test setup and 
teardown. The bookkeeping required would in truth be nothing more than a set() 
where you add keys. The monkey patching would entail nothing more than a line 
to add the passed keys to that set and then a call to the original function. 
I'm just not sure it's much more fiddly than the fiddling we already do for 
those examples is what I'm saying.

I"m not quite sure what you mean about celery poking keys in. Maybe you can 
elaborate a bit? To be clear I'm not sure we could keep the actual underling 
cache entirely pristine if you were using it in different ways from different 
systems; the goal would really be to ensure calls using the cache backend API 
are rolled back with every test.

Anyhow thanks once again for the feedback and the critical thoughts.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Resetting cache gracefully at the end of every test?

[Jeremy Dunck]

On Fri, Jul 1, 2011 at 10:54 AM, Jim D.  wrote:
> This issue came up again for me recently: Because the cache is not reset or
> flushed after every test, cache values linger there. This can create
> headaches when writing tests.
> I would like to propose that we treat the cache backend(s) like database
> backends during the test run. Specifically, I propose that we:
> * Prefix "test_" before the KEY_PREFIX in each backend during global setup.
> This is equivalent to adding the test_ prefix to DB names. In doing this we
> prevent cache calls during test form overwriting possible legitimate values.
> * Monkey patch cache.set() (and cache.set_many()) such that we record all
> keys used to set values during a test.
> * Use cache.delete_many() on the list of keys we record being set during
> test teardown. This will restore the cache to its previous state after each
> test run. It's better than flush() which clears the whole cache and is too
> destructive to be run during testing.
> I don't think it needs to be more complicated than this.

Well, I think you forgot all the other low-level cache options, like
.incr, .add, etc.?

If I were to do this, I wouldn't have cache shared between tests or
environments, so I'd be tempted to use flush.

Rather than "test_" and bookkeeping on key names, the key prefix was
the hostname + timestamp from the beginning of the run, so that test
keys would never collide?

Or, what if we just had hooks called from setUp and tearDown, either
NotImplemented methods, or signals?

Book-keeping with monkey-patched methods sounds fiddly to me; consider
the case where celery pokes keys in, but our bookkeeping doesn't know
about it?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Design Czar

[Jacob Kaplan-Moss]

On Fri, Jul 1, 2011 at 12:07 AM, Victor Hooi  wrote:
> Has there been any recent news?

If you watch the commit timeline
(https://code.djangoproject.com/timeline) you might have seen a
handful of UI cleanups that've done in over the past few months;
Idan's contributed to those. I can't recall 'em all off the top of my
head, but one example is the multi-column sorting in the admin now on
trunk.

There's also some... stuff... going on behind the scenes I'd rather
not talk about just yet. Not because it's confidential or anything,
but because I'm superstitious and don't like talking about plans until
they're actually implemented.

Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Resetting cache gracefully at the end of every test?

[Jim D.]

This issue came up again for me recently: Because the cache is not reset or 
flushed after every test, cache values linger there. This can create 
headaches when writing tests.

I would like to propose that we treat the cache backend(s) like database 
backends during the test run. Specifically, I propose that we:

* Prefix "test_" before the KEY_PREFIX in each backend during global setup. 
This is equivalent to adding the test_ prefix to DB names. In doing this we 
prevent cache calls during test form overwriting possible legitimate values.
* Monkey patch cache.set() (and cache.set_many()) such that we record all 
keys used to set values during a test.
* Use cache.delete_many() on the list of keys we record being set during 
test teardown. This will restore the cache to its previous state after each 
test run. It's better than flush() which clears the whole cache and is too 
destructive to be run during testing.

I don't think it needs to be more complicated than this.

I checked around to see if anyone had opened a ticket related to this or had 
previous discussions on the subject but couldn't find anything. I did 
briefly touch upon this with Russ in a comment thread 
on https://code.djangoproject.com/ticket/15026 and I noticed he made a 
comment in this vein on Django 
Users: https://groups.google.com/forum/#!topic/django-users/OOGO-3MIO_c . If 
there is something already open on this or a decision has already been made, 
please do feel free to point me in the right direction.

Otherwise, if there is interest I'd be happy to open a ticket and get a 
patch started.

Thanks

Jim

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/dyMCEKilRuMJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Reversing translated urls in templates

[Ramiro Morales]

On Sat, Jun 25, 2011 at 6:58 AM, Jannis Leidel  wrote:
>
> As I said initially when we discussed this on IRC, I believe a "i18n_url" is 
> good enough
> and would fit well with the other i18n related template tags.
>
> Besides the ambiguity of a potential keyword argument "lang", I'd like to 
> mention that
> we specifically chose to use "i18n_patterns" as the name for the function 
> that returns
> i18n-enabled URL patterns to be explicit about when a URL should be 
> translated. I believe
> this pattern should also be applied to the template tag which a user might 
> use to point to
> one of those translatable URLs.
>

On Sat, Jun 25, 2011 at 2:50 AM, maxk  wrote:
> What do you think about something like below
>
>  {% url viewname k1=v1, k2=v2 with lang=lang %}

Briefly discussed this via IRC th Jannis. I'm +1 on keeping the new tag
in the i18 ttag lib for the reasons exposed by Jannis. So a prefix like
'i18n_' would fit well IMHO. +1 on this.

Adding that functionality to the i18n tag can also work, I think.
Especially now that Jannis makes me see it wouldn't cause loading of the
i18n machinery for people with USE_i18N=False. But I'd like to
completely avoid the possibility of clashing with existing 'lang'
arguments and I'm not 100% convinced by the "with lang='xx'" part of the
proposed syntax. It is a hybrid between our 'old' "with var as alias"
and the new (from future) "var=alias" form. I'm -0 on this option


-- 
Ramiro Morales

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: MultipleObjectMixin and Pagination

[Andrew Ingram]

Hi Julien,

I'm already involved on that ticket, the pagination issue is part of what's 
holding me back from contributing a patch. So I was looking to get some 
discussion about whether it's something that needs to be solved in core. In 
light of what you've said, I'm trying to think of a more straightforward 
test case that isn't tied to my ModelFormSetView.

- Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/0-7LrGdIEo8J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.