Re: Automatic prefetching in querysets

[Malcolm Box]

Hi,

I think there's a potential to make this opt-in, and improve the out-of-box 
experience.

Summarising the discussion, it seems that the rough consensus is that if we 
were building the ORM from scratch, then this would be entirely sensible 
behaviour (with necessary per-QS ways to disable) - it would remove a 
common performance problem (N+1 queries), would improve areas where adding 
prefetch_related to queries is awkward, and in rare cases where it 
decreased performance there would be documented ways to fix.

So the main disagreement is about how to get there from here, and there's 
concern about three types of users:

1 - Existing, non-expert users whose sites work now (because otherwise they 
would have already fixed the problem) but who have lurking N+1 issues which 
will break them later
2 - Existing, non-expert users whose sites work now, but would have 
performance issues if this was enabled by an upgrade
3 - Existing, expert users who have already found and fixed the issues and 
who could therefore get no benefit but might suffer a performance 
degradation.

I'll assert that the size of these populations above is listed in roughly 
size order, with #1 being the biggest. This is a hunch based on most sites 
not having huge tables where N+1 becomes a problem - at least not until 
they've been running for a few years and accumulated lots of data...

There is another population that hasn't been considered - users starting 
django projects (ie people running django-admin startproject). Over time, 
this is by far the largest population. 

So would a sensible approach be:

- Feature is globally opt-in 
- startproject opts in for new projects
- Release notes mention the new flag loudly, and encourage people to try 
switching it on
- We add the debug tracing to help people find places where this setting 
would help - and encourage them to enable it globally before trying 
individual queryset.prefetch_related

Then over time, all new projects will have the new behaviour. Old projects 
will gradually upgrade - everyone in category 1 will hit the "make it work" 
switch the first time they see the warning / see a problem. Experts can 
choose how they migrate - as Adam points out, even experts can miss things.

Finally after a suitable warning period, this can become an opt-out feature 
and we arrive in the sunny world of an ORM that works better for all users.

Cheers,

Malcolm

On Wednesday, 16 August 2017 21:17:49 UTC+1, Aymeric Augustin wrote:
>
> On 15 Aug 2017, at 11:44, Gordon Wrigley  > wrote:
>
> I'd like to discuss automatic prefetching in querysets. Specifically 
> automatically doing prefetch_related where needed without the user having 
> to request it.
>
>
>
> Hello,
>
> I'm rather sympathetic to this proposal. Figuring out N + 1 problems in 
> the admin or elsewhere gets old.
>
>
> In addition to everything that was said already, I'd like to point out 
> that Django already has a very similar "magic auto prefetching" behavior in 
> some cases :-)
>
> I'm referring to the admin which calls select_related() on non-nullable 
> foreign keys in the changelist view. The "non-nullable" condition makes 
> that behavior hard to predict — I'd go as far as to call it non 
> deterministic. For details, see slide 54 of 
> https://myks.org/data/20161103-Django_Under_the_Hood-Debugging_Performance.pdf
>  and 
> the audio commentary at https://youtu.be/5fheDDj3oHY?t=2024.
>
>
> The feature proposed here is most useful if it's opt-out because it 
> targets people who aren't aware that the problem even exists — at best they 
> notice that Django is slow and that reminds them vaguely of a rant that 
> explains why ORMs are the worst thing since object oriented programming.
>
> It should kick in only when no select_related or prefetch_related is in 
> effect, to avoid interfering with pre-existing optimizations. It's still 
> easy to construct an example where it would degrade performance but I don't 
> think such situations will be common in practice. Still, there should be a 
> per-queryset opt-out for these cases.
>
> We may want to introduce it with a deprecation path, that is, make it 
> opt-in at first and log a deprecation warning where the behavior would 
> kick-in, so developers who want to disable it can add the per-queryset 
> opt-out.
>
>
> At this point, my main concerns are:
>
> 1) The difficulty of identifying where the queryset originates, given that 
> querysets are lazy. Passing objects around is common; sometimes it can be 
> hard to figure out where an object comes from. It isn't visible in the 
> stack trace. In my opinion this is the strongest argument against the 
> feature.
>
> 2) The lack of this feature for reverse one-to-one relations; it's only 
> implemented for foreign keys. It's hard to tell them apart in Python code. 
> The subtle differences, like return None vs. raise ObjectDoesNotExist when 
> there's no related object, degrade the developer 

AssertNumQueries and call stacks

[Malcolm Box]

Hi,

I'd like to open a discussion on bug #9363
<https://code.djangoproject.com/ticket/9363> and #22320
<https://code.djangoproject.com/ticket/22320> which are about improving
query logging to capture call stacks. The 9363 was marked wontfix, but I
don't understand the reason given.

The use case is simple: when using assertNumQueries in a testcase, it would
be very useful to get the stack trace for where the queries were executed
in the output, as well as the queries.

The original bug was accepted as a good idea by Claude Peroz but required a
fix to #22320 <https://code.djangoproject.com/ticket/22320> first. This has
now been fixed, removing the concern about memory usage.

The wontfix reason given by Aymeric was "use Django Debug Toolbar" and "the
code looks bad". DDT is a great tool, but doesn't solve this use case, and
the code is a three-line patch (and 5 lines of test code)

https://github.com/mbox/django/commit/c0a5fa22770b90d3ad0fd24ee80b93bebe03fbec

Can this be reconsidered for re-opening? I'm happy to improve the patch if
that's needed.

Cheers,

Malcolm

-- 
Malcolm Box

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAF3R4sUfrPj7vf5ir%2BGJfhpk7rrmiq%2B%3DhnRL0jevgyBrP6pqYw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Extend support for long surnames in Django Auth

[Malcolm Box]

I've opened ticket 26993 to track this - 
https://code.djangoproject.com/ticket/26993#ticket

On Friday, 29 July 2016 12:15:43 UTC+1, Raony Guimaraes Corrêa Do Carmo 
Lisboa Cardenas wrote:
>
> Hello everyone,
>
> For a long time I was having problems to login to djangopackages.com 
> using my github account (pydanny/djangopackages#338 
> ). After 
> investigating I discovered the problem was because my surname is longer 
> than 30 characters. I don't know why both first_name and last_name fields 
> have the same size limit of 30 characters in Django. That doesn't sound 
> very reasonable.
>
> I'm sure there are other people on the same situation and this already 
> happened with me trying to login in other django websites.
>
>
> [image: selection_086] 
> 
>
>
> Tim Graham suggested I should first ask on this maillist (
> https://github.com/django/django/pull/6988#issuecomment-235945422) to see 
> if there is consensus to make the change.
>
> I would like to ask your opinion about an increase from 30 to 60 
> characters on last_name field so that my login and others won't break again 
> in the future. I can create a Trac ticket if the response is positive.
>
> Kind Regards,
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e46bef3f-3680-470e-8fe0-85145872f9b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Extend support for long surnames in Django Auth

[Malcolm Box]

A setting seems like a dangerous option here, since changing it would cause
a DB migration to be required.

It's not *that* hard for us to find a value and apply it. Any user that
really wanted something different could create their own migration to
change the default - but "leave it to the user" still means picking a value
for the sane default...

Cheers,

Malcolm

On 2 August 2016 at 14:01, Lee Trout <leetr...@gmail.com> wrote:

> I know there's always resistance to adding more settings but this seems
> like a candidate for a value in a setting with a sane default that a user
> could quickly and easily change.
>
> On Tuesday, August 2, 2016, James Pic <james...@gmail.com> wrote:
>
>> Thanks for your reply Aymeric. If I understand correctly the best way to
>> approach this, besides increasing the current limits - which I've had to do
>> myself a few times - is to create a separate app providing a custom model
>> with an ArrayField for name (sorting) and a migration script, and let time
>> tell if this is more useful than the current approach and wether it should
>> become default or not after a long-enough while ?
>>
>> Again, thanks for your reply, every time I read messages from engineers
>> like you it makes me a better one myself and I'm sure it's the case for
>> most other persons. I understand it can sometimes be hard for you - and
>> other experienced core devs- to have to deal with us, so I really want to
>> show my appreciation and love and I think I can speak for everyone of us
>> when I say thank you for your contribution and your sharing and making
>> everyone of us better every time you take some time for us.
>>
>> Keep up the great work
>>
>> Best regards
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/CALC3KaeNyMhAeG0_4L_j0KDg5hV_eAzucy42G1xiLMmfyHOtVQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/django-developers/CALC3KaeNyMhAeG0_4L_j0KDg5hV_eAzucy42G1xiLMmfyHOtVQ%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/h98-oEi7z7g/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAABi-DoVpZoTCKL9Mw-J%3DrSJhHFR1jo-WCr9gTVAa%2BBKWbsCjQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CAABi-DoVpZoTCKL9Mw-J%3DrSJhHFR1jo-WCr9gTVAa%2BBKWbsCjQ%40mail.gmail.com?utm_medium=email_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Malcolm Box

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAF3R4sVikNLgr1DVtKw_3ps7gFuHrZbO3K1Zhv14ZuUzM5zR0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Extend support for long surnames in Django Auth

[Malcolm Box]

On Monday, 1 August 2016 13:56:55 UTC+1, Aymeric Augustin wrote:
>
> > On 30 Jul 2016, at 23:15, Donald Stufft  
> wrote: 
> > 
> > See #6 of 
> https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/
>  
>
> I’m aware of this article. It's a entertaining read but, unlike the W3 Q 
> mentioned earlier, it doesn’t contain actionable advice for designing a 
> generic system that won’t perform too poorly in many use cases when you 
> have no idea of what these use cases will be. 
>
>
Having read the W3C Q carefully, the relevant comment on field lengths is 
"avoid limiting the field size for names in your database". There is no 
reference to 60 characters being "enough".
 

> Last names containing over 30 characters are sufficiently common — likely 
> tens of millions of people at this time — to deserve consideration. That’s 
> where this thread started. Let’s not block an easy win for these tens of 
> millions because the general problem is intractable. Besides, a last_name 
> field is already a severe simplification, as we all know. 
>
> Last names containing over 100 characters are sufficiently uncommon to be 
> the subject of trivia articles on the Internet. I’m absolutely certain that 
> no website has tens or thousands of millions of last names over 100 
> characters; in fact, not even tens of such names. 
>
> If someone has access to real-life stats from a very large database of 
> names in a country that has long last names that could help us make an 
> optimal decision. 
>
>
Aren't we in danger of premature optimisation here? The field lengths are 
currently 30 characters. Even if we expanded it to 255, we would be adding 
at worst 255 bytes to the storage requirements for a user. If a site has 1 
million users, that's 250MB of disk space - which for a site with 1M users 
is unlikely to be significant, let alone the main driver of disk usage. 
After all, with 1M users, you'd only need each of them to do something 
requiring a 200 byte record to use the same amount of space.

For sites with far less users, it's proportionally much less of a problem - 
and those sites are the majority, and hence will benefit most from being 
able to deal with more user's names, while suffering the least cost. Sure, 
we'd be wasting some space on disks - but we'd be enabling Django to serve 
more of the world's population (and allowing sites to just use last_name as 
a full name if they don't want the fun of a custom user model).

So if we don't want to go to full TextField for the username, at least 
let's go to the biggest number possible (currently 255, thanks MySQL).

Premature optimisation is the root of all evil...

Cheers,

Malcolm

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/161214ba-560e-4ef6-ada8-4eb0067b3181%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Converting ModelAdmin system checks to run against instances rather than the class

[Malcolm Box]

Hi,

Raising this here as suggested by Tim on this 
bug: https://code.djangoproject.com/ticket/25374, with discussion also on 
django users 
here: https://groups.google.com/forum/#!topic/django-users/lsDP5oUWOsw

The underlying bug is that ModelAdmin checks are run against the class not 
the instance - which means that checks can fail when the code actually 
works. This sort of false positive feels bad to me.

The proposed PR changes the ModelAdmin checks to validating the created 
instance that will be registered rather than the ModelAdmin class itself. 
This mirrors how Django actually uses the ModelAdmin objects, and means 
that validation checks that previously threw false positives now pass.

Does this change make sense, or is there something subtle that I'm missing? 
More broadly, am I right in thinking that false-positive system check 
errors are a bad thing, and so we should aim to minimise/eliminate them, 
even at the cost of potential false negatives?

Cheers,

Malcolm




-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/411901cb-7e72-4a94-99ef-8d6cbc24f4d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Malcolm Box]

At the risk of re-opening this thread, I have just run into a very similar 
situation (see https://code.djangoproject.com/ticket/25374) and find that 
having the errors printed to the console is actually helpful - as I've had 
to silence errors to get the correct code to run, seeing if there's any 
*other* errors being "accidentally" silenced is helpful.

It seems there's actually two behaviours being conflated in 
"SILENCE_SYSTEM_CHECKS":

* Ignoring the error, so that the code will run (ie downgrade errors to 
warnings)
* Not printing the message

It's useful to be able to do the two things separately - I may want to 
downgrade an error because it's not one in my codebase, but still see if 
new instances of it creep in. Or I might want to not see any of a 
particular warning printed to the console.

Could we split this to:

IGNORED_SYSTEM_CHECKS

and 

SILENCED_SYSTEM_CHECKS


Malcolm

On Thursday, 27 August 2015 14:09:06 UTC+1, Tim Graham wrote:
>
> I created a ticket and pull request for the silencing of error/critical 
> messages:
> https://code.djangoproject.com/ticket/25318#ticket
> https://github.com/django/django/pull/5197
>
> On Tuesday, August 25, 2015 at 11:03:18 AM UTC-4, Carl Meyer wrote:
>>
>> On 08/25/2015 08:57 AM, Marcin Nowak wrote: 
>> > Well, I'm not sure now, because Tim wrote that it was a design decision 
>> > and it is well documented. 
>>
>> I still think we should change that design. It is simply wrong to have a 
>> setting called `SILENCED_SYSTEM_CHECKS` that fails to actually _silence_ 
>> the check IDs listed. 
>>
>> > Also please note that silencing specific errors is not the best 
>> solution. 
>> > In the first case E116 may be untrue and may be silenced, but for other 
>> > models this error may be true and shouldn't be silenced at all. 
>> > Silencing all E116 will give us messy information about state. 
>>
>> That's why I think in this case the best solution is to actually fix the 
>> error, not silence it; Django should be able to find a find a field even 
>> if it's monkeypatched-in, as long as the monkeypatch happens in time. 
>>
>> Carl 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dc4e410c-708f-472b-bde2-49a45ad4e39d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Terms for database replication

[Malcolm Box]

FWIW, I think the main objection is to the word "slave", not to "master". 
Otherwise we'll be renaming the git branches soon...

So "master" / "replica" would work.

Malcolm

On Thursday, 5 June 2014 16:26:07 UTC+1, Justin Holmes wrote:
>
> I think I agree that "primary" is a bad choice.  Can you suggest something 
> other than master?  Something that will address the concerns posed in the 
> past two threads?
>
> I saw that someone suggested "leader" and "follower" - I haven't thought 
> through whether I find this more palatable.
>
>
> On Thu, Jun 5, 2014 at 11:15 AM, Cal Leeming [Simplicity Media Ltd] <
> cal.l...@simplicitymedialtd.co.uk > wrote:
>
>> For once, I'm going to +1 you Tom.
>>
>> Cal
>>
>>
>> On Thu, Jun 5, 2014 at 2:52 PM, Tom Evans > > wrote:
>>
>>> Please revert this change as soon as possible.
>>>
>>> If the project has become so PC sensitive that the word "slave" is no
>>> longer permitted to be uttered, then "replica" is an alternate term,
>>> but "primary" is not.
>>>
>>> Have you ever set up "primary-primary replication"? No, neither have
>>> I. Master-master replication is common, please do not take it upon
>>> yourselves to re-program our vocabulary.
>>>
>>> I have a primary master, I do not have a primary primary!
>>>
>>> Cheers
>>>
>>> Tom
>>>
>>> --
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com .
>>> To post to this group, send email to django-d...@googlegroups.com 
>>> .
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BUa81AruHH7mRK9SyQCABQrprPDd7sdTRrNN0_DpNPOg%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAHKQagGa1WE8yH2xpf0DrKv9i_m9%2B6akcDTHexeUZmq%3DQZHUuA%40mail.gmail.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Justin Holmes
> Chief Chocobo Breeder, slashRoot
>
> slashRoot: Coffee House and Tech Dojo
> New Paltz, NY 12561
> 845.633.8330 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0e3f8c99-38b6-424e-9f2c-7c0fe569b6a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Changing development server threads type.

[Malcolm Box]

On Wednesday, 4 June 2014 17:41:53 UTC+1, Ramiro Morales wrote:
>
> For this particular change I'd go with what Moayad proposes but 
> without the backward compatibility command line switch (assuming it 
> actually enhance the reloadind responsiveness). 
>
>
>
I've tried the 1-liner version here on Mac OS X 10.9 and it provides a 
massive speedup to reload. I haven't noticed any adverse effects.

So I'd vote wholeheartedly for this change.

Malcolm 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5641d37d-e19d-41ab-8aff-44f115e650d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Feature request: ttl method for cache

[Malcolm Box]

Ticket filed and pull request for fix
https://code.djangoproject.com/ticket/22606


On 9 May 2014 10:38, Malcolm Box <malc...@tellybug.com> wrote:

>
> On Thursday, 8 May 2014 16:59:55 UTC+1, Piotr Gosławski wrote:
>>
>> I was fixing my little helper function to behave more like Sean's and I
>> think I've found a bug in locmem. Could you please take a look at this:
>>
>> Either I'm missing something or has_key() is not working correctly for
>> keys without expiration time.
>>
>
> I think you're right - this is related to this bug:
> https://code.djangoproject.com/ticket/22495 and caused by the same patch.
>
> File a ticket, and I'll have a look at fixing.
>
> Malcolm
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/nwZBw64cD4c/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/956bfca6-3ef7-4b4c-ac91-970628ec7d4b%40googlegroups.com<https://groups.google.com/d/msgid/django-developers/956bfca6-3ef7-4b4c-ac91-970628ec7d4b%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Malcolm Box
CTO & Co-Founder, Tellybug  http://tellybug.com
malc...@tellybug.com   +44 7766 990123@malcolmbox

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAF3R4sVVpHxBXmB%3D1pZUGv%2BpgvmxbgWxjZCo19OLELFuQw%2BtqQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Feature request: ttl method for cache

[Malcolm Box]

On Thursday, 8 May 2014 16:59:55 UTC+1, Piotr Gosławski wrote:
>
> I was fixing my little helper function to behave more like Sean's and I 
> think I've found a bug in locmem. Could you please take a look at this:
>
> Either I'm missing something or has_key() is not working correctly for 
> keys without expiration time.
>

I think you're right - this is related to this 
bug: https://code.djangoproject.com/ticket/22495 and caused by the same 
patch.

File a ticket, and I'll have a look at fixing.

Malcolm

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/956bfca6-3ef7-4b4c-ac91-970628ec7d4b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Minor feature: Make TestCase pass the testcase instance to self.client's constructor

[Malcolm Box]

On Sat, Jan 5, 2013 at 9:11 AM, Shai Berger <s...@platonix.com> wrote:

> On Friday 04 January 2013, Malcolm Box wrote:
> >
> > The general pattern I want to implement is have a test client that makes
> > assertions about all the requests made during a set of tests. For
> example,
> > it could check that every get() returned cache headers, or that
> > content_type is always specified in responses. Or that the response has
> > certain HTML, uses certain templates etc - ie all of the assertion
> testing
> > goodness in django.test.TestCase.
>
> 

> >
> > def test():
> >r = self.client.get(...)
> >self.check_response(r)
> >
> > but this is error prone, verbose etc etc.
> >
> > The right thing is that within a test suite, the get()/post() etc to do
> the
> > checks for me - and so it should be possible to create a testclient that
> > does this, and be able to use this testclient in various test suites.
> >
>
> No, you're mixing concerns.
>
> > Is there a simple, clean way to solve this that I'm missing?
> >
>
> class MyTestCase(TestCase):
>
> def check_response(self, response):
> self.assertContains(response, )
>
> def checked_get(self, *args, **kw):
> r = self.client.get(*args, **kw)
> self.check_response(r)
> return r
>
> class SpecificTest(MyTestCase):
>
> def test():
> r = self.checked_get(...)
> ...
>
> That's how I would do it.
>
>
Which is lovely, right up to the point where someone writes a test with:

self.client.get()

Extremely likely to occur (in fact 100% probable in any sizeable test
suite), won't show up as an error and yet suddenly the tests aren't testing
what they should be.

I don't understand your "mixing concerns" argument - the concern of the
TEST client should be testing just as much as it's the concern of the
TESTcase and TESTsuite. The hint is in the name :)

Now if the test client was truly decoupled (ie expected to be used in other
places) then I'd totally agree with you/Russ about mixing concerns and
coupling. But AFAIK it's not - it's explicitly a utility that does all
sorts of jiggery-pokery to support testing. Adding a parameter to the
initialiser seems a very minor sin (if a sin at all) in comparison.

However, if I'm not convincing anyone I'll shut up about this - the "fix"
is simply:

class Tests(TestCase):
  def setUp(self):
 self.client = MyBetterTestClient(self)

(although this does then require any derived test case to remember to call
super(Derived, self).setUp(*args, **kw))

I'd just have liked to be able to do that using the existing machinery:

class Tests(TestCase):
   client_class = MyBetterTestClient


Cheers,

Malcolm

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Minor feature: Make TestCase pass the testcase instance to self.client's constructor

[Malcolm Box]

On Wednesday, January 2, 2013 11:58:04 PM UTC, Russell Keith-Magee wrote:

>
> On Thu, Jan 3, 2013 at 1:56 AM, Malcolm Box <mal...@tellybug.com
> > wrote:
>
>> Hi,
>>
>> When creating self.client in TestCase, it would be very useful if the 
>> testcase instance was passed to the client. 
>>
>> I'm using a replacement client class that does various validation checks, 
>> so wants to use assert* functions on TestCase, thus takes a testcase 
>> instance as a parameter at creation time. However this means it can't be 
>> used as the client_class attribute on TestCase, as this is instantiated 
>> with no parameters (
>> https://github.com/django/django/blob/master/django/test/testcases.py#L475
>> )
>>
>> There are work-arounds, but it feels to me like a reasonably generic 
>> requirement that a test client may want to refer to the test case it's 
>> being used with. I think the change can be made largely backwardly 
>> compatible by changing to:
>>
>> self.client = self.client_class(testcase=self)
>>
>> which would only break an existing replacement client class that had an 
>> __init__ method without **kwargs.
>>
>> I'm happy to code this up, but wanted to check for any objections first.
>>
>
> Personally, I'm suspicious of any "A owns B, but B knows about A" 
> relationships. There are certainly occasions when they're required, but 
> whenever they pop up, it's generally an indication of a set of interfaces 
> that are closely coupled.
>
 

> In this case, I'm not sure I see why this coupling is required. A test 
> case is a test case. A test client is a test client. Their concerns are 
> completely separate (evidenced by the fact that you can have a test case 
> without any client usage, and vice versa). I very much see the test client 
> as a utility tool for the test case -- really not much more than a 
> convenient factory for requests -- not an integral part of a test case. 
>
>
I don't disagree - and this feature wouldn't require any test client to 
care about the testcase. 

Your feature request seems to be stem entirely from the fact that you want 
> to invoke assertions in the test client code itself -- something that 
> you're doing because you've got a bunch of extensions in your test client. 
> I'll leave it up to you to determine if this is the right approach for your 
> own project, but I'm not convinced it's a general requirement that warrants 
> binding the test client to the test case. 
>

The general pattern I want to implement is have a test client that makes 
assertions about all the requests made during a set of tests. For example, 
it could check that every get() returned cache headers, or that 
content_type is always specified in responses. Or that the response has 
certain HTML, uses certain templates etc - ie all of the assertion testing 
goodness in django.test.TestCase.

My concrete use case is a JSONClient that adds a json_get / json_post 
methods which makes sure to setup content_type etc on the call, and then 
validates that what came back was valid JSON.

The simple, wrong way is to do:

def check_response(self, response):
   self.assertContains(response, )
   

def test():
   r = self.client.get(...)
   self.check_response(r)

but this is error prone, verbose etc etc. 

The right thing is that within a test suite, the get()/post() etc to do the 
checks for me - and so it should be possible to create a testclient that 
does this, and be able to use this testclient in various test suites.

Is there a simple, clean way to solve this that I'm missing?

Malcolm

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/LNtkPS3EcdsJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Minor feature: Make TestCase pass the testcase instance to self.client's constructor

[Malcolm Box]

Hi,

When creating self.client in TestCase, it would be very useful if the 
testcase instance was passed to the client. 

I'm using a replacement client class that does various validation checks, 
so wants to use assert* functions on TestCase, thus takes a testcase 
instance as a parameter at creation time. However this means it can't be 
used as the client_class attribute on TestCase, as this is instantiated 
with no parameters 
(https://github.com/django/django/blob/master/django/test/testcases.py#L475)

There are work-arounds, but it feels to me like a reasonably generic 
requirement that a test client may want to refer to the test case it's 
being used with. I think the change can be made largely backwardly 
compatible by changing to:

self.client = self.client_class(testcase=self)

which would only break an existing replacement client class that had an 
__init__ method without **kwargs.

I'm happy to code this up, but wanted to check for any objections first.

Cheers,

Malcolm

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/j2agzjXZajsJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.