Re: Set a reasonable default for EMAIL_TIMEOUT

[Federico Capoano]

Hi everyone,

do you think you are against on opening an issue for this?
I had already opened an issue 
<https://code.djangoproject.com/ticket/34167#ticket> which was marked as 
wontfix and I was asked to open this discussion here.

Best regards
Federico

On Monday, November 21, 2022 at 5:31:27 PM UTC-3 Federico Capoano wrote:

> Hi there,
>
> yes I can add EMAIL_TIMEOUT in my applications and I am already doing that.
>
> The reason I wanted to bring this up here is that I care about Django.
> This behavior goes against a well known concept of the Unix Philosophy 
> which is to  "Write programs which fail in a way that is easy to diagnose".
>
> I believe we can all agree that any potentially long running operation 
> should have a default timeout after which the operation fails and a trace 
> is left on a log somewhere for the system administrators to find
> I can't see the benefit in not failing, not leaving log traces and keeping 
> an app blocked indefinitely.
>
> What are the benefits vs the downsides of such a change?
>
> Benefit: any issue in production systems will be quickly understood and 
> dealt with.
> Downsides? Backward compatibility? Systems relying on the default timeout 
> being set elsewhere? Anything else?
>
> Even if we allow configuring different email backends, I am still 
> wonderfing: why do we think it's good to have an email sending operation 
> which can potentially stall indefinitely?
>
> Best regards
> Federico
> On Friday, November 18, 2022 at 11:28:14 AM UTC-3 Michiel Beijen wrote:
>
>> Hi Mariusz, 
>>
>> > Op 17 nov. 2022, om 18:18 heeft Mariusz Felisiak  
>> het volgende geschreven: 
>> > 
>> > "Why is there no default for EMAIL_TIMEOUT?" 
>> > The Django's SMTP backend is a wrapper around smtplib.SMTP/SMTP_SSL and 
>> we only pass the "timeout" parameter to the smtplib API, so I'd first ask 
>> why there is no default value for a timeout parameter in smtplib. 
>>
>> Well, Django’s docs say: 
>>
>> > If unspecified, the default timeout will be the one provided by 
>> socket.getdefaulttimeout(), which defaults to None(no timeout). 
>> (ref https://docs.djangoproject.com/en/4.1/topics/email/#smtp-backend) 
>>
>> So if you say: why is there no default value in smtplib, the smtplib 
>> maintainer can so go one level up and ask why there is no default timeout 
>> for `socket` :D 
>>
>> Nevertheless, I would argue that having a timeout in django, set to a 
>> rather conservative value such as 60 seconds, is much better than having no 
>> timeout at all; especially because what the OP posted: 
>>
>> > so at least if and when this happens, developers will find error traces 
>> in the logs and quickly understand where the problem is coming from, 
>> instead of spending hours to debug it like I did in the past week. 
>>
>> In the typical scenario, people are much better off with specified 
>> timeouts than with a timeout of None 
>>
>> — 
>> Michiel 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c62d1e69-5b9a-487f-8ca8-a843a7bbc914n%40googlegroups.com.

Re: Set a reasonable default for EMAIL_TIMEOUT

[Federico Capoano]

Hi there,

yes I can add EMAIL_TIMEOUT in my applications and I am already doing that.

The reason I wanted to bring this up here is that I care about Django.
This behavior goes against a well known concept of the Unix Philosophy 
which is to  "Write programs which fail in a way that is easy to diagnose".

I believe we can all agree that any potentially long running operation 
should have a default timeout after which the operation fails and a trace 
is left on a log somewhere for the system administrators to find
I can't see the benefit in not failing, not leaving log traces and keeping 
an app blocked indefinitely.

What are the benefits vs the downsides of such a change?

Benefit: any issue in production systems will be quickly understood and 
dealt with.
Downsides? Backward compatibility? Systems relying on the default timeout 
being set elsewhere? Anything else?

Even if we allow configuring different email backends, I am still 
wonderfing: why do we think it's good to have an email sending operation 
which can potentially stall indefinitely?

Best regards
Federico
On Friday, November 18, 2022 at 11:28:14 AM UTC-3 Michiel Beijen wrote:

> Hi Mariusz,
>
> > Op 17 nov. 2022, om 18:18 heeft Mariusz Felisiak  
> het volgende geschreven:
> > 
> > "Why is there no default for EMAIL_TIMEOUT?"
> > The Django's SMTP backend is a wrapper around smtplib.SMTP/SMTP_SSL and 
> we only pass the "timeout" parameter to the smtplib API, so I'd first ask 
> why there is no default value for a timeout parameter in smtplib.
>
> Well, Django’s docs say:
>
> > If unspecified, the default timeout will be the one provided by 
> socket.getdefaulttimeout(), which defaults to None(no timeout).
> (ref https://docs.djangoproject.com/en/4.1/topics/email/#smtp-backend)
>
> So if you say: why is there no default value in smtplib, the smtplib 
> maintainer can so go one level up and ask why there is no default timeout 
> for `socket` :D
>
> Nevertheless, I would argue that having a timeout in django, set to a 
> rather conservative value such as 60 seconds, is much better than having no 
> timeout at all; especially because what the OP posted:
>
> > so at least if and when this happens, developers will find error traces 
> in the logs and quickly understand where the problem is coming from, 
> instead of spending hours to debug it like I did in the past week.
>
> In the typical scenario, people are much better off with specified 
> timeouts than with a timeout of None
>
> —
> Michiel
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8ae42cd2-6c6c-47cb-8f56-7f0f46424df6n%40googlegroups.com.

Set a reasonable default for EMAIL_TIMEOUT

[Federico Capoano]

Hi all,

Why is there no default for EMAIL_TIMEOUT?

Applications built in Django can potentially stall indefinitely if email
sending starts to hang (eg: if the SMTP server is overloaded), when this
happens, since there's no timeout, there's also no error being logged, so
it's very hard and time consuming to debug.

Wouldn't it be better to set a timeout? Some high value like 2 minutes
which wouldn't really make sense to wait any longer, so at least if and
when this happens, developers will find error traces in the logs and
quickly understand where the problem is coming from, instead of spending
hours to debug it like I did in the past week.

Alternatively, we could set a default timeout only if
"socket.getdefaulttimeout()" returns None, just to avoid applications
stalling indefinitely, which is something that has been happening to me
lately.

Best regards
Federico Capoano

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6Vs6%2BWnJMsw95Uc_b006PZ2N_Pr%3DNgx4ztPDkgjVbkivA%40mail.gmail.com.

Default change password UX: area for improvements

[Federico Capoano]

Hey everyone,

Given the following text in the user change page:

*Raw passwords are not stored, so there is no way to see this user’s
password, but you can change the password using this form.*

Linking the change user password form using the word "this form" is on the
same level of links with "click here", which is notoriously bad UX and I
believe the django community is made of a lot of experts so there shouldn't
be the need to explain this here.

However, we can do better than just changing the link form, wouldn't it be
possible to add a link styled as a button?

Eg:

Raw passwords are not stored, so there is no way to see this user’s
password, but the password can be changed.

*Change the password*

BTW, this idea came from this issue:
https://github.com/openwisp/openwisp-radius/issues/265.
We can easily fix it in our django app but I thought it may be good to
raise this here so we may fix it at the root instead.

Best regards
Federico Capoano

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6VUm-bRsHkY_1LX9TwpJvMW%2B-Eyqf0Uye6FL%2B_Pb_%3D%2BQw%40mail.gmail.com.

Re: App label with dot

[Federico Capoano]

Created: https://code.djangoproject.com/ticket/32285#ticket
Not sure if the categorization is correct, I don't see where AppConfig
could fit there? Let's continue there.

Thanks

On Fri, Dec 18, 2020 at 11:59 AM Tom Forbes  wrote:

> Yes, a more friendly error message would be far better here! User-facing
> runtime exceptions like unpacking errors are never good.
>
> This should be quite a simple change to make, I’d suggest opening a trac
> ticket first and we can go from there.
>
> Tom
>
> On 18 Dec 2020, at 16:38, Federico Capoano 
> wrote:
>
> 
> Thanks Tim, would it be useful to have a more user friendly error message?
>
> Il ven 18 dic 2020, 11:08 Tim Graham  ha scritto:
>
>> app_label "should be a valid Python identifier." Dots aren't allowed.
>>
>>
>> https://docs.djangoproject.com/en/stable/ref/applications/#django.apps.AppConfig.label
>>
>> On Friday, December 18, 2020 at 10:47:45 AM UTC-5 Federico Capoano wrote:
>>
>>> Defining the app_label of an AppConfig with a dot, like
>>> "myproject.subapp" is allowed, but raises an exception in the migration
>>> framework (too many values to unpack).
>>>
>>> Did anybody notice this?
>>>
>>> Is it supposed to be allowed or not at all?
>>>
>>> Best regards
>>> Federico Capoano
>>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/django-developers/3cbMviLrSzI/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/134214d9-e3c1-4f74-9920-f0a62d4d675bn%40googlegroups.com
>> <https://groups.google.com/d/msgid/django-developers/134214d9-e3c1-4f74-9920-f0a62d4d675bn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAERYH6XvS3_z-akSZw_xy8FRHbdZoxeOYN3D_fVyRJ%3D7B9zUFA%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CAERYH6XvS3_z-akSZw_xy8FRHbdZoxeOYN3D_fVyRJ%3D7B9zUFA%40mail.gmail.com?utm_medium=email_source=footer>
> .
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/3cbMviLrSzI/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/98CF92C9-1E9D-4F56-9D4E-A6F78737002E%40tomforb.es
> <https://groups.google.com/d/msgid/django-developers/98CF92C9-1E9D-4F56-9D4E-A6F78737002E%40tomforb.es?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6VcL07cxDre8jNDacZ_T_r2jFP2YRruQF-VQieD2izQ8g%40mail.gmail.com.

Re: App label with dot

[Federico Capoano]

Thanks Tim, would it be useful to have a more user friendly error message?

Il ven 18 dic 2020, 11:08 Tim Graham  ha scritto:

> app_label "should be a valid Python identifier." Dots aren't allowed.
>
>
> https://docs.djangoproject.com/en/stable/ref/applications/#django.apps.AppConfig.label
>
> On Friday, December 18, 2020 at 10:47:45 AM UTC-5 Federico Capoano wrote:
>
>> Defining the app_label of an AppConfig with a dot, like
>> "myproject.subapp" is allowed, but raises an exception in the migration
>> framework (too many values to unpack).
>>
>> Did anybody notice this?
>>
>> Is it supposed to be allowed or not at all?
>>
>> Best regards
>> Federico Capoano
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/3cbMviLrSzI/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/134214d9-e3c1-4f74-9920-f0a62d4d675bn%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/134214d9-e3c1-4f74-9920-f0a62d4d675bn%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6XvS3_z-akSZw_xy8FRHbdZoxeOYN3D_fVyRJ%3D7B9zUFA%40mail.gmail.com.

SynchronousOnlyOperation: bug or misconfiguration?

[Federico Capoano]

I have a celery task which performs queries and network operations.

When deployed with celery with gevent concurrency, I occasionally (but not 
always) get reports of SynchronousOnlyOperation exceptions being raised.

I described the problem in detail on StackOverflow: SynchronousOnlyOperation 
from celery task using gevent execution pool 

.

I tried debugging the issue and looking for information about similar 
issues with django+celery+gevent with little luck, but I have started to 
wonder if the fact that Django is raising this error there is a bug.

I see this error is raised if django detects an event loop, should this 
apply also to when gevent is using the eventloop?

Thanks in advance
Best regards
Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/75a43f29-2bda-4d36-a891-960734264b9cn%40googlegroups.com.

App label with dot

[Federico Capoano]

Defining the app_label of an AppConfig with a dot, like "myproject.subapp" 
is allowed, but raises an exception in the migration framework (too many 
values to unpack).

Did anybody notice this?

Is it supposed to be allowed or not at all?

Best regards
Federico Capoano

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/72c66644-4b3a-4f4e-a27b-8b9b707c8b80n%40googlegroups.com.

Re: revisiting the Python version support policy

[Federico Capoano]

I would ask: what are the pros and cons of dropping support for python 3.5?

I think allowing users to easily use and install django based applications 
is more important than strictly follow a python version support policy.

I think that if we drop support for python 3.5, which is the default python 
version on many linux platforms right now, we will make the life of our 
users and developers harder.
I don't understand the reason for doing so, if we have to do it for a good 
reason, like a security issue, or because django has to take advantage of 
features that are available only from python 3.6 onwards, I would be in 
favour, but if we have to do it only because the policy says so, without 
any other advantage, I would amend the policy.

My 2 cents.

Thanks for your hard work maintaining django
Federico 


On Monday, January 21, 2019 at 10:56:40 AM UTC-5, Tim Graham wrote:
>
> When deciding when to drop support for Python 2 in Django, there was 
> consensus to adopt this Python version support policy [0]: "Typically, we 
> will support a Python version up to and including the first Django LTS 
> release whose security support ends after security support for that version 
> of Python ends. For example, Python 3.3 security support ends September 
> 2017 and Django 1.8 LTS security support ends April 2018. Therefore Django 
> 1.8 is the last version to support Python 3.3."
>
> Since then, we didn't abide by this policy when dropping Python 3.4, 
> mainly because Debian stable still used Python 3.4 at the time and Claude 
> argued that some people like him would have difficulty contributing to 
> Django if they had to install another version of Python [1].
>
> Based on the policy, it's time to drop support for Python 3.5 in the 
> master branch (Django 3.0) -- with Django 2.2 LTS supported until April 
> 2022 and Python 3.5 supported until September 2020). I created a ticket [2] 
> and PR [3] for dropping support for Python 3.5 [2], however, Claude 
> commented, "I'm not so enthusiast to drop Python 3.5 now (it is still the 
> default version in Debian stable). Couldn't this be done in Django 3.1 
> instead?"
>
> Are you in favor of amending the Python support version policy to account 
> for the Python version in Debian stable?
>
> [0] 
> https://docs.djangoproject.com/en/dev/faq/install/#what-python-version-can-i-use-with-django
> [1] 
> https://groups.google.com/d/msg/django-developers/4rbVKJYm8DI/TTh3i04pBQAJ
> [2] https://code.djangoproject.com/ticket/30116
> [3] https://github.com/django/django/pull/10864
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/51dfc517-8b5a-4803-9f8d-daa1bf920e7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Widening participation (Thoughts from DjangoCon)

[Federico Capoano]

Hi everyone,

It's been great to read some good insights on this discussion.

How to attract new contributors from a different demographic (geographic 
area and age)? Good question.

I wanted to share with you my experience as (second year) organization 
administrator and mentor for OpenWISP  (an open 
source network management system built on top of Django) in the Google 
Code-In program , which I think can really 
help in attracting new contributors from a different demographic group, 
which is really great for a project, because brings in a more diverse 
perspective and fresh lymph and enthusiasm.

*Google Code-In* is the sibling of the *Google Summer of Code* (in which 
django participated several times), and it is a global contest in which 
pre-university students in the range of 13-17 learn how to contribute to 
open source.
It works as follows:

- some open source organizations are selected each year, the list of 2018 
participating orgs  counts 
some notable participants as Postgres, Drupal, Wikimedia, KDE, OSGeo, Fedora
- these organizations publish a series of tasks with the aim of helping 
students familiarize with their codebase and practices in order to start 
contributing
- each tasks contains information, tags and list available mentors for that 
tasks
- students can see the list of tasks in a dashboard, it is publicly 
available so you can also see it: https://codein.withgoogle.com/tasks/
- students choose the tasks they want to work on and they "claim" it, they 
then use the organization's support channels (eg: IRC) to interact with 
mentors and other fellows to look for help
- at the end of the contest, each organization chooses 2 winners and 4 
finalists who receive prizes, the 2 winners win a trip to Google's HQ in 
California

Tasks can be about coding, QA/testing, design, outreach/research, 
documentation/training.

This program brings some challenges and needs a lot of patience, but has 
been great for the OpenWISP community for the following reasons:

- we've been lucky to get some really great students, their skills for 
their age is just incredible (teenage prodigies really)
- they learn really fast, so total beginners become productive in 2 months 
while more skilled students become great contributors in no time
- the big influx of beginners who seem to all stumble on the same issues 
really helped us to understand which roadblocks had to be removed in order 
to improve our documentation and make it easier for them to onboard
- the demographic is really diverse, many students are from Asia but we got 
some great students from Poland, Russia and the Americas
- the contest runs online, worldwide and remotely, no physical presence is 
needed
- we've been able to close a lot of the easier issues and some very useful 
higher impact (non-trivial) issues
- just OpenWISP trained over 950 students in 2018 
, some of these 
students will become mentors next year, allowing us to train even more
- some of these students stick around and keep contributing also when the 
program ends
- they bring in a fresh perspective, helping us to keep the project modern 
and ensure generational handover (which I think it's an overlooked problem 
in today's open source community)

*One important note*: many of the good students of OpenWISP choose us 
because our system is built in Python and Django and they either already 
know and like this stack or want to learn to use it.

So I think having some Django mentors to participate in this program, 
either with a dedicated organization or under an umbrella organization, 
could be beneficial for Django and Open Source in general.

It's not a silver bullet, it's not a panacea, but it can help.

PS: in one of our training tasks we have our students do the DjangoGirls 
tutorial , they 
really love this task :-)

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e4b230d9-7b09-4a99-9268-43dc4b07a810%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Testing admin inlines painful to debug ("ManagementForm data is missing or has been tampered with")

[Federico Capoano]

While writing tests for the admin, I'm clashing with this error message:

"ManagementForm data is missing or has been tampered with".

It's not the first time I do this, but every time the process has been 
extremely painful because the error message doesn't point out the exact 
problem.
This time I'm pretty sure to have done everything properly but I just can't 
find out what's missing, not even by dropping pdb.set_trace() into django's 
internal code.

I'm wondering if we can improve this error message so that if DEBUG is True 
we can have some more information about what is missing.

Would that be possible and welcome?

Best regards
Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2e91370b-fe2b-4e97-afa8-aa80b36273cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Admin raw_id_fields should trigger javascript change event

[Federico Capoano]

Hi everyone,

I have a raw_id_field in the admin and I need to trigger some javascript 
logic when it's value is filled by the user through the popup, but 
unfortunately I found out that when the popup is closed and the value is 
filled into the field the change event is not fired.

I found this quite surprising!

After looking for a solution and digging, I found a workaround which I 
tested and ensured works:
https://groups.google.com/d/msg/django-users/7sIM2-9XirY/d6Yr5_fFBwAJ

I was wondering: why don't we include the change event trigger into django 
itself instead of requiring django users to add this sort of hacks in their 
code?
This is an easy and low risk fix. Is there already an issue for this? I 
volunteer to do this.

Thanks
Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5508c560-d186-4b78-bde4-e4807791c291%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: A New Design for the "Congratulations!" Page

[Federico Capoano]

That's a great news, let us know if we can help with more feedback.

Federico

On Fri, Apr 21, 2017 at 9:20 AM Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> On 18 Apr 2017, at 20:31, Tim Allen  wrote:
>
> Your point on design by committee is spot-on; I think that's the direction
> we'll head in.
>
>
> Hello,
>
> For the sake of transparency, a group of five people (Tim, Chad, Collin
> Anderson, Frank Wiles, and myself) is now working on polishing the
> proposal. We'll ask for feedback on the next iteration when it's ready.
>
> Best regards,
>
> --
> Aymeric.
>
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/G09hwOQIdvU/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/3FB44932-5729-479D-BF74-771BAE4A3E27%40polytechnique.org
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6VD5Y%2BWy_0hYSWB%2BcPv7wDmFu%2B8s6u_nq%3DRhyw6Gw340w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

A New Design for the "Congratulations!" Page

[Federico Capoano]

Looks like a great improvement for newcomers! I hope to see this included to 
Django.

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cdff13ec-bf9e-406e-addf-ec2c78b2a8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Surprising behaviour in case of exception in form widget when DEBUG=False

[Federico Capoano]

Right, I see the log entries. Good to know there's a way to log them, for 
those who use sentry it will be even easier to notice this problem,
just ensure you're logging WARNING messages.

Thanks
Federico


On Friday, March 11, 2016 at 1:18:27 PM UTC+1, Tim Graham wrote:
>
> Yes, I think it's {% include %} rendering an empty string if there's an 
> exception. In Django 1.9, we added a logger for these exceptions to make 
> debugging easier: 
> https://docs.djangoproject.com/en/stable/topics/logging/#django-template.
>
> On Friday, March 11, 2016 at 6:30:35 AM UTC-5, Federico Capoano wrote:
>>
>> This never happened to me and I find it very strange. Maybe it's a bug or 
>> maybe it's the intended behaviour, but I find it REALLY strange.
>>
>> Consider a widget which, for some reason, raises an exception:
>>
>> class CoolWidget(AdminTextareaWidget):
>> def render(self, name, value, attrs={}):
>> WRONG()
>> return super(JsonSchemaWidget, self).render(name, value, attrs)
>>
>> The widget is included in an admin form like:
>>
>> class PizzaForm(forms.ModelForm):
>> class Meta:
>> model = Pizza
>> exclude = []
>> widgets = {'toppings': CoolWidget}
>>
>>
>> class PizzaAdmin(ModelAdmin):
>> form = TemplateForm
>>
>> Now, if DEBUG is True, trying to open the relative pizza change or add 
>> form will result in:
>>
>> NameError: name 'WRONG' is not defined
>>
>> When DEBUG to False what I see is really strange:
>>
>> I get an empty change/add form!
>>
>>
>> <https://lh3.googleusercontent.com/-k2-DznqIO2E/VuKq3sx74OI/Blo/AFAvi1vQ3EI6o2mshUBCLQgd8vpvvF5VQ/s1600/widget-problem-django.png>
>>
>>
>> Now suppose that the condition that raises the exception happens only on 
>> a remote server
>>
>> which has DEBUG=False. The exception happens because the configuration of 
>> a django project
>>
>> has not been updated correctly yet.
>>
>>
>> Instead of failing loud, it will fail silently, leaving an empty 
>> add/change form.
>>
>>
>> Is it the intended behaviour? Or a bug?
>>
>> I guess it is related due to the fact that template errors are silenced.
>>
>> Federico
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c6158dd8-cc88-4c26-ac25-4b86eef2ea3f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Surprising behaviour in case of exception in form widget when DEBUG=False

[Federico Capoano]

This never happened to me and I find it very strange. Maybe it's a bug or 
maybe it's the intended behaviour, but I find it REALLY strange.

Consider a widget which, for some reason, raises an exception:

class CoolWidget(AdminTextareaWidget):
def render(self, name, value, attrs={}):
WRONG()
return super(JsonSchemaWidget, self).render(name, value, attrs)

The widget is included in an admin form like:

class PizzaForm(forms.ModelForm):
class Meta:
model = Pizza
exclude = []
widgets = {'toppings': CoolWidget}


class PizzaAdmin(ModelAdmin):
form = TemplateForm

Now, if DEBUG is True, trying to open the relative pizza change or add form 
will result in:

NameError: name 'WRONG' is not defined

When DEBUG to False what I see is really strange:

I get an empty change/add form!




Now suppose that the condition that raises the exception happens only on a 
remote server

which has DEBUG=False. The exception happens because the configuration of a 
django project

has not been updated correctly yet.


Instead of failing loud, it will fail silently, leaving an empty add/change 
form.


Is it the intended behaviour? Or a bug?

I guess it is related due to the fact that template errors are silenced.

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e5f6999e-b94b-4ecc-a14c-02c01c789f09%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fellow Report - February 6, 2016

[Federico Capoano]

Great work!


On Saturday, February 6, 2016 at 4:27:42 PM UTC+1, Tim Graham wrote:
>
> Triaged
>
> ---
>
> https://code.djangoproject.com/ticket/26162 - Data loss when 
> ManyToManyField refers to the wrong table (accepted)
>
> https://code.djangoproject.com/ticket/26168 - BooleanField is forced to 
> be blank=True (accepted)
>
> https://code.djangoproject.com/ticket/26171 - ForeignKey with 
> db_constraint=False doesn't generate an index on MySQL (accepted)
>
> https://code.djangoproject.com/ticket/26173 - localize_input() shouldn't 
> put thousands separator in boolean values (accepted)
>
> https://code.djangoproject.com/ticket/26170 - ModelAdmin 
> add/change/delete views always run transaction on default database 
> (accepted)
>
> https://code.djangoproject.com/ticket/26177 - Using a Postgres database 
> with TIME_ZONE set to None and USE_TZ set to False causes an exception 
> (accepted)
>
> https://code.djangoproject.com/ticket/26179 - Remove null assignment 
> check for non-nullable foreign key fields (created)
>
> https://code.djangoproject.com/ticket/26180 - Altering unique_together 
> still sometimes missing deleted fields (accepted)
>
> Authored
>
> 
>
> https://github.com/django/django/pull/6092 - Fixed #26176 -- Fixed E123 
> flake8 warnings.
>
> https://github.com/django/django/pull/6093 - Fixed #26175 -- Removed SHA1 
> password hashes in tests.
>
> https://github.com/django/django/pull/6096 - Fixed #26177 -- Fixed a 
> PostgreSQL crash with TIME_ZONE=None and USE_TZ=False.
>
> https://github.com/django/djangoproject.com/issues/629 - Update to 
> elasticsearch 2.0
>
> Reviewed/committed
>
> --
>
> https://github.com/django/django/pull/6054 - Fixed #7923 -- Added links 
> to objects displayed by ModelAdmin.raw_id_fields.
>
> https://github.com/django/django/pull/6035 - Fixed #26124 -- Added 
> missing code formatting to docs headers.
>
> https://github.com/django/django/pull/6060 - Fixed #26152 -- Documented 
> how to avoid django.setup() deadlock in standalone scripts.
>
>
> https://github.com/django/django/commit/62f3acc70a43a3c4f4970839d490ac8ea6c79047
>  
> - Fixed incorrect permissions check for admin's "Save as new".
>
> https://github.com/django/django/pull/6073 - Fixed #26155 -- Skipped URL 
> checks if no ROOTURL_CONF setting.
>
> https://github.com/django/django/pull/5686 - Fixed #11313 -- Made 
> ModelAdmin.list_editable more resilient to concurrent edits.
>
> https://github.com/django/django/pull/5980 - Fixed #23971 -- Added "Has 
> date"/"No date" choices for DateFieldListFilter.
>
> https://github.com/django/django/pull/6037 - Fixed #25731 -- Removed 
> unused choices kwarg for Select.render()
>
> https://github.com/django/django/pull/5193 - Fixed #12405 -- Added 
> LOGOUT_REDIRECT_URL setting.
>
> https://github.com/django/django/pull/5745 - Fixed #25833 -- Added 
> support for non-atomic migrations
>
> https://github.com/django/django/pull/6078 - Fixed #13875 -- Made admin's 
> submit_row template tag pass whole Context.
>
> https://github.com/django/django/pull/6051 - Fixed #26144 -- Warned when 
> dumping proxy model without concrete parent.
> https://github.com/django/django/pull/5745 - Fixed #25833 -- Added 
> support for non-atomic migrations.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ff1c7f67-1c29-436e-ab87-1eebbed916bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [admin] submit_row template tag reusability issue

[Federico Capoano]

Updated the ticket and sent a PR, I report the comment here too for 
convenience:

Just opened PR https://github.com/django/django/pull/6078.

A more paranoid test could be written, like creating a custom app with an 
admin that extends the context, loads a modified version of the subimt_row 
template and check the extended context is available in the template. But I 
think all the complexity needed wouldn't add much value.

Federico


On Tuesday, February 2, 2016 at 5:13:26 PM UTC+1, Tim Graham wrote:
>
> The patch is 5 years old and doesn't apply cleanly, so it needs 
> improvement in that sense but that shouldn't be difficult. The main blocker 
> to getting it merged looks like "Needs tests."
>
> On Tuesday, February 2, 2016 at 10:58:40 AM UTC-5, Federico Capoano wrote:
>>
>> Hi Tim,
>>
>> thank you for letting me know of the open ticket.
>> The "patch needs improvement" flag is set to "no". Is that correct or not?
>>
>> Federico
>>
>> On Mon, Feb 1, 2016 at 1:18 PM Tim Graham <timog...@gmail.com> wrote:
>>
>>> Here is an accepted ticket: https://code.djangoproject.com/ticket/13875
>>>
>>> Feel free to update and/or improve the patch, add tests, and send a pull 
>>> request. Thanks!
>>>
>>>
>>> On Monday, February 1, 2016 at 6:18:35 AM UTC-5, Federico Capoano wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I came across the need of modifying the submit row functionality in the 
>>>> django admin and I found out a small issue which prevents me from 
>>>> modifying 
>>>> the template without rewriting (duplicating) also the templatetag entirely.
>>>>
>>>> If you take a look at the code of submit_row templatetag 
>>>> <https://github.com/django/django/blob/master/django/contrib/admin/templatetags/admin_modify.py#L41-L70>,
>>>>  
>>>> you will notice the original view context is not passed entirely to the 
>>>> return statement. A new ctx dictionary is created and filled only with a 
>>>> smaller subset of key/value pairs.
>>>>
>>>> I took a quick look at other template tags and I noticed they return 
>>>> the entire context and not a stripped version of it.
>>>>
>>>> I would prefer submit_row to behave in the same way as the other 
>>>> templatetags so I wouldn't have to duplicate its code to add new context 
>>>> values to it.
>>>> If there's consensus on this issue I can create a ticket and send a 
>>>> pull request, let me know.
>>>>
>>>> Federico
>>>>
>>> -- 
>>> You received this message because you are subscribed to a topic in the 
>>> Google Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this topic, visit 
>>> https://groups.google.com/d/topic/django-developers/yrngp53RvxY/unsubscribe
>>> .
>>> To unsubscribe from this group and all its topics, send an email to 
>>> django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/bbb82460-8a55-47ff-8e7f-acfc063f6cf0%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/django-developers/bbb82460-8a55-47ff-8e7f-acfc063f6cf0%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6a8e218c-6d00-40c4-9a37-58a58961aaf2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [admin] submit_row template tag reusability issue

[Federico Capoano]

Hi Tim,

thank you for letting me know of the open ticket.
The "patch needs improvement" flag is set to "no". Is that correct or not?

Federico

On Mon, Feb 1, 2016 at 1:18 PM Tim Graham <timogra...@gmail.com> wrote:

> Here is an accepted ticket: https://code.djangoproject.com/ticket/13875
>
> Feel free to update and/or improve the patch, add tests, and send a pull
> request. Thanks!
>
>
> On Monday, February 1, 2016 at 6:18:35 AM UTC-5, Federico Capoano wrote:
>>
>> Hi all,
>>
>> I came across the need of modifying the submit row functionality in the
>> django admin and I found out a small issue which prevents me from modifying
>> the template without rewriting (duplicating) also the templatetag entirely.
>>
>> If you take a look at the code of submit_row templatetag
>> <https://github.com/django/django/blob/master/django/contrib/admin/templatetags/admin_modify.py#L41-L70>,
>> you will notice the original view context is not passed entirely to the
>> return statement. A new ctx dictionary is created and filled only with a
>> smaller subset of key/value pairs.
>>
>> I took a quick look at other template tags and I noticed they return the
>> entire context and not a stripped version of it.
>>
>> I would prefer submit_row to behave in the same way as the other
>> templatetags so I wouldn't have to duplicate its code to add new context
>> values to it.
>> If there's consensus on this issue I can create a ticket and send a pull
>> request, let me know.
>>
>> Federico
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/yrngp53RvxY/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/bbb82460-8a55-47ff-8e7f-acfc063f6cf0%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/bbb82460-8a55-47ff-8e7f-acfc063f6cf0%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6Vfrb%2B4YD%2B-9bsCWdGPDR8QzxGYJrC9e7PH9uWQJ%3D7asg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[admin] submit_row template tag reusability issue

[Federico Capoano]

Hi all,

I came across the need of modifying the submit row functionality in the 
django admin and I found out a small issue which prevents me from modifying 
the template without rewriting (duplicating) also the templatetag entirely.

If you take a look at the code of submit_row templatetag 
,
 
you will notice the original view context is not passed entirely to the 
return statement. A new ctx dictionary is created and filled only with a 
smaller subset of key/value pairs.

I took a quick look at other template tags and I noticed they return the 
entire context and not a stripped version of it.

I would prefer submit_row to behave in the same way as the other 
templatetags so I wouldn't have to duplicate its code to add new context 
values to it.
If there's consensus on this issue I can create a ticket and send a pull 
request, let me know.

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8172f221-a312-4cae-b8ee-fd2e9f971b7c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Links from django docs to djangopackages.com should be 'officially endorsed' or not?

[Federico Capoano]

It seems like a good proposal, It would be good to know why Sergei doesn't 
think it is appropiate for Django to add links to third party package 
comparison grids.

django-rest-framework links to third party extensions in its documentation 
and this had a positive effect on the whole DRF ecosystem.

Federico


On Monday, February 1, 2016 at 10:39:44 AM UTC+1, guettli wrote:
>
> This ticket was closed as invalid: 
> https://code.djangoproject.com/ticket/26159
>
> {{{
>
> Here comes a pull request to add a link to the djangopackages comparison 
> grid "Pagination".
> Background: The pagination in django core is only very basic. Most people 
> want more. I think re-use helps more then re-inventing :-)
> }}}
>
>
> {{{
> sergei-maertens
>
> Resolution set to invalid
> Status changed from new to closed
>
> I don't think the docs are the appropriate place to point out 
> alternatives. I feel if Django starts going down that road, then you should 
> do the same not only for pagination, but for forms, views, other db 
> adapters, other template engines... I think it's a slippery slope.
>
> While I think that djangopackages.com is a great resource, I'm not sure 
> if it (and no other resources) should be 'officially endorsed' by Django in 
> the docs.
> }}}
>
> I think a lot of time gets wasted by re-inventing stuff. Yes, programmers 
> love to program. But time have
> changed. There are a lot of great re-usable apps and libraries and 
> programmers who wants to
> get things done use them.
>
> What do you think for *this* issue: the docs for pagination should point 
> to the comparison grid or not?
>
> Regards,
>   Thomas Güttler
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/47c444a6-d524-415f-b352-9cba2c4dffe4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Validation of m2m

[Federico Capoano]

It could be a potential ticket to work on my next django dev sprint.

But first it would be nice to have some basic consensus on how to proceed.

Was it ever discussed in any older thread or ticket?



On Thursday, December 3, 2015 at 5:21:06 PM UTC+1, Tim Graham wrote:
>
> Here's an open ticket about adding model level validation for 
> many-to-many: https://code.djangoproject.com/ticket/12938
>
> On Thursday, December 3, 2015 at 11:04:22 AM UTC-5, Federico Capoano wrote:
>>
>> Thanks Aymeric,
>>
>> I decided to post this problem on django-developers because I've read 
>> this ticket:
>> https://code.djangoproject.com/ticket/24731
>> Sorry for omitting this information.
>>
>> Has there been a discussion about this topic already?
>>
>> Would it be hard to implement an easier solution into django?
>>
>> I spent a few hours working on this issue, I consider myself fluent with 
>> django and it's quite shocking I had to put such an amount of effort just 
>> to validate many2many relationships before they are saved to the database.
>>
>> IMHO it would be better if we could do one of these two (or even both) 
>> things:
>>
>> 1. make this process easier in future django versions
>> 2. document the current best practice to solve this problem in current 
>> django to save people's time
>>
>> What do people you think?
>>
>> Here's my solution working with django 1.9:
>> https://github.com/openwisp/django-netjsonconfig/compare/4082988...master
>>
>> What do you think of it? Can it be improved in some way?
>>
>> Federico
>>
>>
>> On Thursday, December 3, 2015 at 1:43:21 PM UTC+1, Aymeric Augustin wrote:
>>>
>>> Hello Frederico,
>>>
>>> It appears that you're hitting the problem described in the "Avoid 
>>> catching exceptions inside atomic!" warning on this page:
>>>
>>> https://docs.djangoproject.com/en/1.8/topics/db/transactions/#handling-exceptions-within-postgresql-transactions
>>>
>>> To obtain that sort of result, I suppose you must be catching an 
>>> IntegrityError, re-raising a ValidationError, which Django in turn catches, 
>>> and then you hit that traceback.
>>>
>>> Adding an atomic block inside your try/catch block that catches the 
>>> IntegrityError will resolve that particular problem — putting that part of 
>>> the discussion into django-users territory.
>>>
>>> If this isn't what happens, please post your code and ask for help on 
>>> django-users.
>>>
>>> -- 
>>> Aymeric
>>>
>>> 2015-12-03 13:28 GMT+01:00 Federico Capoano <federico...@gmail.com>:
>>>
>>>> Hi everybody,
>>>>
>>>> I am sure it has happened to many of you.
>>>>
>>>> Validating m2m BEFORE saving the relationships is very hard and time 
>>>> consuming.
>>>>
>>>> Now this solution:
>>>> http://schinckel.net/2012/02/06/pre-validating-many-to-many-fields./
>>>>
>>>> Proposes to solve it with a ModelForm in the admin.
>>>> Cool, that works.
>>>>
>>>> But, if I want to enforce validation on the model, to avoid corrupted 
>>>> data, I notice the signal is executed in a transaction block, in which if 
>>>> I 
>>>> raise a ValidationError I get the following:
>>>>
>>>> Traceback (most recent call last):
>>>>   File 
>>>> "/var/www/django-netjsonconfig/django_netjsonconfig/tests/test_device.py", 
>>>> line 106, in test_m2m_validation
>>>> d.templates.add(t)
>>>>   File 
>>>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/fields/related_descriptors.py",
>>>>  
>>>> line 843, in add
>>>> self._add_items(self.source_field_name, self.target_field_name, 
>>>> *objs)
>>>>   File 
>>>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/sortedm2m/fields.py",
>>>>  
>>>> line 138, in _add_items
>>>> for val in vals:
>>>>   File 
>>>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
>>>>  
>>>> line 258, in __iter__
>>>> self._fetch_all()
>>>>   File 
>>>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
>&g

Re: Validation of m2m

[Federico Capoano]

Thanks Aymeric,

I decided to post this problem on django-developers because I've read this 
ticket:
https://code.djangoproject.com/ticket/24731
Sorry for omitting this information.

Has there been a discussion about this topic already?

Would it be hard to implement an easier solution into django?

I spent a few hours working on this issue, I consider myself fluent with 
django and it's quite shocking I had to put such an amount of effort just 
to validate many2many relationships before they are saved to the database.

IMHO it would be better if we could do one of these two (or even both) 
things:

1. make this process easier in future django versions
2. document the current best practice to solve this problem in current 
django to save people's time

What do people you think?

Here's my solution working with django 1.9:
https://github.com/openwisp/django-netjsonconfig/compare/4082988...master

What do you think of it? Can it be improved in some way?

Federico


On Thursday, December 3, 2015 at 1:43:21 PM UTC+1, Aymeric Augustin wrote:
>
> Hello Frederico,
>
> It appears that you're hitting the problem described in the "Avoid 
> catching exceptions inside atomic!" warning on this page:
>
> https://docs.djangoproject.com/en/1.8/topics/db/transactions/#handling-exceptions-within-postgresql-transactions
>
> To obtain that sort of result, I suppose you must be catching an 
> IntegrityError, re-raising a ValidationError, which Django in turn catches, 
> and then you hit that traceback.
>
> Adding an atomic block inside your try/catch block that catches the 
> IntegrityError will resolve that particular problem — putting that part of 
> the discussion into django-users territory.
>
> If this isn't what happens, please post your code and ask for help on 
> django-users.
>
> -- 
> Aymeric
>
> 2015-12-03 13:28 GMT+01:00 Federico Capoano <federico...@gmail.com 
> >:
>
>> Hi everybody,
>>
>> I am sure it has happened to many of you.
>>
>> Validating m2m BEFORE saving the relationships is very hard and time 
>> consuming.
>>
>> Now this solution:
>> http://schinckel.net/2012/02/06/pre-validating-many-to-many-fields./
>>
>> Proposes to solve it with a ModelForm in the admin.
>> Cool, that works.
>>
>> But, if I want to enforce validation on the model, to avoid corrupted 
>> data, I notice the signal is executed in a transaction block, in which if I 
>> raise a ValidationError I get the following:
>>
>> Traceback (most recent call last):
>>   File 
>> "/var/www/django-netjsonconfig/django_netjsonconfig/tests/test_device.py", 
>> line 106, in test_m2m_validation
>> d.templates.add(t)
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/fields/related_descriptors.py",
>>  
>> line 843, in add
>> self._add_items(self.source_field_name, self.target_field_name, *objs)
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/sortedm2m/fields.py",
>>  
>> line 138, in _add_items
>> for val in vals:
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
>>  
>> line 258, in __iter__
>> self._fetch_all()
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
>>  
>> line 1074, in _fetch_all
>> self._result_cache = list(self.iterator())
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
>>  
>> line 158, in __iter__
>> for row in compiler.results_iter():
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/sql/compiler.py",
>>  
>> line 806, in results_iter
>> results = self.execute_sql(MULTI)
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/sql/compiler.py",
>>  
>> line 852, in execute_sql
>> cursor.execute(sql, params)
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/backends/utils.py",
>>  
>> line 59, in execute
>> self.db.validate_no_broken_transaction()
>>   File 
>> "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/backends/base/base.py",
>>  
>> line 429, in validate_no_broken_transaction
>> "An error occurred in the current transaction. You can't "
>> dja

Validation of m2m

[Federico Capoano]

Hi everybody,

I am sure it has happened to many of you.

Validating m2m BEFORE saving the relationships is very hard and time 
consuming.

Now this solution:
http://schinckel.net/2012/02/06/pre-validating-many-to-many-fields./

Proposes to solve it with a ModelForm in the admin.
Cool, that works.

But, if I want to enforce validation on the model, to avoid corrupted data, 
I notice the signal is executed in a transaction block, in which if I raise 
a ValidationError I get the following:

Traceback (most recent call last):
  File 
"/var/www/django-netjsonconfig/django_netjsonconfig/tests/test_device.py", 
line 106, in test_m2m_validation
d.templates.add(t)
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/fields/related_descriptors.py",
 
line 843, in add
self._add_items(self.source_field_name, self.target_field_name, *objs)
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/sortedm2m/fields.py",
 
line 138, in _add_items
for val in vals:
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
 
line 258, in __iter__
self._fetch_all()
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
 
line 1074, in _fetch_all
self._result_cache = list(self.iterator())
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
 
line 158, in __iter__
for row in compiler.results_iter():
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/sql/compiler.py",
 
line 806, in results_iter
results = self.execute_sql(MULTI)
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/sql/compiler.py",
 
line 852, in execute_sql
cursor.execute(sql, params)
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/backends/utils.py",
 
line 59, in execute
self.db.validate_no_broken_transaction()
  File 
"/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/backends/base/base.py",
 
line 429, in validate_no_broken_transaction
"An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the 
current transaction. You can't execute queries until the end of the 
'atomic' block.

This is surely an area that needs improvement in django.

Why is it so hard?

Best regards
Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2e6e82d0-0645-4fd7-8905-d327c99b6352%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

I just changed the URL in the to
http://www.postgresql.org/docs/current/static/non-durability.html as
suggested.

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6Wtm7V6o_DqqJn7ZgJQFNyC0AH%3DrEHb6HyE58NTDrzrXQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Errata, PR link is https://github.com/django/django/pull/5021 - sorry

On Mon, Jul 20, 2015 at 11:42 AM, Federico Capoano
<federico.capo...@gmail.com> wrote:
> Thank you for all the feedback.
>
> I opened this PR with the proposed addition to the docs:
> https://github.com/django/django/compare/master...nemesisdesign:patch-1
>
> Let me know if there's anything I can improve.
>
> Just a final note: I think this solution is not the optimal one, let
> me explain you why. The main reason I came on this list asking for
> information about UNLOGGED tables in postgresql was exactly because of
> that page which suggests a few solutions. I spent quite many hours
> trying all of them (including running the entire DB in RAM), and
> measuring test execution time with each one of those settings. In the
> end I figured out that the only settings worth touching in my setup
> are those 3 I mentioned before.
>
> In order to share what I learnt and hoping to avoid some pain to
> people that will come on the same path, I quickly wrote a blog post
> with these suggestions:
> http://nemesisdesign.net/blog/coding/how-to-speed-up-tests-django-postgresql/
>
> Nothing new really, I found a few other blog posts with similar
> suggestions, but the suggestions were scattered on different pages and
> they didn't mention which changes were the most effective ones. That's
> why I felt the need of writing this.
>
> And by the way, I'm really happy of the outcome!
>
> Federico
>
> On Mon, Jul 20, 2015 at 2:47 AM, Curtis Maloney
> <cur...@acommoncreative.com> wrote:
>> I second what Aymeric say rather than take on the burden of
>> maintaining correct warnings, let's point at the people whose
>> responsibility it really is :)
>>
>> --
>> Curtis
>>
>> On 20 July 2015 at 06:44, Aymeric Augustin
>> <aymeric.augus...@polytechnique.org> wrote:
>>> I agree with pointing to the relevant section of the PostgreSQL 
>>> documentation. It will always be more complete, accurate and up-to-date 
>>> that what we could write.
>>>
>>> --
>>> Aymeric.
>>>
>>>
>>>
>>>> On 19 juil. 2015, at 19:43, Christophe Pettus <x...@thebuild.com> wrote:
>>>>
>>>> This can be achieved by pointing to the relevant section in the PostgreSQL 
>>>> documentation with a general "Test execution may be sped up by adjusting 
>>>> the data integrity parameters in PostgreSQL; be sure to read the 
>>>> appropriate warnings before making any changes" warning.
>>>>
>>>> Putting actual recommended settings in the Django documentation seems, at 
>>>> a minimum, pointlessly duplicative, and ties the Django documentation to 
>>>> the current state of the world in PostgreSQL gratuitously.
>>>>
>>>>
>>>> On Jul 19, 2015, at 10:32 AM, Luke Plant <l.plant...@cantab.net> wrote:
>>>>
>>>>> I agree with Federico on this - as long as we slap a big warning on it — 
>>>>> "This is dangerous - it could make your database more likely to lose data 
>>>>> or become corrupted, only use on a development machine where you can 
>>>>> restore the entire contents of all databases in the cluster easily" — I 
>>>>> don't see a problem in this being in our docs.
>>>>>
>>>>> If people refuse to read a clear warning, they shouldn't be doing web 
>>>>> development. They are just as likely to find similar instructions on the 
>>>>> internet, but without warnings, and having it in our docs with the 
>>>>> warning will be helpful.
>>>>>
>>>>> Having a fast test suite is such an important part of development that it 
>>>>> shouldn't be held back by  attempting to protect the world from people 
>>>>> who cannot be helped.
>>>>>
>>>>> Luke
>>>>>
>>>>> On 16/07/15 16:49, Christophe Pettus wrote:
>>>>>> On Jul 16, 2015, at 1:16 AM, Federico Capoano 
>>>>>> <federico.capo...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>> I also don't like the idea of believing django users are too stupid to
>>>>>>> understand that this advice si valid for development only. Generally
>>>>>>> python and django users are intelligent enough to properly read the
>>>>>>> docs and understand what's written on it.
>>>

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Thank you for all the feedback.

I opened this PR with the proposed addition to the docs:
https://github.com/django/django/compare/master...nemesisdesign:patch-1

Let me know if there's anything I can improve.

Just a final note: I think this solution is not the optimal one, let
me explain you why. The main reason I came on this list asking for
information about UNLOGGED tables in postgresql was exactly because of
that page which suggests a few solutions. I spent quite many hours
trying all of them (including running the entire DB in RAM), and
measuring test execution time with each one of those settings. In the
end I figured out that the only settings worth touching in my setup
are those 3 I mentioned before.

In order to share what I learnt and hoping to avoid some pain to
people that will come on the same path, I quickly wrote a blog post
with these suggestions:
http://nemesisdesign.net/blog/coding/how-to-speed-up-tests-django-postgresql/

Nothing new really, I found a few other blog posts with similar
suggestions, but the suggestions were scattered on different pages and
they didn't mention which changes were the most effective ones. That's
why I felt the need of writing this.

And by the way, I'm really happy of the outcome!

Federico

On Mon, Jul 20, 2015 at 2:47 AM, Curtis Maloney
<cur...@acommoncreative.com> wrote:
> I second what Aymeric say rather than take on the burden of
> maintaining correct warnings, let's point at the people whose
> responsibility it really is :)
>
> --
> Curtis
>
> On 20 July 2015 at 06:44, Aymeric Augustin
> <aymeric.augus...@polytechnique.org> wrote:
>> I agree with pointing to the relevant section of the PostgreSQL 
>> documentation. It will always be more complete, accurate and up-to-date that 
>> what we could write.
>>
>> --
>> Aymeric.
>>
>>
>>
>>> On 19 juil. 2015, at 19:43, Christophe Pettus <x...@thebuild.com> wrote:
>>>
>>> This can be achieved by pointing to the relevant section in the PostgreSQL 
>>> documentation with a general "Test execution may be sped up by adjusting 
>>> the data integrity parameters in PostgreSQL; be sure to read the 
>>> appropriate warnings before making any changes" warning.
>>>
>>> Putting actual recommended settings in the Django documentation seems, at a 
>>> minimum, pointlessly duplicative, and ties the Django documentation to the 
>>> current state of the world in PostgreSQL gratuitously.
>>>
>>>
>>> On Jul 19, 2015, at 10:32 AM, Luke Plant <l.plant...@cantab.net> wrote:
>>>
>>>> I agree with Federico on this - as long as we slap a big warning on it — 
>>>> "This is dangerous - it could make your database more likely to lose data 
>>>> or become corrupted, only use on a development machine where you can 
>>>> restore the entire contents of all databases in the cluster easily" — I 
>>>> don't see a problem in this being in our docs.
>>>>
>>>> If people refuse to read a clear warning, they shouldn't be doing web 
>>>> development. They are just as likely to find similar instructions on the 
>>>> internet, but without warnings, and having it in our docs with the warning 
>>>> will be helpful.
>>>>
>>>> Having a fast test suite is such an important part of development that it 
>>>> shouldn't be held back by  attempting to protect the world from people who 
>>>> cannot be helped.
>>>>
>>>> Luke
>>>>
>>>> On 16/07/15 16:49, Christophe Pettus wrote:
>>>>> On Jul 16, 2015, at 1:16 AM, Federico Capoano <federico.capo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>> I also don't like the idea of believing django users are too stupid to
>>>>>> understand that this advice si valid for development only. Generally
>>>>>> python and django users are intelligent enough to properly read the
>>>>>> docs and understand what's written on it.
>>>>>>
>>>>> It's not a matter of being "intelligent" or not.  Developers are busy and 
>>>>> can simply google things, see a particular line, and drop it in without 
>>>>> fully understanding exactly what is going on.  (Simply read this group 
>>>>> for a while if you don't believe this to be the case!)  People already 
>>>>> turn off fsync, in production, after having read the PostgreSQL 
>>>>> documentation, without actually realizing that they've put their database 
>>>>> in dan

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Hey Christophe,

On Thu, Jul 16, 2015 at 8:34 AM, Christophe Pettus  wrote:
[CUT]
>
> By the way, I would strongly advise *against* *ever* even mentioning fsync = 
> off anywhere in the Django documentation; that is such a horribly bad idea in 
> 99.95% of real-life situations that steering people towards it as a "go 
> faster" button is very unwise.

We were not mentioning to advise to set non durable options for
production use, but for development, which I think it's more than
accetable, considering that I got a 70% boost, which together with the
django 1.8 --keepdb option and a bit of mocking, got my entire huge
test suite run in less than one minute, while running tests for
specific modules now (thank God) take seconds.
This is immensely important if you are doing TDD or refactoring often.

I also don't like the idea of believing django users are too stupid to
understand that this advice si valid for development only. Generally
python and django users are intelligent enough to properly read the
docs and understand what's written on it.

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6UrK5%3DnV-jX6nXK%2BGrhR-GMjbWhuCGYce_UvBBo2g8ZxA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Renaming the postgresql_psycopg2 backend

[Federico Capoano]

I would definitely second this.


On Wednesday, July 15, 2015 at 3:17:30 PM UTC+2, Baptiste Mispelon wrote:
>
> Hi everyone, 
>
> After starting a new project recently and failing three times to type 
> the name of the postgres backend correctly, I was wondering if there's 
> really any value in keeping around this strange name. 
>
>  From what I understand, it's a historical artifact from a time when 
> there was more than one postgres backend. 
>
>
> Could we at least make it an alias of a more simply named "postgresql" 
> backend? 
>
>
> Thanks, 
> Baptiste 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/042eb416-701c-46a9-b5e7-845d270f8ce7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

That's quite baffling.


On Tuesday, July 14, 2015 at 7:03:03 PM UTC+2, Tim Graham wrote:
>
> I see a thirty second increase in the test suite (from 7.5 minutes to 8 
> minutes) on my local machine with:
>
> sql_create_table = "CREATE UNLOGGED TABLE %(table)s (%(definition)s)"
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/239f0228-5cec-4957-8d3a-146fed22377b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

That's also a viable alternative, although it also mention unlogged tables,
which are not supported by django.

We could list the quick hint with config sample, a link to pg non durable
options page and a warning that UNLOGGED tables are not supported yet. What
do you think about this?

BTW has anyone had the time to try unlogged tables with their test suite to
see if it gave them any improvements? I am quite surprised i didn't even
get a 5-10% improvement out of the promised 10-17%.

Federico
Wouldn't it be enough to just have link to the correct PostgreSQL site:
http://www.postgresql.org/docs/current/static/non-durability.html
And then some text with "if you want to run your tests on postgresql please
see the docs around non durable postgres"

2015-07-13 18:46 GMT+02:00 Federico Capoano <federico.capo...@gmail.com>:

> Sure, i've just done something similar for this project:
>
> http://nodeshot.readthedocs.org/en/latest/topics/install_development.html#how-to-setup-the-test-environment
>
> The *keepdb* option allows to avoid recreating the test database at each
> run, hence saving precious time.
>
> If you want to speed up tests even more, tweak your local postgresql
> configuration by setting these values:
>
> # /etc/postgresql/9.1/main/postgresql.conf
> # only for development!
> fsync = off
> synchronous_commit = off
> full_page_writes = off
>
>
> We just have to make sure people actually find this information via search
> engines.
>
> The best thing would be to have a subsection called "How to speed up tests
> with Postgres" or something similar.
>
> What do you think?
>
> Federico
>
>
>
> On Monday, July 13, 2015 at 5:54:08 PM UTC+2, Aymeric Augustin wrote:
>>
>> Hello Federico,
>>
>> 2015-07-13 16:26 GMT+02:00 Federico Capoano <federico...@gmail.com>:
>>
>>> While these are the winners:
>>>
>>>- fsync = off
>>>- synchronous_commit = off
>>>- full_page_writes = off
>>>
>>> Would you like to submit a patch to docs/ref/databases.txt, in the
>> "PostgreSQL
>> notes" section, explaining this configuration?
>>
>> I'm sure it would help many people run their tests faster on PostgreSQL.
>> It would
>> also counter the trend to run tests on SQLite "because it's faster" — it
>> isn't.
>>
>> --
>> Aymeric.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com?utm_medium=email_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

 --
You received this message because you are subscribed to a topic in the
Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/django-developers/IkRgMxTTzPQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAPDLAU7bYbeYb7pRgmqTR--zUrdU0fJyB5mHPWHoGEgawbB-MA%40mail.gmail.com
<https://groups.google.com/d/msgid/django-developers/CAPDLAU7bYbeYb7pRgmqTR--zUrdU0fJyB5mHPWHoGEgawbB-MA%40mail.gmail.com?utm_medium=email_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6XLR6idaHzYOuuJ65iR%3DqXw8FsTvCyj-8OfjSeMAmS5qA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: deprecate and remove egg template loader

[Federico Capoano]

+1 
I am assuming the deprecation is done according to the process that was 
discussed in the thread "An easier path to upgrade from one LTS release to 
another".


On Sunday, July 12, 2015 at 4:53:23 PM UTC+2, James Bennett wrote:
>
> There's not much to this, really, except what's in the subject line of 
> this message.
>
> The problem of providing a single-file, no-build-step format for 
> distributing and installing Python packages has been solved by wheels, and 
> wheels also don't cause the pile of weirdness that comes with using eggs.
>
> So Django should really stop encouraging/supporting the use of eggs. At a 
> minimum, this should involve Django 1.9 starting the deprecation process 
> for the egg template loader, and any other parts of Django which contain 
> special-case workarounds to deal with eggs.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/691ba2b2-9a45-470a-afe8-3a7bd0508a97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Sure, i've just done something similar for this project:
http://nodeshot.readthedocs.org/en/latest/topics/install_development.html#how-to-setup-the-test-environment

The *keepdb* option allows to avoid recreating the test database at each 
run, hence saving precious time.

If you want to speed up tests even more, tweak your local postgresql 
configuration by setting these values:

# /etc/postgresql/9.1/main/postgresql.conf
# only for development!
fsync = off
synchronous_commit = off
full_page_writes = off


We just have to make sure people actually find this information via search 
engines.

The best thing would be to have a subsection called "How to speed up tests 
with Postgres" or something similar.

What do you think?

Federico



On Monday, July 13, 2015 at 5:54:08 PM UTC+2, Aymeric Augustin wrote:
>
> Hello Federico,
>
> 2015-07-13 16:26 GMT+02:00 Federico Capoano <federico...@gmail.com 
> >:
>
>> While these are the winners:
>>
>>- fsync = off
>>- synchronous_commit = off
>>- full_page_writes = off
>>
>> Would you like to submit a patch to docs/ref/databases.txt, in the 
> "PostgreSQL
> notes" section, explaining this configuration?
>
> I'm sure it would help many people run their tests faster on PostgreSQL. 
> It would
> also counter the trend to run tests on SQLite "because it's faster" — it 
> isn't.
>
> -- 
> Aymeric.
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

I just ran into this ticket:
https://code.djangoproject.com/ticket/24306

I found myself in a situation where it would be really useful.

Two questions:

   - has anybody verified that UNLOGGED tables really perform faster when 
   running tests with Postgres?
   - is this feature on the TODO list for the next release? If yest what 
   needs to be done in ordere to help out?

Thanks
Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/35f7d80b-7c79-4caf-be0f-7b9c1796fcb2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: virtual fields and the migration framework

[Federico Capoano]

Thank you very much for the suggestions.

So after using:

cls._meta.add_field(self, virtual=True)

instead of:

cls._meta.add_field(self)
cls._meta.virtual_fields.append(self)

The VirtualField implementation of django-hstore is much cleaner and does 
not conflict with the migration framework.

I wonder if it's possible to achieve a similar result in django 1.7?

Regarding this VirtualField implementation, in the near future I'd like to 
extract it in a separate python package. It doesn't have to be tied to 
django-hstore like it is now, it can potentially be used with any text 
field.
The idea is that you have something like a (hidden) JSONField or a 
TextField which contains the data of several other VirtualFields.

This would go in the direction of experimenting more with dynamic models in 
django.
I read something in the responses to the django community survey, so it 
seems I'm not the only one who would like to have flexible/dynamic models 
:-)

Federico


On Saturday, April 25, 2015 at 11:48:25 AM UTC+2, Federico Capoano wrote:
>
> Yes I am on 1.7 because 1.8 is not supported yet but I'm working on it.
>
> I'll come back when I'll have more info.
>
> Federico
>
>
> On Thursday, April 23, 2015 at 8:09:42 PM UTC+2, Florian Apolloner wrote:
>>
>>
>>
>> On Thursday, April 23, 2015 at 6:59:58 PM UTC+2, Federico Capoano wrote:
>>>
>>> cls._meta.add_field(self, virtual=True)
>>>
>>> gives:
>>>
>>>   File "/var/www/django-hstore/django_hstore/virtual.py", line 29, in 
>>> contribute_to_class
>>> cls._meta.add_field(self, virtual=True)
>>> TypeError: Error when calling the metaclass bases
>>> add_field() got an unexpected keyword argument 'virtual'
>>>
>>>
>> This should definitely work on master/1.8: 
>> https://github.com/django/django/blob/master/django/db/models/options.py#L294
>>  
>> -- seems like you are on 1.7 or something.
>>
>> cheers,
>> florian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1d827e48-34d5-4db7-94e7-754df9f4241c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: django.channels: "async" for Django

[Federico Capoano]

Hi Andrew,

i have been taking a look at your work because I find it interesting and I 
also think that django should have some core mechanism for integrating 
websockets.

I was wondering, have you also tried one of the other popular django 
websocket apps? Have you found any interesting one that inspired your work?

Federico



On Friday, June 19, 2015 at 3:07:29 AM UTC+2, Andrew Godwin wrote:
>
> The only threading is when you're using runserver - normally I'd suggest 
> this is a multi-process model instead.
>
> In my view, they already are namespaced, as well; the communication/socket 
> channels live under django.wsgi and django.websocket prefixes, whereas the 
> custom ones don't (perhaps we could suggest that people prefix them with 
> their app name). I think enforcing separation beyond this isn't sensible if 
> the behaviour is going to be exactly the same anyway.
>
> Andrew
>
> On Thu, Jun 18, 2015 at 12:47 PM, BearXu  
> wrote:
>
>> It might be a better idea to set the task-pool and intersocket--async 
>> into two namespaces. It would become confused if we put them together under 
>> the "Channel". Like in Java, Channel is like a wrapped-socket, but here in 
>> django-channel, it also means a task queue.
>>
>>
>> On 17 June 2015 at 23:40, BearXu  wrote:
>>
>>> Sorry, I haven't read the source code. But I basically read the doc. 
>>> Thus in a multithreaded env, we will use Redis or something to implement 
>>> the one to one chat function, am I right?
>>>
>>> Thus people might be confused with the different channels' definitions.
>>>
>>> On 16 June 2015 at 08:51, Andrew Godwin >> > wrote:
>>>
 The in-memory backend is not meant to enable any new functionality, 
 just allow things to run in-place as it currently works - any cross-socket 
 communication would be disabled in that mode. You can already see this in 
 my beta code - trying to run the WebSocket or Worker processes in memory 
 mode will just fail.

 Andrew

 On Tue, Jun 16, 2015 at 12:48 AM, BearXu  wrote:

> Personally I don't agree with implementing the Channel using in-memory 
> mode first. Writing code to communicate among  sockets will become very 
> difficult for me(ordinary developer). If we see async in Django as a tool 
> instead of a coding style, I think Redis/NoSql/Database backend/ is good 
> enough as a first step.
>
> On 15 June 2015 at 03:25, Andrew Godwin  > wrote:
>
>> Hello everyone,
>>
>> I've been formulating a plan to decouple Django from the 
>> request-response cycle for a couple of years now and after discussing it 
>> with some people at DjangoCon Europe I thought it was high time that I 
>> actually wrote it up, wrote some code, and solicited feedback.
>>
>> So here it is. The proposal is very long, and is in a Gist here: 
>> https://gist.github.com/andrewgodwin/b3f826a879eb84a70625
>>
>> As a top-level summary:
>>
>>  - Fully backwards-compatible
>>  - No actual "in-process" async like asyncio/greenlets (apart from a 
>> couple of optional "interface server" modules where you can pick your 
>> async 
>> implementation)
>>  - New base unit of Django work is no longer the view, it is the 
>> channel consumer
>>  - Supports WebSockets, long-polling, offsetting complex tasks 
>> outside views and more
>>  - Code looks and works the same - since it'll be a worker model, 
>> there's no need to retrofit any core Django APIs for async stuff.
>>
>> To help write the proposal and also to help you understand it, I've 
>> semi-implemented the proposal over here as a third-party app: 
>> https://github.com/andrewgodwin/django-channels
>>
>> While the goal is to merge this into Django proper, prototyping and 
>> iterating outside is much faster and more accessible for everyone 
>> involved, 
>> and the goal is to get it working on 1.7 and 1.8 (currently only 1.8).
>>
>> The basic chat example in the README of that project is up and 
>> running on this code (and the Redis channel backend) here: 
>> http://www.aeracode.org/chat/
>>
>> Any and all feedback is welcomed, in this thread is probably best. 
>> This is by no means a final plan, but I hope it will lead us towards 
>> something that will make Django much more suitable for a modern Web.
>>
>> (I'm about to get on a long flight to Tokyo, so I may not be 
>> immediately responsive! And my example might crash, it's only running 
>> the 
>> websocket server in screen!)
>>
>> Andrew
>>
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving 

Re: Feature: Support Server-Sent Events

[Federico Capoano]

Hey Emil,

On Sunday, May 31, 2015 at 10:52:26 AM UTC+2, Emil Stenström wrote:
... 

> Also, I don't think you would need to mix redis (or any other persistent 
> storage) into this. The connected clients could simply be stored in an 
> in-memory array, that is discarded if the server crashes. When the server 
> is started again the clients will automatically connect again, and the list 
> of clients would be built up again.
>

How would a distributed setup be handled by this approach?
I mean: multiple instances of the sse behind a loadbalancer.

Federico 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a421b8d5-4794-42b5-a1a6-289a9d476d78%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: virtual fields and the migration framework

[Federico Capoano]

Yes I am on 1.7 because 1.8 is not supported yet but I'm working on it.

I'll come back when I'll have more info.

Federico


On Thursday, April 23, 2015 at 8:09:42 PM UTC+2, Florian Apolloner wrote:
>
>
>
> On Thursday, April 23, 2015 at 6:59:58 PM UTC+2, Federico Capoano wrote:
>>
>> cls._meta.add_field(self, virtual=True)
>>
>> gives:
>>
>>   File "/var/www/django-hstore/django_hstore/virtual.py", line 29, in 
>> contribute_to_class
>> cls._meta.add_field(self, virtual=True)
>> TypeError: Error when calling the metaclass bases
>> add_field() got an unexpected keyword argument 'virtual'
>>
>>
> This should definitely work on master/1.8: 
> https://github.com/django/django/blob/master/django/db/models/options.py#L294 
> -- seems like you are on 1.7 or something.
>
> cheers,
> florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0c06e657-5963-4731-a414-807265b1c9c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: virtual fields and the migration framework

[Federico Capoano]

I think the correct syntax is:

cls._meta.add_virtual_field(self)

The problem is that if I use this syntax it seems that the admin ignores 
these fields, what happens is that the changes I make to the virtual fields 
are ignored.

These are the test results related to the VirtualField implementation:

Creating test database for alias 'default'...
.FFF..E...E.F.F..
==
ERROR: test_create (django_hstore_tests.tests.SchemaTests)
--
Traceback (most recent call last):
  File "/var/www/django-hstore/tests/django_hstore_tests/tests.py", line 
1427, in test_create
text='create3'
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/models/manager.py",
 
line 92, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/models/query.py",
 
line 370, in create
obj = self.model(**kwargs)
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/models/base.py",
 
line 452, in __init__
raise TypeError("'%s' is an invalid keyword argument for this function" 
% list(kwargs)[0])
TypeError: 'boolean_true' is an invalid keyword argument for this function

==
ERROR: test_migrations (django_hstore_tests.tests.SchemaTests)
failing test for https://github.com/djangonauts/django-hstore/issues/103
--
Traceback (most recent call last):
  File "/var/www/django-hstore/tests/django_hstore_tests/tests.py", line 
1565, in test_migrations
call_command('migrate', 'django_hstore_tests')
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/core/management/__init__.py",
 
line 115, in call_command
return klass.execute(*args, **defaults)
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/core/management/base.py",
 
line 338, in execute
output = self.handle(*args, **options)
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py",
 
line 161, in handle
executor.migrate(targets, plan, fake=options.get("fake", False))
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/migrations/executor.py",
 
line 68, in migrate
self.apply_migration(migration, fake=fake)
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/migrations/executor.py",
 
line 102, in apply_migration
migration.apply(project_state, schema_editor)
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/migrations/migration.py",
 
line 108, in apply
operation.database_forwards(self.app_label, schema_editor, 
project_state, new_state)
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/migrations/operations/fields.py",
 
line 127, in database_forwards
from_field = from_model._meta.get_field_by_name(self.name)[0]
  File 
"/home/nemesis/.virtualenvs/django-hstore/local/lib/python2.7/site-packages/django/db/models/options.py",
 
line 420, in get_field_by_name
% (self.object_name, name))
FieldDoesNotExist: SchemaDataBag has no field named u'datetime'

==
FAIL: test_admin_add (django_hstore_tests.tests.SchemaTests)
--
Traceback (most recent call last):
  File "/var/www/django-hstore/tests/django_hstore_tests/tests.py", line 
1305, in test_admin_add
self.assertEqual(d.number, 3)
AssertionError: 0 != 3

==
FAIL: test_admin_add_utf8 (django_hstore_tests.tests.SchemaTests)
--
Traceback (most recent call last):
  File "/var/www/django-hstore/tests/django_hstore_tests/tests.py", line 
1317, in test_admin_add_utf8
self.assertEqual(d.number, 3)
AssertionError: 0 != 3

==
FAIL: test_admin_change (django_hstore_tests.tests.SchemaTests)
--
Traceback (most recent call last):
  File "/var/www/django-hstore/tests/django_hstore_tests/tests.py", line 
1334, in test_admin_change
self.assertEqual(d.number, 6)
AssertionError: 1 != 6

==
FAIL: test_reload_schema (django_hstore_tests.tests.SchemaTests)
--
Traceback (most recent call last):
  

Re: virtual fields and the migration framework

[Federico Capoano]

cls._meta.add_field(self, virtual=True)

gives:

  File "/var/www/django-hstore/django_hstore/virtual.py", line 29, in 
contribute_to_class
cls._meta.add_field(self, virtual=True)
TypeError: Error when calling the metaclass bases
add_field() got an unexpected keyword argument 'virtual'

I remember that I already tried flagging the field as virtual, but the 
result was not what I wanted, that is a field that would also show up in 
the admin.

There is an interesting ticket:

*Provide real support for virtual fields*
https://code.djangoproject.com/ticket/16508

Federico


On Wednesday, April 22, 2015 at 9:54:39 AM UTC+2, Federico Capoano wrote:
>
> Hi Markus,
>
> thank you very much for your suggestion. I'll try it later this afternoon 
> and come back to report.
>
> Federico
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/33c7aa8d-a7a6-4746-9f49-a1f57f9e7969%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Idea/request for help: Django Developers Community Survey

[Federico Capoano]

Great idea.
The questions look good enough to me.

I love django-apps or libraries shipped in python packages. One of the 
reason I love Django is the fact that it didn't frenetically add stuff to 
the framework just because it's cool.
The good thing of python packages is that you can get some data from pypi 
(downloads) and github (clones). But we can't do that with contrib apps 
unfortunately.

Federico


On Saturday, April 18, 2015 at 1:00:13 AM UTC+2, Tim Graham wrote:
>
> I had an idea to conduct a survey to get a sense of how developers are 
> using Django. I first had the idea when the question of maintenance of the 
> Oracle GIS backend came up. We really have no idea whether or not anyone is 
> actually using that backend, and it would be helpful to know so we are not 
> wasting resources on unused features. Also there's the question of how 
> often to release Django. I think it would be nice to make that decision 
> based on some data from the community.
>
> Is anyone is interested in coordinating such a survey (collating 
> questions, preparing the online survey, etc.).
>
> Some question ideas to get started:
>
> Which database backend(s) do you use?
> [ ] Checkbox for each one
>
> Which contrib apps do you use?
> [ ] Checkbox for each one
>
> How often would you like to see new major version of Django released?
> [ ] List some options, probably between 6 and 9 months.
>
> Describe which version of Django you use (check all the apply):
> [ ] I only use long-term support releases.
> [ ] I upgrade to the latest stable version as quickly as possible.
> [ ] I run off of master.
> [ ] I upgrade Django when the version I am using is nearing the end of its 
> support (or after).
> [ ] I don't bother upgrading Django, even if it becomes unsupported.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8a4ac444-2b4e-4321-ad34-8e8e6f8131ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: virtual fields and the migration framework

[Federico Capoano]

Hi Markus,

thank you very much for your suggestion. I'll try it later this afternoon 
and come back to report.

Federico


On Wednesday, April 22, 2015 at 12:54:32 AM UTC+2, Markus Holtermann wrote:
>
> Hey Federico,
>
> I just had a brief look at the code. If I understand 
> the HStoreVirtualMixin.contribute_to_class() correctly and its implications 
> regarding add_field(), I'd try to use cls._meta.add_field(self, 
> virtual=True) instead of a simple cls._meta.add_field(self). That way the 
> virtual fields shouldn't show up in migrations at all: 
> https://github.com/django/django/blob/master/django/db/migrations/state.py#L357
>  
> -- The way the e.g. DictionaryField is implemented, when you put the schema 
> into the deconstruct() output, contribute_to_class() should still be called 
> generating the additional fields on its own.
>
> Note that this is just a shot in the dark. I didn't try it out myself.
>
> /Markus
>
> On Tuesday, April 21, 2015 at 7:53:04 PM UTC+2, Federico Capoano wrote:
>>
>> Hi everyone,
>>
>> some time ago I submitted this ticket: 
>> https://code.djangoproject.com/ticket/23159 - because we had problems 
>> with the migration framework.
>>
>> To sum up, the django-hstore 
>> <https://github.com/djangonauts/django-hstore> library has a 
>> VirtualField implementation which maps hstore keys (by supplying a schema 
>> parameter) to virtual fields so that they can be used on models, the admin, 
>> and even in django-rest-framework APIs. 
>>
>> After some interactions with Andrew Godwin, I implemented his suggestion, 
>> that was to let db_type() return None, which worked well, until this other 
>> issue came up: https://github.com/djangonauts/django-hstore/issues/103
>>
>> In that case the migration framework would throw an error even if db_type 
>> returned None.
>>
>> Luckily, Tim Graham suggested me how to reach the no-op branch of the 
>> migration framework: 
>> https://github.com/django/django/blob/190afb86187e42c449afe626cff31f65b4781aa2/django/db/backends/base/schema.py#L470-L475
>>
>> I met Florian Apolloner at Pycon Italy, so I took advantage of his 
>> presence to ask some questions and hear some feedback, which I really 
>> appreciated and which really helped me.
>>
>> I have been able to replicate the bug in a test case and implement the 
>> workaround, now comes the interesting part: here's the workaround code:
>>
>> https://github.com/djangonauts/django-hstore/commit/47f5aa05834365e2d6c41234e03b028a827ee91c
>>
>> As you can see, I had to add quite some ugly code just to enter that 
>> no-op branch there must be a better way! :-D
>>
>> I talked with Florian about the possibility to have some way to flag 
>> virtual fields in order to enter that no-op branch more easily and he was 
>> not very sure about it but nonetheless he encouraged me to bring up this 
>> issue here. One of the reasons he was not sure is "what happens if a field 
>> has the skip flag and then it doesn't or viceversa?"
>>
>> So I thought about proposing a definition of virtual field:
>>
>> "A virtual field is a model field which it correlates to one or multiple 
>> concrete fields, but doesn't add or alter columns in the database."
>>
>> By following this definition I suppose we can:
>>
>>- allow the migration framework to enter the no-op branch if 
>>analyzing a virtual field
>>- if a concrete field becomes virtual (i can't immagine the use case 
>>but i'm reasoning just in case), the migration framework will drop the 
>>column, and the developer will have to adjust the migration in order to 
>>avoid losing data
>>- if a virtual field becomes concrete, the migration framework will 
>>add the right column and the developer will have to adjust the migration 
>> in 
>>order to correctly fill the concrete field with the data that is probably 
>>stored in another concrete field
>>
>> Thank you for your attention.
>> Best regards
>> Federico Capoano
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c7ef8579-4ba8-4386-8601-ef4d6f376d78%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

virtual fields and the migration framework

[Federico Capoano]

Hi everyone,

some time ago I submitted this 
ticket: https://code.djangoproject.com/ticket/23159 - because we had 
problems with the migration framework.

To sum up, the django-hstore <https://github.com/djangonauts/django-hstore> 
library has a VirtualField implementation which maps hstore keys (by 
supplying a schema parameter) to virtual fields so that they can be used on 
models, the admin, and even in django-rest-framework APIs. 

After some interactions with Andrew Godwin, I implemented his suggestion, 
that was to let db_type() return None, which worked well, until this other 
issue came up: https://github.com/djangonauts/django-hstore/issues/103

In that case the migration framework would throw an error even if db_type 
returned None.

Luckily, Tim Graham suggested me how to reach the no-op branch of the 
migration 
framework: 
https://github.com/django/django/blob/190afb86187e42c449afe626cff31f65b4781aa2/django/db/backends/base/schema.py#L470-L475

I met Florian Apolloner at Pycon Italy, so I took advantage of his presence 
to ask some questions and hear some feedback, which I really appreciated 
and which really helped me.

I have been able to replicate the bug in a test case and implement the 
workaround, now comes the interesting part: here's the workaround code:
https://github.com/djangonauts/django-hstore/commit/47f5aa05834365e2d6c41234e03b028a827ee91c

As you can see, I had to add quite some ugly code just to enter that no-op 
branch there must be a better way! :-D

I talked with Florian about the possibility to have some way to flag 
virtual fields in order to enter that no-op branch more easily and he was 
not very sure about it but nonetheless he encouraged me to bring up this 
issue here. One of the reasons he was not sure is "what happens if a field 
has the skip flag and then it doesn't or viceversa?"

So I thought about proposing a definition of virtual field:

"A virtual field is a model field which it correlates to one or multiple 
concrete fields, but doesn't add or alter columns in the database."

By following this definition I suppose we can:

   - allow the migration framework to enter the no-op branch if analyzing a 
   virtual field
   - if a concrete field becomes virtual (i can't immagine the use case but 
   i'm reasoning just in case), the migration framework will drop the column, 
   and the developer will have to adjust the migration in order to avoid 
   losing data
   - if a virtual field becomes concrete, the migration framework will add 
   the right column and the developer will have to adjust the migration in 
   order to correctly fill the concrete field with the data that is probably 
   stored in another concrete field

Thank you for your attention.
Best regards
Federico Capoano

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1fa02745-5c39-4b18-871e-c89db9c87966%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.