Re: Support for UNLOGGED tables in PostgreSQL

[Shai Berger]

On Monday 20 July 2015 12:42:47 Federico Capoano wrote:
> 
> Just a final note: I think this solution is not the optimal one

I agree. The optimal solution would be for us to be able to define a test 
database with unlogged tables and all the data-integrity-sync options off, 
while the "production" database (production in quotes because it applies also 
to the main development database) keeps sane settings.

Since PostgreSQL defines most of these at the cluster level, this would require 
setting up a separate cluster for testing (Debian makes this easy with its 
pg_createcluster command, but on other systems it would be more involved). But 
on the Django side, this requires the ability to specify accessing a different 
cluster (that is, usually, separate ports on the machine) for testing. We 
currently do not have this ability.

The only danger I see in this is a "slippery slope" -- next thing, people will 
want a separate engine for testing, and before we know it, all testing will be 
run only against Sqlite. I don't think this fear is justified.

Are there good reasons to avoid giving users more control over their test 
databases?

Shai.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

I just changed the URL in the to
http://www.postgresql.org/docs/current/static/non-durability.html as
suggested.

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6Wtm7V6o_DqqJn7ZgJQFNyC0AH%3DrEHb6HyE58NTDrzrXQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Florian Apolloner]

The doc building process includes a linkchecker, if the link ceases to 
exist we will get notified (so the worst thing could be that the content 
changes to something completely different).

On Tuesday, July 21, 2015 at 8:46:29 AM UTC+2, Christian Schmitt wrote:
>
> I mean I made the proposal, but one drawback will be when linking to 
> Postgresql Docs, what if they change the links? I mean that's barely 
> happend in the past, however the "
> http://www.postgresql.org/docs/current/static/non-durability.html; is 
> fairly new. (9.x). 
> So we need to track every major upgrade on Postgresql if the docs changed.
>
> 2015-07-19 22:44 GMT+02:00 Aymeric Augustin  >:
>
>> I agree with pointing to the relevant section of the PostgreSQL 
>> documentation. It will always be more complete, accurate and up-to-date 
>> that what we could write.
>>
>> --
>> Aymeric.
>>
>>
>>
>> > On 19 juil. 2015, at 19:43, Christophe Pettus > > wrote:
>> >
>> > This can be achieved by pointing to the relevant section in the 
>> PostgreSQL documentation with a general "Test execution may be sped up by 
>> adjusting the data integrity parameters in PostgreSQL; be sure to read the 
>> appropriate warnings before making any changes" warning.
>> >
>> > Putting actual recommended settings in the Django documentation seems, 
>> at a minimum, pointlessly duplicative, and ties the Django documentation to 
>> the current state of the world in PostgreSQL gratuitously.
>> >
>> >
>> > On Jul 19, 2015, at 10:32 AM, Luke Plant > > wrote:
>> >
>> >> I agree with Federico on this - as long as we slap a big warning on it 
>> — "This is dangerous - it could make your database more likely to lose data 
>> or become corrupted, only use on a development machine where you can 
>> restore the entire contents of all databases in the cluster easily" — I 
>> don't see a problem in this being in our docs.
>> >>
>> >> If people refuse to read a clear warning, they shouldn't be doing web 
>> development. They are just as likely to find similar instructions on the 
>> internet, but without warnings, and having it in our docs with the warning 
>> will be helpful.
>> >>
>> >> Having a fast test suite is such an important part of development that 
>> it shouldn't be held back by  attempting to protect the world from people 
>> who cannot be helped.
>> >>
>> >> Luke
>> >>
>> >> On 16/07/15 16:49, Christophe Pettus wrote:
>> >>> On Jul 16, 2015, at 1:16 AM, Federico Capoano > >
>> >>> wrote:
>> >>>
>> >>>
>>  I also don't like the idea of believing django users are too stupid 
>> to
>>  understand that this advice si valid for development only. Generally
>>  python and django users are intelligent enough to properly read the
>>  docs and understand what's written on it.
>> 
>> >>> It's not a matter of being "intelligent" or not.  Developers are busy 
>> and can simply google things, see a particular line, and drop it in without 
>> fully understanding exactly what is going on.  (Simply read this group for 
>> a while if you don't believe this to be the case!)  People already turn off 
>> fsync, in production, after having read the PostgreSQL documentation, 
>> without actually realizing that they've put their database in danger.
>> >>>
>> >>> Among other things, developers often have local data in their 
>> PostgreSQL instance that is valuable, and advising them to do a setting 
>> that runs the risk of them losing that data seems like a bad idea.
>> >>>
>> >>> The Django documentation is not the place to go into the 
>> ramifications of fsync (or even synchronous_commit, although that's 
>> significantly less risky).
>> >>>
>> >>> --
>> >>> -- Christophe Pettus
>> >>>
>> >>> x...@thebuild.com 
>> >>>
>> >>>
>> >>>
>> >>
>> >> --
>> >> "I was sad because I had no shoes, until I met a man who had no
>> >> feet. So I said, "Got any shoes you're not using?"  (Steven Wright)
>> >>
>> >> Luke Plant ||
>> >> http://lukeplant.me.uk/
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google 
>> Groups "Django developers (Contributions to Django itself)" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send 
>> an email to django-develop...@googlegroups.com .
>> >> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> >> Visit this group at http://groups.google.com/group/django-developers.
>> >> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/55ABDF21.9060106%40cantab.net
>> .
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > -- Christophe Pettus
>> >   x...@thebuild.com 
>> >
>> > --
>> > You received this message because you are subscribed to the Google 
>> Groups "Django developers  (Contributions to Django itself)" group.
>> > To unsubscribe from this group and stop receiving emails from it, 

Re: Support for UNLOGGED tables in PostgreSQL

[Christophe Pettus]

I made a small comment about the URL form.  To my ear, the text could use a bit 
of wordsmithing, but I think it's the right content.

On Jul 20, 2015, at 3:10 AM, Federico Capoano  
wrote:

> Errata, PR link is https://github.com/django/django/pull/5021 - sorry
> 
> On Mon, Jul 20, 2015 at 11:42 AM, Federico Capoano
>  wrote:
>> Thank you for all the feedback.
>> 
>> I opened this PR with the proposed addition to the docs:
>> https://github.com/django/django/compare/master...nemesisdesign:patch-1
>> 
>> Let me know if there's anything I can improve.
>> 
>> Just a final note: I think this solution is not the optimal one, let
>> me explain you why. The main reason I came on this list asking for
>> information about UNLOGGED tables in postgresql was exactly because of
>> that page which suggests a few solutions. I spent quite many hours
>> trying all of them (including running the entire DB in RAM), and
>> measuring test execution time with each one of those settings. In the
>> end I figured out that the only settings worth touching in my setup
>> are those 3 I mentioned before.
>> 
>> In order to share what I learnt and hoping to avoid some pain to
>> people that will come on the same path, I quickly wrote a blog post
>> with these suggestions:
>> http://nemesisdesign.net/blog/coding/how-to-speed-up-tests-django-postgresql/
>> 
>> Nothing new really, I found a few other blog posts with similar
>> suggestions, but the suggestions were scattered on different pages and
>> they didn't mention which changes were the most effective ones. That's
>> why I felt the need of writing this.
>> 
>> And by the way, I'm really happy of the outcome!
>> 
>> Federico
>> 
>> On Mon, Jul 20, 2015 at 2:47 AM, Curtis Maloney
>>  wrote:
>>> I second what Aymeric say rather than take on the burden of
>>> maintaining correct warnings, let's point at the people whose
>>> responsibility it really is :)
>>> 
>>> --
>>> Curtis
>>> 
>>> On 20 July 2015 at 06:44, Aymeric Augustin
>>>  wrote:
 I agree with pointing to the relevant section of the PostgreSQL 
 documentation. It will always be more complete, accurate and up-to-date 
 that what we could write.
 
 --
 Aymeric.
 
 
 
> On 19 juil. 2015, at 19:43, Christophe Pettus  wrote:
> 
> This can be achieved by pointing to the relevant section in the 
> PostgreSQL documentation with a general "Test execution may be sped up by 
> adjusting the data integrity parameters in PostgreSQL; be sure to read 
> the appropriate warnings before making any changes" warning.
> 
> Putting actual recommended settings in the Django documentation seems, at 
> a minimum, pointlessly duplicative, and ties the Django documentation to 
> the current state of the world in PostgreSQL gratuitously.
> 
> 
> On Jul 19, 2015, at 10:32 AM, Luke Plant  wrote:
> 
>> I agree with Federico on this - as long as we slap a big warning on it — 
>> "This is dangerous - it could make your database more likely to lose 
>> data or become corrupted, only use on a development machine where you 
>> can restore the entire contents of all databases in the cluster easily" 
>> — I don't see a problem in this being in our docs.
>> 
>> If people refuse to read a clear warning, they shouldn't be doing web 
>> development. They are just as likely to find similar instructions on the 
>> internet, but without warnings, and having it in our docs with the 
>> warning will be helpful.
>> 
>> Having a fast test suite is such an important part of development that 
>> it shouldn't be held back by  attempting to protect the world from 
>> people who cannot be helped.
>> 
>> Luke
>> 
>> On 16/07/15 16:49, Christophe Pettus wrote:
>>> On Jul 16, 2015, at 1:16 AM, Federico Capoano 
>>> 
>>> wrote:
>>> 
>>> 
 I also don't like the idea of believing django users are too stupid to
 understand that this advice si valid for development only. Generally
 python and django users are intelligent enough to properly read the
 docs and understand what's written on it.
 
>>> It's not a matter of being "intelligent" or not.  Developers are busy 
>>> and can simply google things, see a particular line, and drop it in 
>>> without fully understanding exactly what is going on.  (Simply read 
>>> this group for a while if you don't believe this to be the case!)  
>>> People already turn off fsync, in production, after having read the 
>>> PostgreSQL documentation, without actually realizing that they've put 
>>> their database in danger.
>>> 
>>> Among other things, developers often have local data in their 
>>> 

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Errata, PR link is https://github.com/django/django/pull/5021 - sorry

On Mon, Jul 20, 2015 at 11:42 AM, Federico Capoano
 wrote:
> Thank you for all the feedback.
>
> I opened this PR with the proposed addition to the docs:
> https://github.com/django/django/compare/master...nemesisdesign:patch-1
>
> Let me know if there's anything I can improve.
>
> Just a final note: I think this solution is not the optimal one, let
> me explain you why. The main reason I came on this list asking for
> information about UNLOGGED tables in postgresql was exactly because of
> that page which suggests a few solutions. I spent quite many hours
> trying all of them (including running the entire DB in RAM), and
> measuring test execution time with each one of those settings. In the
> end I figured out that the only settings worth touching in my setup
> are those 3 I mentioned before.
>
> In order to share what I learnt and hoping to avoid some pain to
> people that will come on the same path, I quickly wrote a blog post
> with these suggestions:
> http://nemesisdesign.net/blog/coding/how-to-speed-up-tests-django-postgresql/
>
> Nothing new really, I found a few other blog posts with similar
> suggestions, but the suggestions were scattered on different pages and
> they didn't mention which changes were the most effective ones. That's
> why I felt the need of writing this.
>
> And by the way, I'm really happy of the outcome!
>
> Federico
>
> On Mon, Jul 20, 2015 at 2:47 AM, Curtis Maloney
>  wrote:
>> I second what Aymeric say rather than take on the burden of
>> maintaining correct warnings, let's point at the people whose
>> responsibility it really is :)
>>
>> --
>> Curtis
>>
>> On 20 July 2015 at 06:44, Aymeric Augustin
>>  wrote:
>>> I agree with pointing to the relevant section of the PostgreSQL 
>>> documentation. It will always be more complete, accurate and up-to-date 
>>> that what we could write.
>>>
>>> --
>>> Aymeric.
>>>
>>>
>>>
 On 19 juil. 2015, at 19:43, Christophe Pettus  wrote:

 This can be achieved by pointing to the relevant section in the PostgreSQL 
 documentation with a general "Test execution may be sped up by adjusting 
 the data integrity parameters in PostgreSQL; be sure to read the 
 appropriate warnings before making any changes" warning.

 Putting actual recommended settings in the Django documentation seems, at 
 a minimum, pointlessly duplicative, and ties the Django documentation to 
 the current state of the world in PostgreSQL gratuitously.


 On Jul 19, 2015, at 10:32 AM, Luke Plant  wrote:

> I agree with Federico on this - as long as we slap a big warning on it — 
> "This is dangerous - it could make your database more likely to lose data 
> or become corrupted, only use on a development machine where you can 
> restore the entire contents of all databases in the cluster easily" — I 
> don't see a problem in this being in our docs.
>
> If people refuse to read a clear warning, they shouldn't be doing web 
> development. They are just as likely to find similar instructions on the 
> internet, but without warnings, and having it in our docs with the 
> warning will be helpful.
>
> Having a fast test suite is such an important part of development that it 
> shouldn't be held back by  attempting to protect the world from people 
> who cannot be helped.
>
> Luke
>
> On 16/07/15 16:49, Christophe Pettus wrote:
>> On Jul 16, 2015, at 1:16 AM, Federico Capoano 
>> 
>> wrote:
>>
>>
>>> I also don't like the idea of believing django users are too stupid to
>>> understand that this advice si valid for development only. Generally
>>> python and django users are intelligent enough to properly read the
>>> docs and understand what's written on it.
>>>
>> It's not a matter of being "intelligent" or not.  Developers are busy 
>> and can simply google things, see a particular line, and drop it in 
>> without fully understanding exactly what is going on.  (Simply read this 
>> group for a while if you don't believe this to be the case!)  People 
>> already turn off fsync, in production, after having read the PostgreSQL 
>> documentation, without actually realizing that they've put their 
>> database in danger.
>>
>> Among other things, developers often have local data in their PostgreSQL 
>> instance that is valuable, and advising them to do a setting that runs 
>> the risk of them losing that data seems like a bad idea.
>>
>> The Django documentation is not the place to go into the ramifications 
>> of fsync (or even synchronous_commit, although that's significantly less 
>> risky).
>>
>> --
>> -- 

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Thank you for all the feedback.

I opened this PR with the proposed addition to the docs:
https://github.com/django/django/compare/master...nemesisdesign:patch-1

Let me know if there's anything I can improve.

Just a final note: I think this solution is not the optimal one, let
me explain you why. The main reason I came on this list asking for
information about UNLOGGED tables in postgresql was exactly because of
that page which suggests a few solutions. I spent quite many hours
trying all of them (including running the entire DB in RAM), and
measuring test execution time with each one of those settings. In the
end I figured out that the only settings worth touching in my setup
are those 3 I mentioned before.

In order to share what I learnt and hoping to avoid some pain to
people that will come on the same path, I quickly wrote a blog post
with these suggestions:
http://nemesisdesign.net/blog/coding/how-to-speed-up-tests-django-postgresql/

Nothing new really, I found a few other blog posts with similar
suggestions, but the suggestions were scattered on different pages and
they didn't mention which changes were the most effective ones. That's
why I felt the need of writing this.

And by the way, I'm really happy of the outcome!

Federico

On Mon, Jul 20, 2015 at 2:47 AM, Curtis Maloney
 wrote:
> I second what Aymeric say rather than take on the burden of
> maintaining correct warnings, let's point at the people whose
> responsibility it really is :)
>
> --
> Curtis
>
> On 20 July 2015 at 06:44, Aymeric Augustin
>  wrote:
>> I agree with pointing to the relevant section of the PostgreSQL 
>> documentation. It will always be more complete, accurate and up-to-date that 
>> what we could write.
>>
>> --
>> Aymeric.
>>
>>
>>
>>> On 19 juil. 2015, at 19:43, Christophe Pettus  wrote:
>>>
>>> This can be achieved by pointing to the relevant section in the PostgreSQL 
>>> documentation with a general "Test execution may be sped up by adjusting 
>>> the data integrity parameters in PostgreSQL; be sure to read the 
>>> appropriate warnings before making any changes" warning.
>>>
>>> Putting actual recommended settings in the Django documentation seems, at a 
>>> minimum, pointlessly duplicative, and ties the Django documentation to the 
>>> current state of the world in PostgreSQL gratuitously.
>>>
>>>
>>> On Jul 19, 2015, at 10:32 AM, Luke Plant  wrote:
>>>
 I agree with Federico on this - as long as we slap a big warning on it — 
 "This is dangerous - it could make your database more likely to lose data 
 or become corrupted, only use on a development machine where you can 
 restore the entire contents of all databases in the cluster easily" — I 
 don't see a problem in this being in our docs.

 If people refuse to read a clear warning, they shouldn't be doing web 
 development. They are just as likely to find similar instructions on the 
 internet, but without warnings, and having it in our docs with the warning 
 will be helpful.

 Having a fast test suite is such an important part of development that it 
 shouldn't be held back by  attempting to protect the world from people who 
 cannot be helped.

 Luke

 On 16/07/15 16:49, Christophe Pettus wrote:
> On Jul 16, 2015, at 1:16 AM, Federico Capoano 
> wrote:
>
>
>> I also don't like the idea of believing django users are too stupid to
>> understand that this advice si valid for development only. Generally
>> python and django users are intelligent enough to properly read the
>> docs and understand what's written on it.
>>
> It's not a matter of being "intelligent" or not.  Developers are busy and 
> can simply google things, see a particular line, and drop it in without 
> fully understanding exactly what is going on.  (Simply read this group 
> for a while if you don't believe this to be the case!)  People already 
> turn off fsync, in production, after having read the PostgreSQL 
> documentation, without actually realizing that they've put their database 
> in danger.
>
> Among other things, developers often have local data in their PostgreSQL 
> instance that is valuable, and advising them to do a setting that runs 
> the risk of them losing that data seems like a bad idea.
>
> The Django documentation is not the place to go into the ramifications of 
> fsync (or even synchronous_commit, although that's significantly less 
> risky).
>
> --
> -- Christophe Pettus
>
> x...@thebuild.com
>
>
>

 --
 "I was sad because I had no shoes, until I met a man who had no
 feet. So I said, "Got any shoes you're not using?"  (Steven Wright)

 Luke Plant ||
 http://lukeplant.me.uk/

 --
 You received 

Re: Support for UNLOGGED tables in PostgreSQL

[Curtis Maloney]

I second what Aymeric say rather than take on the burden of
maintaining correct warnings, let's point at the people whose
responsibility it really is :)

--
Curtis

On 20 July 2015 at 06:44, Aymeric Augustin
 wrote:
> I agree with pointing to the relevant section of the PostgreSQL 
> documentation. It will always be more complete, accurate and up-to-date that 
> what we could write.
>
> --
> Aymeric.
>
>
>
>> On 19 juil. 2015, at 19:43, Christophe Pettus  wrote:
>>
>> This can be achieved by pointing to the relevant section in the PostgreSQL 
>> documentation with a general "Test execution may be sped up by adjusting the 
>> data integrity parameters in PostgreSQL; be sure to read the appropriate 
>> warnings before making any changes" warning.
>>
>> Putting actual recommended settings in the Django documentation seems, at a 
>> minimum, pointlessly duplicative, and ties the Django documentation to the 
>> current state of the world in PostgreSQL gratuitously.
>>
>>
>> On Jul 19, 2015, at 10:32 AM, Luke Plant  wrote:
>>
>>> I agree with Federico on this - as long as we slap a big warning on it — 
>>> "This is dangerous - it could make your database more likely to lose data 
>>> or become corrupted, only use on a development machine where you can 
>>> restore the entire contents of all databases in the cluster easily" — I 
>>> don't see a problem in this being in our docs.
>>>
>>> If people refuse to read a clear warning, they shouldn't be doing web 
>>> development. They are just as likely to find similar instructions on the 
>>> internet, but without warnings, and having it in our docs with the warning 
>>> will be helpful.
>>>
>>> Having a fast test suite is such an important part of development that it 
>>> shouldn't be held back by  attempting to protect the world from people who 
>>> cannot be helped.
>>>
>>> Luke
>>>
>>> On 16/07/15 16:49, Christophe Pettus wrote:
 On Jul 16, 2015, at 1:16 AM, Federico Capoano 
 wrote:


> I also don't like the idea of believing django users are too stupid to
> understand that this advice si valid for development only. Generally
> python and django users are intelligent enough to properly read the
> docs and understand what's written on it.
>
 It's not a matter of being "intelligent" or not.  Developers are busy and 
 can simply google things, see a particular line, and drop it in without 
 fully understanding exactly what is going on.  (Simply read this group for 
 a while if you don't believe this to be the case!)  People already turn 
 off fsync, in production, after having read the PostgreSQL documentation, 
 without actually realizing that they've put their database in danger.

 Among other things, developers often have local data in their PostgreSQL 
 instance that is valuable, and advising them to do a setting that runs the 
 risk of them losing that data seems like a bad idea.

 The Django documentation is not the place to go into the ramifications of 
 fsync (or even synchronous_commit, although that's significantly less 
 risky).

 --
 -- Christophe Pettus

 x...@thebuild.com



>>>
>>> --
>>> "I was sad because I had no shoes, until I met a man who had no
>>> feet. So I said, "Got any shoes you're not using?"  (Steven Wright)
>>>
>>> Luke Plant ||
>>> http://lukeplant.me.uk/
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/55ABDF21.9060106%40cantab.net.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> -- Christophe Pettus
>>   x...@thebuild.com
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/648BE2F3-E869-4E9D-BCB9-248E425D5A1C%40thebuild.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to 

Re: Support for UNLOGGED tables in PostgreSQL

[Aymeric Augustin]

I agree with pointing to the relevant section of the PostgreSQL documentation. 
It will always be more complete, accurate and up-to-date that what we could 
write.

-- 
Aymeric.



> On 19 juil. 2015, at 19:43, Christophe Pettus  wrote:
> 
> This can be achieved by pointing to the relevant section in the PostgreSQL 
> documentation with a general "Test execution may be sped up by adjusting the 
> data integrity parameters in PostgreSQL; be sure to read the appropriate 
> warnings before making any changes" warning.
> 
> Putting actual recommended settings in the Django documentation seems, at a 
> minimum, pointlessly duplicative, and ties the Django documentation to the 
> current state of the world in PostgreSQL gratuitously.
> 
>   
> On Jul 19, 2015, at 10:32 AM, Luke Plant  wrote:
> 
>> I agree with Federico on this - as long as we slap a big warning on it — 
>> "This is dangerous - it could make your database more likely to lose data or 
>> become corrupted, only use on a development machine where you can restore 
>> the entire contents of all databases in the cluster easily" — I don't see a 
>> problem in this being in our docs.
>> 
>> If people refuse to read a clear warning, they shouldn't be doing web 
>> development. They are just as likely to find similar instructions on the 
>> internet, but without warnings, and having it in our docs with the warning 
>> will be helpful.
>> 
>> Having a fast test suite is such an important part of development that it 
>> shouldn't be held back by  attempting to protect the world from people who 
>> cannot be helped.
>> 
>> Luke
>> 
>> On 16/07/15 16:49, Christophe Pettus wrote:
>>> On Jul 16, 2015, at 1:16 AM, Federico Capoano 
>>> wrote:
>>> 
>>> 
 I also don't like the idea of believing django users are too stupid to
 understand that this advice si valid for development only. Generally
 python and django users are intelligent enough to properly read the
 docs and understand what's written on it.
 
>>> It's not a matter of being "intelligent" or not.  Developers are busy and 
>>> can simply google things, see a particular line, and drop it in without 
>>> fully understanding exactly what is going on.  (Simply read this group for 
>>> a while if you don't believe this to be the case!)  People already turn off 
>>> fsync, in production, after having read the PostgreSQL documentation, 
>>> without actually realizing that they've put their database in danger.
>>> 
>>> Among other things, developers often have local data in their PostgreSQL 
>>> instance that is valuable, and advising them to do a setting that runs the 
>>> risk of them losing that data seems like a bad idea.
>>> 
>>> The Django documentation is not the place to go into the ramifications of 
>>> fsync (or even synchronous_commit, although that's significantly less 
>>> risky).
>>> 
>>> --
>>> -- Christophe Pettus
>>> 
>>> x...@thebuild.com
>>> 
>>> 
>>> 
>> 
>> -- 
>> "I was sad because I had no shoes, until I met a man who had no 
>> feet. So I said, "Got any shoes you're not using?"  (Steven Wright)
>> 
>> Luke Plant || 
>> http://lukeplant.me.uk/
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/55ABDF21.9060106%40cantab.net.
>> For more options, visit https://groups.google.com/d/optout.
> 
> --
> -- Christophe Pettus
>   x...@thebuild.com
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/648BE2F3-E869-4E9D-BCB9-248E425D5A1C%40thebuild.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 

Re: Support for UNLOGGED tables in PostgreSQL

[Christophe Pettus]

This can be achieved by pointing to the relevant section in the PostgreSQL 
documentation with a general "Test execution may be sped up by adjusting the 
data integrity parameters in PostgreSQL; be sure to read the appropriate 
warnings before making any changes" warning.

Putting actual recommended settings in the Django documentation seems, at a 
minimum, pointlessly duplicative, and ties the Django documentation to the 
current state of the world in PostgreSQL gratuitously.


On Jul 19, 2015, at 10:32 AM, Luke Plant  wrote:

> I agree with Federico on this - as long as we slap a big warning on it — 
> "This is dangerous - it could make your database more likely to lose data or 
> become corrupted, only use on a development machine where you can restore the 
> entire contents of all databases in the cluster easily" — I don't see a 
> problem in this being in our docs.
> 
> If people refuse to read a clear warning, they shouldn't be doing web 
> development. They are just as likely to find similar instructions on the 
> internet, but without warnings, and having it in our docs with the warning 
> will be helpful.
> 
> Having a fast test suite is such an important part of development that it 
> shouldn't be held back by  attempting to protect the world from people who 
> cannot be helped.
> 
> Luke
> 
> On 16/07/15 16:49, Christophe Pettus wrote:
>> On Jul 16, 2015, at 1:16 AM, Federico Capoano 
>>  wrote:
>> 
>> 
>>> I also don't like the idea of believing django users are too stupid to
>>> understand that this advice si valid for development only. Generally
>>> python and django users are intelligent enough to properly read the
>>> docs and understand what's written on it.
>>> 
>> It's not a matter of being "intelligent" or not.  Developers are busy and 
>> can simply google things, see a particular line, and drop it in without 
>> fully understanding exactly what is going on.  (Simply read this group for a 
>> while if you don't believe this to be the case!)  People already turn off 
>> fsync, in production, after having read the PostgreSQL documentation, 
>> without actually realizing that they've put their database in danger.
>> 
>> Among other things, developers often have local data in their PostgreSQL 
>> instance that is valuable, and advising them to do a setting that runs the 
>> risk of them losing that data seems like a bad idea.
>> 
>> The Django documentation is not the place to go into the ramifications of 
>> fsync (or even synchronous_commit, although that's significantly less risky).
>> 
>> --
>> -- Christophe Pettus
>>
>> x...@thebuild.com
>> 
>> 
>> 
> 
> -- 
> "I was sad because I had no shoes, until I met a man who had no 
> feet. So I said, "Got any shoes you're not using?"  (Steven Wright)
> 
> Luke Plant || 
> http://lukeplant.me.uk/
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/55ABDF21.9060106%40cantab.net.
> For more options, visit https://groups.google.com/d/optout.

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/648BE2F3-E869-4E9D-BCB9-248E425D5A1C%40thebuild.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Luke Plant]

I agree with Federico on this - as long as we slap a big warning on it — 
"This is dangerous - it could make your database more likely to lose 
data or become corrupted, only use on a development machine where you 
can restore the entire contents of all databases in the cluster easily" 
— I don't see a problem in this being in our docs.


If people refuse to read a clear warning, they shouldn't be doing web 
development. They are just as likely to find similar instructions on the 
internet, but without warnings, and having it in our docs with the 
warning will be helpful.


Having a fast test suite is such an important part of development that 
it shouldn't be held back by  attempting to protect the world from 
people who cannot be helped.


Luke

On 16/07/15 16:49, Christophe Pettus wrote:

On Jul 16, 2015, at 1:16 AM, Federico Capoano  
wrote:


I also don't like the idea of believing django users are too stupid to
understand that this advice si valid for development only. Generally
python and django users are intelligent enough to properly read the
docs and understand what's written on it.

It's not a matter of being "intelligent" or not.  Developers are busy and can 
simply google things, see a particular line, and drop it in without fully understanding 
exactly what is going on.  (Simply read this group for a while if you don't believe this 
to be the case!)  People already turn off fsync, in production, after having read the 
PostgreSQL documentation, without actually realizing that they've put their database in 
danger.

Among other things, developers often have local data in their PostgreSQL 
instance that is valuable, and advising them to do a setting that runs the risk 
of them losing that data seems like a bad idea.

The Django documentation is not the place to go into the ramifications of fsync 
(or even synchronous_commit, although that's significantly less risky).

--
-- Christophe Pettus
x...@thebuild.com



--
"I was sad because I had no shoes, until I met a man who had no
feet. So I said, "Got any shoes you're not using?"  (Steven Wright)

Luke Plant || http://lukeplant.me.uk/

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55ABDF21.9060106%40cantab.net.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Christophe Pettus]

On Jul 16, 2015, at 1:16 AM, Federico Capoano  
wrote:

> I also don't like the idea of believing django users are too stupid to
> understand that this advice si valid for development only. Generally
> python and django users are intelligent enough to properly read the
> docs and understand what's written on it.

It's not a matter of being "intelligent" or not.  Developers are busy and can 
simply google things, see a particular line, and drop it in without fully 
understanding exactly what is going on.  (Simply read this group for a while if 
you don't believe this to be the case!)  People already turn off fsync, in 
production, after having read the PostgreSQL documentation, without actually 
realizing that they've put their database in danger.

Among other things, developers often have local data in their PostgreSQL 
instance that is valuable, and advising them to do a setting that runs the risk 
of them losing that data seems like a bad idea.

The Django documentation is not the place to go into the ramifications of fsync 
(or even synchronous_commit, although that's significantly less risky).

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/44C44B64-1FD1-48A3-8DC8-ADD5BCCCA27C%40thebuild.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Hey Christophe,

On Thu, Jul 16, 2015 at 8:34 AM, Christophe Pettus  wrote:
[CUT]
>
> By the way, I would strongly advise *against* *ever* even mentioning fsync = 
> off anywhere in the Django documentation; that is such a horribly bad idea in 
> 99.95% of real-life situations that steering people towards it as a "go 
> faster" button is very unwise.

We were not mentioning to advise to set non durable options for
production use, but for development, which I think it's more than
accetable, considering that I got a 70% boost, which together with the
django 1.8 --keepdb option and a bit of mocking, got my entire huge
test suite run in less than one minute, while running tests for
specific modules now (thank God) take seconds.
This is immensely important if you are doing TDD or refactoring often.

I also don't like the idea of believing django users are too stupid to
understand that this advice si valid for development only. Generally
python and django users are intelligent enough to properly read the
docs and understand what's written on it.

Federico

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6UrK5%3DnV-jX6nXK%2BGrhR-GMjbWhuCGYce_UvBBo2g8ZxA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Christophe Pettus]

On Jul 15, 2015, at 8:35 PM, Curtis Maloney  wrote:

> On 16 July 2015 at 05:01, Shai Berger  wrote:
>> This is a shot in the dark: Could it be that rolling back transactions
>> involving unlogged tables is harder? The idea does make sense, and running 
>> the
>> test suite does an extremely untypical amount of rollbacks.
> 
> I thought at some point I read that unlogged tables didn't support
> transactions... however, the docs don't agree.

Transactions behave the same in PostgreSQL for both logged and unlogged tables 
(except for, of course, the lack of a commit / rollback entry in the WAL), and 
there's no appreciable performance benefit on COMMIT and ROLLBACK time for 
logged vs unlogged.

My guess is that the Django tests are not generating enough data to make the 
WAL activity be a significant time sink.

By the way, I would strongly advise *against* *ever* even mentioning fsync = 
off anywhere in the Django documentation; that is such a horribly bad idea in 
99.95% of real-life situations that steering people towards it as a "go faster" 
button is very unwise.

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9B2A4A22-09FF-4634-AF4A-536C022FC57E%40thebuild.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Curtis Maloney]

On 16 July 2015 at 05:01, Shai Berger  wrote:
> This is a shot in the dark: Could it be that rolling back transactions
> involving unlogged tables is harder? The idea does make sense, and running the
> test suite does an extremely untypical amount of rollbacks.

I thought at some point I read that unlogged tables didn't support
transactions... however, the docs don't agree.

I was also considering going to the PG people and asking if the
UNLOGGED docs needed review... there could be all manner of reasons
why, on modern hardware, going via the WAL is as fast now...

--
C

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAG_XiSCcZ%3Df-919dSdOif0ZVMT6fma5yE3xy20%2Bw7E04y2E3kQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Shai Berger]

This is a shot in the dark: Could it be that rolling back transactions 
involving unlogged tables is harder? The idea does make sense, and running the 
test suite does an extremely untypical amount of rollbacks.

On Wednesday 15 July 2015 16:19:47 Federico Capoano wrote:
> That's quite baffling.
> 
> On Tuesday, July 14, 2015 at 7:03:03 PM UTC+2, Tim Graham wrote:
> > I see a thirty second increase in the test suite (from 7.5 minutes to 8
> > minutes) on my local machine with:
> > 
> > sql_create_table = "CREATE UNLOGGED TABLE %(table)s (%(definition)s)"

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

That's quite baffling.


On Tuesday, July 14, 2015 at 7:03:03 PM UTC+2, Tim Graham wrote:
>
> I see a thirty second increase in the test suite (from 7.5 minutes to 8 
> minutes) on my local machine with:
>
> sql_create_table = "CREATE UNLOGGED TABLE %(table)s (%(definition)s)"
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/239f0228-5cec-4957-8d3a-146fed22377b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Tim Graham]

I see a thirty second increase in the test suite (from 7.5 minutes to 8 
minutes) on my local machine with:

sql_create_table = "CREATE UNLOGGED TABLE %(table)s (%(definition)s)"

On Tuesday, July 14, 2015 at 4:20:45 AM UTC-4, Federico Capoano wrote:
>
> That's also a viable alternative, although it also mention unlogged 
> tables, which are not supported by django.
>
> We could list the quick hint with config sample, a link to pg non durable 
> options page and a warning that UNLOGGED tables are not supported yet. What 
> do you think about this?
>
> BTW has anyone had the time to try unlogged tables with their test suite 
> to see if it gave them any improvements? I am quite surprised i didn't even 
> get a 5-10% improvement out of the promised 10-17%.
>
> Federico
> Wouldn't it be enough to just have link to the correct PostgreSQL site: 
> http://www.postgresql.org/docs/current/static/non-durability.html
> And then some text with "if you want to run your tests on postgresql 
> please see the docs around non durable postgres"
>
> 2015-07-13 18:46 GMT+02:00 Federico Capoano  >:
>
>> Sure, i've just done something similar for this project:
>>
>> http://nodeshot.readthedocs.org/en/latest/topics/install_development.html#how-to-setup-the-test-environment
>>
>> The *keepdb* option allows to avoid recreating the test database at each 
>> run, hence saving precious time.
>>
>> If you want to speed up tests even more, tweak your local postgresql 
>> configuration by setting these values:
>>
>> # /etc/postgresql/9.1/main/postgresql.conf
>> # only for development!
>> fsync = off
>> synchronous_commit = off
>> full_page_writes = off
>>
>>
>> We just have to make sure people actually find this information via 
>> search engines.
>>
>> The best thing would be to have a subsection called "How to speed up 
>> tests with Postgres" or something similar.
>>
>> What do you think?
>>
>> Federico
>>
>>
>>
>> On Monday, July 13, 2015 at 5:54:08 PM UTC+2, Aymeric Augustin wrote:
>>>
>>> Hello Federico,
>>>
>>> 2015-07-13 16:26 GMT+02:00 Federico Capoano :
>>>
 While these are the winners:

- fsync = off
- synchronous_commit = off
- full_page_writes = off

 Would you like to submit a patch to docs/ref/databases.txt, in the 
>>> "PostgreSQL
>>> notes" section, explaining this configuration?
>>>
>>> I'm sure it would help many people run their tests faster on PostgreSQL. 
>>> It would
>>> also counter the trend to run tests on SQLite "because it's faster" — it 
>>> isn't.
>>>
>>> -- 
>>> Aymeric.
>>>  
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  -- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/django-developers/IkRgMxTTzPQ/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to 
> django-develop...@googlegroups.com .
> To post to this group, send email to django-d...@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAPDLAU7bYbeYb7pRgmqTR--zUrdU0fJyB5mHPWHoGEgawbB-MA%40mail.gmail.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/50e5a810-66d5-434a-b202-c19a2379f985%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

That's also a viable alternative, although it also mention unlogged tables,
which are not supported by django.

We could list the quick hint with config sample, a link to pg non durable
options page and a warning that UNLOGGED tables are not supported yet. What
do you think about this?

BTW has anyone had the time to try unlogged tables with their test suite to
see if it gave them any improvements? I am quite surprised i didn't even
get a 5-10% improvement out of the promised 10-17%.

Federico
Wouldn't it be enough to just have link to the correct PostgreSQL site:
http://www.postgresql.org/docs/current/static/non-durability.html
And then some text with "if you want to run your tests on postgresql please
see the docs around non durable postgres"

2015-07-13 18:46 GMT+02:00 Federico Capoano :

> Sure, i've just done something similar for this project:
>
> http://nodeshot.readthedocs.org/en/latest/topics/install_development.html#how-to-setup-the-test-environment
>
> The *keepdb* option allows to avoid recreating the test database at each
> run, hence saving precious time.
>
> If you want to speed up tests even more, tweak your local postgresql
> configuration by setting these values:
>
> # /etc/postgresql/9.1/main/postgresql.conf
> # only for development!
> fsync = off
> synchronous_commit = off
> full_page_writes = off
>
>
> We just have to make sure people actually find this information via search
> engines.
>
> The best thing would be to have a subsection called "How to speed up tests
> with Postgres" or something similar.
>
> What do you think?
>
> Federico
>
>
>
> On Monday, July 13, 2015 at 5:54:08 PM UTC+2, Aymeric Augustin wrote:
>>
>> Hello Federico,
>>
>> 2015-07-13 16:26 GMT+02:00 Federico Capoano :
>>
>>> While these are the winners:
>>>
>>>- fsync = off
>>>- synchronous_commit = off
>>>- full_page_writes = off
>>>
>>> Would you like to submit a patch to docs/ref/databases.txt, in the
>> "PostgreSQL
>> notes" section, explaining this configuration?
>>
>> I'm sure it would help many people run their tests faster on PostgreSQL.
>> It would
>> also counter the trend to run tests on SQLite "because it's faster" — it
>> isn't.
>>
>> --
>> Aymeric.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

 --
You received this message because you are subscribed to a topic in the
Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/django-developers/IkRgMxTTzPQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAPDLAU7bYbeYb7pRgmqTR--zUrdU0fJyB5mHPWHoGEgawbB-MA%40mail.gmail.com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAERYH6XLR6idaHzYOuuJ65iR%3DqXw8FsTvCyj-8OfjSeMAmS5qA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Schmitt, Christian]

Wouldn't it be enough to just have link to the correct PostgreSQL site:
http://www.postgresql.org/docs/current/static/non-durability.html
And then some text with "if you want to run your tests on postgresql please
see the docs around non durable postgres"

2015-07-13 18:46 GMT+02:00 Federico Capoano :

> Sure, i've just done something similar for this project:
>
> http://nodeshot.readthedocs.org/en/latest/topics/install_development.html#how-to-setup-the-test-environment
>
> The *keepdb* option allows to avoid recreating the test database at each
> run, hence saving precious time.
>
> If you want to speed up tests even more, tweak your local postgresql
> configuration by setting these values:
>
> # /etc/postgresql/9.1/main/postgresql.conf
> # only for development!
> fsync = off
> synchronous_commit = off
> full_page_writes = off
>
>
> We just have to make sure people actually find this information via search
> engines.
>
> The best thing would be to have a subsection called "How to speed up tests
> with Postgres" or something similar.
>
> What do you think?
>
> Federico
>
>
>
> On Monday, July 13, 2015 at 5:54:08 PM UTC+2, Aymeric Augustin wrote:
>>
>> Hello Federico,
>>
>> 2015-07-13 16:26 GMT+02:00 Federico Capoano :
>>
>>> While these are the winners:
>>>
>>>- fsync = off
>>>- synchronous_commit = off
>>>- full_page_writes = off
>>>
>>> Would you like to submit a patch to docs/ref/databases.txt, in the
>> "PostgreSQL
>> notes" section, explaining this configuration?
>>
>> I'm sure it would help many people run their tests faster on PostgreSQL.
>> It would
>> also counter the trend to run tests on SQLite "because it's faster" — it
>> isn't.
>>
>> --
>> Aymeric.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU7bYbeYb7pRgmqTR--zUrdU0fJyB5mHPWHoGEgawbB-MA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Federico Capoano]

Sure, i've just done something similar for this project:
http://nodeshot.readthedocs.org/en/latest/topics/install_development.html#how-to-setup-the-test-environment

The *keepdb* option allows to avoid recreating the test database at each 
run, hence saving precious time.

If you want to speed up tests even more, tweak your local postgresql 
configuration by setting these values:

# /etc/postgresql/9.1/main/postgresql.conf
# only for development!
fsync = off
synchronous_commit = off
full_page_writes = off


We just have to make sure people actually find this information via search 
engines.

The best thing would be to have a subsection called "How to speed up tests 
with Postgres" or something similar.

What do you think?

Federico



On Monday, July 13, 2015 at 5:54:08 PM UTC+2, Aymeric Augustin wrote:
>
> Hello Federico,
>
> 2015-07-13 16:26 GMT+02:00 Federico Capoano  >:
>
>> While these are the winners:
>>
>>- fsync = off
>>- synchronous_commit = off
>>- full_page_writes = off
>>
>> Would you like to submit a patch to docs/ref/databases.txt, in the 
> "PostgreSQL
> notes" section, explaining this configuration?
>
> I'm sure it would help many people run their tests faster on PostgreSQL. 
> It would
> also counter the trend to run tests on SQLite "because it's faster" — it 
> isn't.
>
> -- 
> Aymeric.
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Aymeric Augustin]

Hello Federico,

2015-07-13 16:26 GMT+02:00 Federico Capoano :

> While these are the winners:
>
>- fsync = off
>- synchronous_commit = off
>- full_page_writes = off
>
> Would you like to submit a patch to docs/ref/databases.txt, in the
"PostgreSQL
notes" section, explaining this configuration?

I'm sure it would help many people run their tests faster on PostgreSQL. It
would
also counter the trend to run tests on SQLite "because it's faster" — it
isn't.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mXFzO6g40%2B3wuoGsM3iaPOQidNAK%3DmiTvHUvu-mTFUtAg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.