Re: created_at / modified_at on all database tables

[Schmitt, Christian]

I also think that this won't need to be in core since a Model Subclass
works. We use this in our Application since we can't hard delete stuff so
we overwritten the Model Field and the manager that all tables contain a
date_deleted field which will be set when calling delete() objects.delete().
And the Code was really simple, just create a class that extends
models.Model and specifiy it abstract = True and then you can inherit from
it in every of your models.

2015-09-16 13:13 GMT+02:00 Florian Apolloner :

> Settings.py is definitely not the way to go. As for Meta options: Feel
> free, just do it via your own Model subclass, there is no need for this to
> be in core imo.
>
> On Wednesday, September 16, 2015 at 7:44:17 AM UTC+2, Aron Podrigal wrote:
>>
>> Hi,
>>
>> I'd like to propose a created_at / modified_at auto field to models so
>> that one can just turn this functionality on by a setting in settings.py or
>> within the models Meta settings.
>> I want these fields on both of the OneToOne related models when using
>> concrete inheritance, thus using a TimeStampedBaseModel abstract model
>> would cause these fields to clash. The reason I need created_at /
>> updated_at on every table because Django is not the only app using the
>> database I have other services sharing the same database. I assume having
>> created_at updated_at fields on tables to be very common. I can imagine
>> django creating those fields to all tables implicitly while handling those
>> field collisions properly (eg.  When accessing model.created_at on a
>> derived class it will return the value of that model.
>> model.base_model.created_at would give access to its base model).
>>
>> What are your thoughts?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/5693dbd6-e789-4025-8a92-5b491269bf9e%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU4Gu2XkL0U3yt%2BHARfXo7Tc-HfuXxt-4a9d1kP3q%2Bu8TQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support for UNLOGGED tables in PostgreSQL

[Schmitt, Christian]

Wouldn't it be enough to just have link to the correct PostgreSQL site:
http://www.postgresql.org/docs/current/static/non-durability.html
And then some text with "if you want to run your tests on postgresql please
see the docs around non durable postgres"

2015-07-13 18:46 GMT+02:00 Federico Capoano :

> Sure, i've just done something similar for this project:
>
> http://nodeshot.readthedocs.org/en/latest/topics/install_development.html#how-to-setup-the-test-environment
>
> The *keepdb* option allows to avoid recreating the test database at each
> run, hence saving precious time.
>
> If you want to speed up tests even more, tweak your local postgresql
> configuration by setting these values:
>
> # /etc/postgresql/9.1/main/postgresql.conf
> # only for development!
> fsync = off
> synchronous_commit = off
> full_page_writes = off
>
>
> We just have to make sure people actually find this information via search
> engines.
>
> The best thing would be to have a subsection called "How to speed up tests
> with Postgres" or something similar.
>
> What do you think?
>
> Federico
>
>
>
> On Monday, July 13, 2015 at 5:54:08 PM UTC+2, Aymeric Augustin wrote:
>>
>> Hello Federico,
>>
>> 2015-07-13 16:26 GMT+02:00 Federico Capoano :
>>
>>> While these are the winners:
>>>
>>>- fsync = off
>>>- synchronous_commit = off
>>>- full_page_writes = off
>>>
>>> Would you like to submit a patch to docs/ref/databases.txt, in the
>> "PostgreSQL
>> notes" section, explaining this configuration?
>>
>> I'm sure it would help many people run their tests faster on PostgreSQL.
>> It would
>> also counter the trend to run tests on SQLite "because it's faster" — it
>> isn't.
>>
>> --
>> Aymeric.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/7c3c1b6c-68f5-4cf0-b2db-d5fa58a6930d%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU7bYbeYb7pRgmqTR--zUrdU0fJyB5mHPWHoGEgawbB-MA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: Django website redesign launched

[Schmitt, Christian]

Somehow I hate it. The website is the worst website I've seen since a long
time.
The contrast is really aweful.
The issue Tracker got unusable due to the colors that aren't focused on
readability.

Overall it looks like a mess, just to have a new design.

Doesn't look like a designer or a graphic guy had his hands on that.
I can't realized why some should replace a good and usable design which had
some practical usage and also a good readability with the "thing" we have
now.

Sorry but I will stick to the old docs as for now or use the PDF ones with
my custom style (since your colors are aweful, too).

Django should focus on a clear design which is helpful for readability and
not this stupid mess.



2014-12-16 22:48 GMT+01:00 Brad Rice :
>
> I noticed today when I went in to look something up. I like it.
>
> On Tuesday, December 16, 2014 11:12:54 AM UTC-5, Jannis Leidel wrote:
>>
>> Hi everyone,
>>
>> We're incredibly proud to share with you the new design of the Django
>> website, the documentation and the issue tracker.
>>
>> This is a long time coming and we couldn't be happier to finally ship it
>> :)
>>
>> As you can imagine, there will be bugs, so please bear with us and report
>> issues to the issue tracker at https://github.com/django/
>> djangoproject.com/issues
>>
>> More infos about the redesign and its history can be found in the blog
>> post:
>>
>>   https://www.djangoproject.com/weblog/2014/dec/15/announcing-
>> django-website-redesign/
>>
>> Happy coding, everyone!
>>
>> Jannis
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-us...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/4a4c04ca-8e57-4351-8f1b-be30e920d64a%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU6%3DHgWC_PTfRaGbQL6Tmj6tqPzcn5wdG46rkF4mBSpX0g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Configurable safety options for high performance Django systems

[Schmitt, Christian]

>
> Nope. a large OFFSET of N will read through N rows, regardless index
> coverage. see http://www.postgresql.org/docs/9.1/static/queries-limit.html
>

That's simple not true.
If you define a Order By with a well indexes query, the database will only
do a bitmap scan.
This wiki isn't well explained. Take this:

>
https://wiki.postgresql.org/images/3/35/Pagination_Done_the_PostgreSQL_Way.pdf

The problem relies on the django paginator, that builds OFFSET queries that
> are unefficient by design. There are alternatives such as ordering by
> primary key and doing SELECT * from auth_user WHERE id > latest_id_returned
> . that "latest_id_returned" has to be passed back and forth to the client
> to keep the pagination hapening. Too bad you can't do "jump to page 15"
> like the django admin does.
>

Paginator needs to use OFFSET, which is slower than your seek method, but
SEEK has certain limitations. As you already said. Index and Order By to
the Rescue, which makes OFFSET as fast as possible. WITH INDEX, ORDER BY
and maybe some READ Slaves or even if you Shard your data the result is as
good as the SEEK method.


read the relevant theread I posted before. There are huge memory allocation
> issues even if you just iterate the first two items of a query that
> potentially returns a million rows: all the rows will be fetched and stored
> in memory by  psycopg and django ORM
>

Thats only happening since django will built something like "SELECT "bla",
"bla", "bla" FROM table LIMIT 10 OFFSET 100;
There is no ORDER BY clause, which is of course really slow, even with only
1000 entries inside a table.


...except when the cache is bigger than your entire VM.
>
> In this case if your dataset is , say, 10 TB, you will need roughly 10 TB
> of memory and 10 TB of data transferred between your django app and your
> postgres server. It is not a matter of optimization: is mere survival.
>

Okai  just as I said, If you deal with that amount of data u certainly will
use READ Slaves AND / OR partition your data.


> Let's first understand what's needed, than we can decide if it has a story
> inside the django core
>


The only thing that could be improved is that we should force a ORDER BY id
clause here and there. The index on the automatic id is already there.

I don't know why we should enforce something just because somebody has a
huge amount of data and can't deal with it. That wouldn't follow the clean
design pattern. Most people rely on the "Page to" part of the Paginator, if
we patch django to use a seek method as a default it wouldn't be that good
for a lot of developers.
The most people that have a performance problem could easily "fix" that by
themself or throwing more hardware at it.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU5HNtE%2BDfSxR7FcgdtAN6U3bSG7AVtGYhT7LLZUdY0Fjw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Configurable safety options for high performance Django systems

[Schmitt, Christian]

A sequence scan will only be made, if you query non indexed values.
So if you add a simple ORDER BY you will make a index scan, which is very
fast.
The problem relies more on the database than on the ORM.
As already said. If you need to deal with that much queries you need to log
your SQL statements and need to optimize them. Nothing to do with django at
all.
We ran into several performance issues of our applications as well even at
something like 10.000 or 100.000 entries per table and it was quite obvious
that the problem relied on the database/queries.

Also we learned that using the cache gives you a big performance win.
"The best query you could have, is the query you never need to make"
I don't think its good to make django more safely when dealing with high
performance systems, since it will make the system more complex and django
is already really complex inside the core orm.



2014-11-19 13:59 GMT+01:00 Marco Paolini :

> also, the offset + limit pagination strategy of django paginator is
> sub-optimal as it has N complexity: doing SELECT * FROM auth_user LIMIT 100
> offset 100 causes a 10-long table scan
>
> 2014-11-19 13:56 GMT+01:00 Marco Paolini :
>
>>
>>
>> 2014-11-19 13:50 GMT+01:00 Rick van Hattem :
>>
>>> Definitely agree on this, silently altering a query's limit is probably
>>> not the way to go. Raising an exception in case of no limit and lots of
>>> results could be useful.
>>>
>>> For the sake of keeping the discussion useful:
>>> - Let's say you have a table with 50,000 items, not an insanely large
>>> amount imho.
>>> - Now someone does a non-restricted loop through "Model.objects.all()"
>>> which would return 50,000 items.
>>>
>> even a restricted loop causes troubles (at least in postgres).
>>
>> if you do:
>>
>> for i, m in enumerate(Model.objects.all()):
>> if i > 4:
>> break
>>
>> then psycopg will fetch (and cache locally) the entire cursor resultset.
>> Of course django orm will only fetch the first batch (100 items I think)
>> from the psycppg cursor, bu that does not solve the underlying issue.
>>
>> There are options to have server-side cursors in psycopg but django does
>> not use them AFAIK
>>
>> see this thread
>> https://groups.google.com/forum/#!searchin/django-developers/psycopg$20results/django-developers/8ZBA4uMinXo/M1osYTGmvbUJ
>>
>>
>>
>>
>> - In what case would this be desirable as opposed to limiting to 10,000
>>> items and raising an error when the database actually returns 10,000 items.
>>>
>>> Naturally this case would only apply if no slice is given, but if you're
>>> really processing over 10,000 items in a single loop you probably know how
>>> to slice the queryset when needed.
>>> Perhaps something like this: raise QuerysetTooLarge('Your query returned
>>> over {LIMIT} results, if this is intentional please slice the queryset')
>>>
>>> Not saying this should be a default straight away but having a default
>>> of 10,000 or even 100,000 should not hurt anyone and protects against
>>> killing a server which is always a positive result in my book.
>>>
>>> On Wednesday, November 19, 2014 1:42:59 AM UTC+1, Josh Smeaton wrote:

 To me, "sane default" means django should not silently alter the query
 to provide a LIMIT when it is not asked for.

 I have also run into situations where doing a .count() or iterating a
 full table has broken the application, or put too much pressure on the
 database. Specifically with django bindings to javascript datatables. But I
 still wouldn't want artificial limiting on such queries.

 What *may* be useful, is to be able to apply queryset methods onto an
 already sliced queryset. That would allow users to implement
 queryset/manager methods that provide pre-sliced querysets to the rest of
 the code. The problem would be, what should happen in this case?

 Model.objects.all()[0:10].filter(field=True)

 Should the filter be logically/internally moved to before the limit? Or
 should the filter be applied to the result of the limit in an outer query?
 Traditionally, django applies mutations in succession, but this wouldn't be
 very useful the for the majority of operations that would occur "after" the
 slicing. We *could* say all slicing is "saved" and applied at the end, but
 we'd definitely run into issues with users reporting that filtering isn't
 working as they expect - after the slice.

>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this 

Re: Configurable safety options for high performance Django systems

[Schmitt, Christian]

Sorry, but I don't think this is suitable.
If somebody has 100M rows per Table then, he should prolly think about
sharding/replication anyway. So the ORM would still suffer anyway.
Currently my company has a few tables with a high count as well but since
we never used the django-admin and managed to shard some of the data, we
never ran into such problems.

Django is a headstart for many projects / products, but once you have
certain amount of data you still need to program a few things by yourself.

If your patches are enough than live with it, but I don't see a reason for
optimizing django against big tables.



2014-11-18 19:27 GMT+01:00 Rick van Hattem :

> That certainly solves one part of the problem. After that I would still
> opt for an optional configurable default for slicing. Personally I prefer
> to raise an error when unsliced querysets are used since it's almost always
> harmful or at least dangerous behaviour.
>
> On 18 November 2014 19:18, Claude Paroz  wrote:
>
>> On Tuesday, November 18, 2014 1:58:00 PM UTC+1, Rick van Hattem wrote:
>>>
>>> Hi guys,
>>>
>>> As it is right now Django has the tendency to kill either your browser
>>> (if you're lucky) or the entire application server when confronted with a
>>> large database. For example, the admin always does counts for pagination
>>> and a count over a table with many rows (say, in the order of 100M) can
>>> really stall your app and database servers.
>>>
>>
>>
>>
>> https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.show_full_result_count
>> seems to go in your direction, at least. count(*) performance is
>> implementation-specific, so once again, no one-size-fits-all.
>> Maybe the documentation could be completed, here and there.
>>
>> Claude
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/django-developers/aYwPykvLaMU/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/5c3ffd8f-7ac6-4ac9-863d-875581c3c127%40googlegroups.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJAwA%3DzwbCdXFun_%3DR17EfdDEcS35GOpPi%2BsdcNPyZZFwRLs5g%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU4C4u0x-B2toa3gAr32sARTu8zMrPvdTPAnsubZaMmqPg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Requiring GitHub login for actions on Trac

[Schmitt, Christian]

I'm a little bit concerned about that.
First I'm using a different user on Trac than on Github, so everything I
wrote so far will getting lost (not that bad problem for me), but I think
there are many users who are in the same situation.

The next thing is vendor lock-in. What will happen if Github don't have
enough money? Then all usernames would need to migrate back or to another
OAuth provider, then everything could be lost a second time.
Or that Github gets bad / mad.

Currently we already live in a world were everything gets connected. And
that is really awful. One must consider that Github is definitely a target
for intelligence agencies. And I don't mean the NSA only.
Maybe I'm a little bit too paranoid but at the current state of the
internet we shouldn't try to connect everything, just it is easier to login.




2014-08-07 8:46 GMT+02:00 Aymeric Augustin <
aymeric.augus...@polytechnique.org>:

> To be clear, I have a working implementation of GitHub OAuth that I can
> activate as soon as we reach a consensus.
>
>
>
> On 7 août 2014, at 02:43, Ben Finney  wrote:
>
> > −1. I am happy to agree to Django's BTS terms of use, not GitHub's.
> > Please don't make the former depend on the latter.
>
> I didn’t know our Trac installation had terms of use. So, are you
> volunteering to jump in and delete spam as it comes in? Or do you
> have an alternative proposal?
>
>
>
> On 7 août 2014, at 02:47, Shai Berger  wrote:
>
> > Today, it is possible to contribute to the Django project without a
> > Github account. I would like this to remain the case.
>
> This is possible but in a limited capacity. To be honest, I think that
> ship sailed when we moved to GitHub. We would have also moved
> issues there if GitHub’s tools were usable.
>
>
>
> On 7 août 2014, at 02:58, Andre Terra  wrote:
>
> > Most importantly, how would Django as a project benefit from this
> > choice other than reducing minimal spam?
>
> Did you just ask “how would Django as a project benefit from having
> core devs work on committing patches rather than fighting spam”?
>
> If you don’t already have a djangoproject.com account, you’re likely to
> give up on reporting a small bug just because it’s too complicated to
> log in. Considering our target demographic, GitHub OAuth would
> eliminate this problem.
>
> Also, if you’re trying to report a bug anonymously, you’re likely to be
> unable to pass the CAPTCHA, and also be unable to report it, because
> you’re still getting blocked by the CAPTCHA. See complaints:
> https://code.djangoproject.com/search?q=captcha=1=on
>
> Finally, to be honest, I’d rather adjust Django’s tools to enthusiastic
> beginners than grumpy freedom extremists who refuse to use GitHub.
>
> > A better solution would be to strengthen what it means to have an
> identity
> > on djangoproject.com. Rather than restricting user actions to Trac, we
> > could motivate users to create something like a Django profile which
> would
> > be used for Trac (among may other uses)
>
> We already have that: https://www.djangoproject.com/~aaugustin/
>
> > and could later be linked to any OAuth providers, including but not
> limited
> > to GitHub.
>
> We don’t have that.
>
> > TL;DR Identity on djangoproject.com, Authentication linked to multiple
> OAuth,
> > Authorization in Trac.
>
> Are you volunteering to do this work, and if so, when will it be done?
>
> > I hope that idea makes sense. I may be just babbling nonsense.
>
>
> I’m sorry, but ideas don’t matter nearly as much as execution here.
> We just need working tools — nothing fancy.
>
>
>
> On 7 août 2014, at 02:59, Josh Smeaton  wrote:
>
> > is it easy enough to support github oauth + the current trac auth
> concurrently?
> > If a user chooses to go through the harder path, that's fine.
>
> It may be doable to provide two authentications endpoints, like /login and
> /login/github. Trac just looks at REMOTE_USER and creates a session that
> lasts until you logout. I’ll look into it.
>
> That solves the “GitHub is evil, I don’t want to touch their bytes with a
> six
> foot pole” problem, but only half of the username mismatch problem. You
> can keep using your djangoproject.com username is you wish, but if
> someone else owns the same username on GitHub, they can impersonate
> you e.g. https://github.com/shai / https://www.djangoproject.com/~shai/.
>
> That said, if you aren’t logged in, you can type anything you want in
> Trac's
> “Your username or email” field. It provides identification, not
> authentication.
> This has never been a problem in the past. So I don’t think we’ll run into
> too much trouble with usernames in general.
>
> The only part where Trac usernames are used for authentication is access
> control, which only applies to people who have special permissions.
>
> --
> Aymeric.
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django 

Re: Updating the organization of the Django Project

[Schmitt, Christian]

> Christian, have you seen the patch review checklist I put together?

Yep and I will definitely do some more work in the future.

Everything i write now is my personal opinion.
first of all trac has some of the weakest Query Filters. (thats my personal
opinion)
the next thing is that changing the ticket and adding comments share the
same button, which leads to strange things (sometimes).
also its really hard to review for specific versions.
voting for a new feature is done on the mailing list which is okai, but i
think that would be better to do in an issue tracker. (voting through
tooling seems to be way simpler than posting a +1.)
also there is no way of saving filters to my profile. I mean i could
bookmark them but my bookmarks are way too much already.
also it would be "a cool" feature if trac would automatically add pr's as
an info to the ticket..
I mean most of these things are comfort features and still trac feels less
intuitive than other tools since they are a complete platform or link
everything together.

However none of these points would stop me or another individual from
helping / commiting / reviewing. its just a matter of time.
tools should help us to use less time.
tools should link/work together since its easier to start if you have
everything together. also i talked about gerrit, if gerrit would be
introduced it woud be even harder to keep these things together since its
another platform, too.

what i want to do in short is that sometimes we also should look at the
infrastructure and improve on these things. i mean i'm just somebody but i
think other people would say something about the currents, too.
with infrastructure i mean the things like documentation, issue tracker,
code review, etc.. and i think there are plenty improvements.
like the online documentation.



2014-08-01 23:18 GMT+02:00 Tim Graham <timogra...@gmail.com>:

> Christian, have you seen the patch review checklist I put together?
>
>
> https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/submitting-patches/#patch-review-checklist
>
> That is basically what I do when I review a patch. If a patch meets those
> guidelines, please mark it RFC. A core dev will always do a final review,
> but it helps a lot if we don't have to explain those basics to new
> contributors. Sure, you will make mistakes, but that is the only way to
> learn and the risks are low. Over time, the core team will see that the
> patches your mark as RFC are really RFC as you said, and we're likely to
> ask you the join the team so you can just commit those patches that you
> review yourself.
>
> Regarding Trac, would you mind explaining what Trac is blocking you from
> doing? Perhaps we can try to address some of those issues.
>
>
> On Friday, August 1, 2014 4:38:54 PM UTC-4, Christian Schmitt wrote:
>
>> its a matter of trust.
>> gerrit tracks your reviews so its way easier to build up trust that the
>> things you mark as RFC is really RFC ;)
>> And yes its a matter of tooling since some tools provide a better way to
>> build trust for such community processes.
>> Also I'm quite young. I wouldn't want to make a "community review" just
>> by myself since I will definitely do some mistakes over time as everybody
>> will do.
>>
>> to come back to the tooling. gerrit is just one tool. i don't know more
>> than gerrit that has such a large feature set. but i think there are
>> definitly some out.
>> In our company we use a lot of atlassian tools and atlassian has a
>> "gerrit" tool, too but I didn't take a look at it yet. so thats another
>> option since atlassian tools are free for community use.
>>
>> the next thing is, reading contribution docks is easy, using trac is not.
>> trac is a really blocker for a lot of things on my side. i think the ux is
>> somewhat bad, but thats just an own perspective.
>>
>>
>>
>>
>> 2014-08-01 21:49 GMT+02:00 Carl Meyer <ca...@oddbird.net>:
>>
>>> Hi Christian,
>>>
>>> On 08/01/2014 01:37 PM, Schmitt, Christian wrote:
>>> [snip]
>>> > i mean maybe we should really look at a review tool and maybe say if
>>> two
>>> > people of the community reviewed it gets pipelined to the "core" which
>>> > can do a final review. so it would be more community oriented and fewer
>>> > "bad patches" will get to the commit pipeline.
>>> > still it takes a lot of time to teach people and setup everything. but
>>> > it will put the project further ahead.
>>> > (also I'm not a core member but if we had such a review system i
>>> > would definitely do more of these reviews since thats the thing i do at
>>>

Re: Updating the organization of the Django Project

[Schmitt, Christian]

its a matter of trust.
gerrit tracks your reviews so its way easier to build up trust that the
things you mark as RFC is really RFC ;)
And yes its a matter of tooling since some tools provide a better way to
build trust for such community processes.
Also I'm quite young. I wouldn't want to make a "community review" just by
myself since I will definitely do some mistakes over time as everybody will
do.

to come back to the tooling. gerrit is just one tool. i don't know more
than gerrit that has such a large feature set. but i think there are
definitly some out.
In our company we use a lot of atlassian tools and atlassian has a "gerrit"
tool, too but I didn't take a look at it yet. so thats another option since
atlassian tools are free for community use.

the next thing is, reading contribution docks is easy, using trac is not.
trac is a really blocker for a lot of things on my side. i think the ux is
somewhat bad, but thats just an own perspective.




2014-08-01 21:49 GMT+02:00 Carl Meyer <c...@oddbird.net>:

> Hi Christian,
>
> On 08/01/2014 01:37 PM, Schmitt, Christian wrote:
> [snip]
> > i mean maybe we should really look at a review tool and maybe say if two
> > people of the community reviewed it gets pipelined to the "core" which
> > can do a final review. so it would be more community oriented and fewer
> > "bad patches" will get to the commit pipeline.
> > still it takes a lot of time to teach people and setup everything. but
> > it will put the project further ahead.
> > (also I'm not a core member but if we had such a review system i
> > would definitely do more of these reviews since thats the thing i do at
> > my company, too)
>
> I'm curious why the system you propose (requiring two community reviews
> to fast-track a patch) would motivate you to do more reviews than the
> current system, in which it only takes a single review (by any community
> member) to mark a ticket as "Ready for Check-in", which already gets it
> fast-tracked for commit.
>
> Although it does happen sometimes, there isn't a big problem (AFAIK)
> with lots of patches getting marked RFC when they aren't ready;
> currently there are only 3 tickets in RFC state. The bigger problem is
> there aren't many people doing reviews and moving tickets to RFC state.
> According to dashboard.djangoproject.com, there are 52 patches awaiting
> review.
>
> Maybe it's mostly a tooling issue, in that the current process requires
> reviewers to read the contribution docs, understand the ticket state
> flow, and then use both GitHub and Trac and manually mark the ticket RFC
> themselves after doing a review, rather than having a review tool with a
> big +1 button on it that automatically handles the state changes.
>
> Carl
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/53DBEF42.809%40oddbird.net
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU4cnH3nJxBV5pTA5yW%2B_N%3D2Bwju2ZWyevgJRd3Fu%2B-7-w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Loading fixtures once for each TestCase to improve running time

[Schmitt, Christian]

Just a quick question regarding to these posts / tickets.

In the last few weeks we fixed: https://code.djangoproject.com/ticket/22487

So I think that we now don't need to rely on fixtures and should just tell
the people to use that behavior in the first place.
I mean I'm not a Django core developer, but since we fix our code for
Django 1.7 our test suite runs faster since we dropped them out.

Also we wrote somethings in the docs about that:
Deprecated since version 1.7: If an application uses migrations, there is
no automatic loading of fixtures. Since migrations will be required for
applications in Django 2.0, this behavior is considered deprecated. If you
want to load initial data for an app, consider doing it in a data migration
.

The question about that is, what happens if you don't use migrations. Will
fixtures still be loaded in Django 2.0 and upwards or not.
If they will be loaded, we should fix this, but if not we should maybe
clear that things up that the people SHOULD migrate away from fixtures to
migrations.

Quick Note: I think, currently the data migrations still uses the old
technique, by serializing / deserialize on every test case, so maybe we
should expand that logic somehow instead of putting work onto fixtures.




2014-06-11 16:37 GMT+02:00 Josh Smeaton :

> manfre was nice enough to test this patch out on mssql for me. The subset
> of tests that use fixtures went from 385 seconds to 158 seconds (+2
> failures). The improvement seems to be stable across backends, but isn't
> very significant when considering the entire test suite.
>
> I'm going to abandon this patch and ticket myself - mainly due to the
> sqlite threading issues. If someone else would like to run with it, I'll be
> happy to share any information I have.
>
> - Josh
>
>
> On Wednesday, 11 June 2014 09:34:14 UTC+10, Josh Smeaton wrote:
>>
>> I used the patch in the above ticket as a base, and implemented fixture
>> loading in the setUpClass classmethod rather than the setUp method. I found
>> that it improved the total running time of the entire test suite by about
>> 10%, but it improved TestCases that use fixtures by a factor of 3.
>>
>> https://github.com/jarshwah/django/tree/testcase-optimization
>>
>> I'm unable to test this patch on Oracle or mssql though, which are known
>> to be a lot slower than most of the other backends (for the test suite).
>> The list of test modules that use fixtures are:
>>
>> admin_changelist admin_custom_urls admin_docs admin_inlines admin_views 
>> admin_widgets aggregation aggregation_regress contenttypes_tests fixtures 
>> fixtures_model_package generic_inline_admin known_related_objects 
>> m2m_through_regress multiple_database proxy_models raw_query servers 
>> syndication_tests test_client test_client_regress test_utils timezones
>>
>>
>> If someone is able to test Oracle or mssql with that set of test modules and 
>> report back the difference in time taken between master and the above 
>> branch, that'd be extremely useful information.
>>
>>
>> As for profiling the entire test suite:
>>
>>
>> vagrant@djangocore:~$ PYTHONPATH=/django 
>> /home/vagrant/.virtualenvs/py2.7/bin/python
>> -m cProfile -s cumulative /django/tests/runtests.py
>>
>> This shows that for psql, mysql (inno), and sqlite, the majority of
>> (cumulative) time is spent inside the request/response phases of the test
>> client, reversing urls, and rendering templates. I don't think the choice
>> of backend would massively influence these tests (unless transactions
>> themselves are especially slow per backend), so I'm betting that the
>> loaddata/rollback operations are the primary pain points. See my output
>> (with loaddata optimisations) here: https://gist.github.com/
>> jarshwah/42c4cd1f54c8fb3dd273
>>
>> Unfortunately, this patch has introduced some bad threading issues for
>> sqlite which seg faults for me and also Aymeric in certain conditions. It's
>> definitely not ok to use this code in production. If there isn't
>> significant improvement for oracle or mssql, I'll probably abandon this
>> patch myself.
>>
>> Cheers
>>
>> On Saturday, 7 June 2014 18:29:12 UTC+10, Aymeric Augustin wrote:
>>>
>>> In fact, I just reinvented https://code.djangoproject.com/ticket/20392.
>>>
>>> The patch on that ticket is pretty good, I’ll try to review it again.
>>>
>>> --
>>> Aymeric.
>>>
>>>
>>>
>>> On 7 juin 2014, at 10:12, Aymeric Augustin <
>>> aymeric@polytechnique.org> wrote:
>>>
>>> > Hi Josh,
>>> >
>>> > Fixtures don’t receive a lot of attention because they’re hard to
>>> maintain and generally inferior to object generators such as factory_boy.
>>> Still, it would be good to optimize them.
>>> >
>>> > On 7 juin 2014, at 09:34, Josh Smeaton  wrote:
>>> >
>>> >> I've been looking into the django testing infrastructure today to see
>>> if there was any particular reason why the test suite takes 

Re: [GSoC] Switching to Jinja2 proposal

[Schmitt, Christian]

I'm not a django-developer, but I'm creating a lot of applications with
Django and I would never want to switch to Jinja2.

Why?

The first thing is that Django Templates are simple to understand, they are
not formed as a new DSL or a Programmable way of a Template language.
Their Syntax is clear and simple and it's really easy to extend the tags
for new programmers aswell.

I only had a quick overview of Jinja2 but the Syntax aswell as other things
just looking more like a programming language for templates than a simple
templating engine.

Also the Template Language or the Django interface is most of the time not
the bottleneck of any application (there are some exceptions where the
template engine or some django views will be a bottleneck but most of the
time there is a easy way to solve it like using c)
Nearly 90% of the time you will have trouble with the database.

Changing the Template Language will maybe looking good on benchmark papers,
but i don't think that it will help scaling websites to need less servers.
(Django also has a really good and easy way to scale up really good, thanks
to the DatabaseRouters, Cache Engine, Session Engine, etc. you could Scale
out/up really, really well)

The next thing is that the internet is changing and template languages that
are on the server side getting less focus.
Mostly applications are more and more and API which will getting consumed
by a Framework like AngularJS or KnockoutJS.
So there is no need for a faster template language, since the Django
Template Language is fast enough for the Django Admin or other applications
where a Template Engine is still needed.

I hope that the most people understood my points and that my english isn't
too bad.

For me switching to jinja2 wouldn't make any sense.
Also we changed a lot of app loading stuff, so my applications needing to
be migrated. While I think that I have a lot of stuff that needs to be
refactored when I switch to Django 1.7 I don't want to have a lot of
backwards incompatible stuff in Django 1.8 or Django 1.9, too.





2014-02-11 14:07 GMT+01:00 Aymeric Augustin <
aymeric.augus...@polytechnique.org>:

> 2014-02-11 13:42 GMT+01:00 Christopher Medrela :
>
>
>
>> What did Armin said about Python 3 exactly?
>>
>
> He wrote an extensive argumentation about "why Python 2 [is] the better
> language for dealing with text and bytes" [1] as well as a number of tweets
> and a few other blog posts along the same lines.
>
> While his arguments are technically correct, I disagree with his
> conclusions
> because he's speaking with the point of view of an expert maintaining
> libraries at the boundary between unicode and bytes (like werkzeug).
> However,
> most Python users aren't experts and aren't maintaining such libraries. In
> my
> experience working with Python programmers ranging from intern to veteran,
> the
> unicode model of Python 3 is a strict improvement over Python 2 in terms of
> pitfalls hit in day-to-day programming. YMMV.
>
> [1] http://lucumr.pocoo.org/2014/1/5/unicode-in-2-and-3/
>
> --
> Aymeric.
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CANE-7mU31zMRkVrJAZY_tde-xdiU63p2s38pB2vVPpeDpPBkxQ%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU6BR8x%3D1UtKzzBypGiLR6SW4zjsWEX8HNK0VzZUNJjz%2BA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: 1.7 Release Schedule

[Schmitt, Christian]

I also hoped that the release date will be way before the 15th of may.

Also just a question.. I currently looking forward to maybe contribute
something back to django. But I always struggle with trac it looks so
extremly messy.

I mean i try to query some unresolved, accepted queries:
https://code.djangoproject.com/query?owner=~nobody=~+=new_patch=0=Accepted=id=summary=status=owner=type=component=priority
as you see, i only can see the queries who don't have an owner, but
there are also issues where the owner is "" instead of nobody, so its
really hard to find them.
maybe i'm just spoiled with jira...



2013/12/9 x13 :
>
>
> On Monday, December 9, 2013 5:53:11 PM UTC-3, Andrew Godwin wrote:
>>
>>
>>
>> This has been suggested before, including by some core developers. The
>> move to using Git has helped in this regard, but we can't do it that fast
>> for this release as I'm moving to the USA during this release cycle
>> (January, most likely) and so I don't have quite enough time to ship
>> migrations faster to a high level of quality - and since it's apparently
>> what everyone is waiting for, it would be a poor move to release with it
>> unpolished or buggy. Don't worry, hopefully we'll have a mostly stable beta
>> people can start developing newer projects on.
>>
>> Andrew
>
>
>
> Great! Thank you a lot.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/4ee23053-6f87-4182-b2b8-71d76ba8f679%40googlegroups.com.
>
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPDLAU7LtMpZRp%2BBQG9MAMHH0ph1r-_mTiOoz3s1pJHaKnSk6w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.