Re: Please don't kill the ecosystem

2014-10-02 Thread Luke Plant 
Hi Pkl,

Sorry for slow reply, but I thought this was worth it:

On 03/09/14 20:45, Pkl wrote:

> Concerning the "rules of open source", I've yet to find a satisfying way
> to apply them regarding these "micro breaks". Imagine that project
> "myvideoplugin" is unmaintained (not handling PR) : unable to patch the
> original Repo, I'd have to fork it ; and unable to push results to the
> original pypi name, I'd have to create a separate myvideoplugin-pkl
> package on Pypi... that's quite some hassle for (often) a 10-lines patch.

There is no need to create a new pypi package and release - you can
install straight from github like this:

pip install git+git://github.com/foo/bar.git@abcde12345

You can put this straight into your requirements.txt as well. I have
dozens of instances of this in my requirements.txt files.

The trick then is to get your changes merged upstream. That is not
always very easy - you have to be willing to genuinely contribute
upstream i.e. contributions that make the maintainer's life easier, not
harder. That means writing unit tests, or bootstrapping use of tox so
that the test suite can be easily run against multiple versions. It
might even mean starting a test suite because upstream didn't have one -
in which case you also should reassess the process by which you choose
dependencies...

However, I've generally found that with effort and *genuinely helpful*
upstream contributions, my PRs are accepted, and I can go back to using
a PyPI release after some time. When I can't, it's because the upstream
dependency wasn't solid, and it kind of serves me right for not being
more careful and adding something to my project that wasn't reliable. In
some cases I might have to take over being maintainer of the project -
which is a burden, but part of the price you pay for being a part of the
open source world and getting tons of amazing quality software for free.

I do agree that we should be more careful about finding and documenting
ways of doing deprecations so that libraries can support both old and
new versions without generating deprecation warnings. I've just added
one fix to 1.6 release notes on this front.

Regards,

Luke


-- 
"Cross country skiing is great if you live in a small country."
(Steven Wright)

Luke Plant || http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/542D3182.3050201%40cantab.net.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-10-02 Thread Luke Plant 
On 03/09/14 14:21, Anssi Kääriäinen wrote:

> For example, lets consider the get_query_set() -> get_queryset() naming
> change done in 1.6. If 3rd party library writers change the method name
> to get_queryset() for 1.6, then their code won't work in 1.5. If they
> don't do the change, then all users trying to run with -Wall will get a
> bunch of deprecation warnings they can't do much about. This will
> effectively ruin the "run with -Wall" advice.
> 
> The usual way to solve these kinds of problems in code that needs to
> work in multiple Django versions is to write code which is conditional
> on Django's version. It looks a bit ugly to define methods
> conditionally, but in practice this works extremely well.
> 
> So, when adding a new deprecation to Django, we should document how 3rd
> party library writers can write code that works *silently* both before
> and after the feature deprecation is added. If there is no easy way to
> achieve that, then we should reconsider the deprecation.

Regarding get_queryset, it's possible to do it without warnings I think,
and the RenameMethodsBase class helps, but it's not entirely
straightforward.

I've added a paragraph to the docs that documents how to do it correctly
for the get_queryset renamings, and I agree that we should make this
kind of thing a standard practice. It might bloat the release notes a
bit, but I think that in general people appreciate the detail.

Luke


-- 
"Cross country skiing is great if you live in a small country."
(Steven Wright)

Luke Plant || http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/542D305C.2010206%40cantab.net.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-03 Thread Collin Anderson 
Considering we're trying to come out with more frequent releases, I wonder
if it would sometimes make sense to sometimes have a 3-release deprecation
for some features.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFO84S4jfKyoQAfAXvRAqonXEH5apHpMDPULY5Qvv0rk-62qQQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-03 Thread Pkl 
Hello,

Hello,

My apologies for sounding rude, and for raising these issues so late.

Yes, upgrading a project itself is a matter of a pair of evenings, the 
problem is with third-party libraries, and more again, plugins of 
third-party libraries (especially CMS-related). 
I've had to bundle a dozen of these with my app, because they didn't quite 
follow the pace of Django evolutions. Now of course, as you say, I might as 
well stabilize for some time on a LTS version, but as was said too, the 
next steps would also be much harder then. Or I could forget about modular 
apps, and go towards monolitic third-party apps instead...

I felt that crucial variables like "request.REQUEST" or "raw_post_data" 
could stay aliased for much more than 2 minor versions (but undocumented 
and with warnings, of course), for teh sake of retrocompatibility.
However if it's acknowledged, in the core team, that code maintainability 
and security would be hindered by keeping these artefacts, I can't argue 
that. 

Concerning the "rules of open source", I've yet to find a satisfying way to 
apply them regarding these "micro breaks". Imagine that project 
"myvideoplugin" is unmaintained (not handling PR) : unable to patch the 
original Repo, I'd have to fork it ; and unable to push results to the 
original pypi name, I'd have to create a separate myvideoplugin-pkl package 
on Pypi... that's quite some hassle for (often) a 10-lines patch.

Thanks Curtis for the proposal, I was thinking about a django-retrocompat 
package indeed, however I realize that doing a decent job of 
retrocompatibility would be a much harder job than initially expected, 
since breaking changes are rarely a matter of simple aliases 
(https://docs.djangoproject.com/en/1.7/releases/1.7/#backwards-incompatible-changes-in-1-7)
 
; django-retrocompat wouldn't stand the expectations of users. But a 
minimal package dealing only with "renamed fields" would be doable, I'll 
have a look at it.

regards, 
Pkl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3317f347-d82b-4edd-90fd-16da41f5dc49%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-03 Thread Anssi Kääriäinen 
On Wed, 2014-09-03 at 14:46 +0200, Aymeric Augustin wrote:


> 1.5 and 1.6 must have been easier than 1.4. 1.7 may be more difficult
> because
> of migrations and app-loading. If anyone has improvements to suggest
> for the
> documentation, please do!

We could perhaps have better documentation of how to handle deprecations
in 3rd party libraries. Namely, how to remove usage of deprecated
features in a way that works silently both before and after the addition
of the deprecation.

For example, lets consider the get_query_set() -> get_queryset() naming
change done in 1.6. If 3rd party library writers change the method name
to get_queryset() for 1.6, then their code won't work in 1.5. If they
don't do the change, then all users trying to run with -Wall will get a
bunch of deprecation warnings they can't do much about. This will
effectively ruin the "run with -Wall" advice.

The usual way to solve these kinds of problems in code that needs to
work in multiple Django versions is to write code which is conditional
on Django's version. It looks a bit ugly to define methods
conditionally, but in practice this works extremely well.

So, when adding a new deprecation to Django, we should document how 3rd
party library writers can write code that works *silently* both before
and after the feature deprecation is added. If there is no easy way to
achieve that, then we should reconsider the deprecation.

 - Anssi


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1409750485.11410.611.camel%40TTY32.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-03 Thread Aymeric Augustin 
2014-09-03 10:40 GMT+02:00 Tom Evans :
>
> I think it was more distraction by topics he had not come across. We
> set him off by saying "look at the release notes, go thru each change
> in turn, see if we are affected and what we need to fix it".
>

Thanks Tom for the details. I understand better. Upgrading Django exposes
you
to components or features you didn't even know existed. The time I spent
triaging Trac tickets gives me a massive advantage there :-)

The problem then was that he was new. He hadn't had to deal too much
> with TZ support, so adding TZ support to our project meant a day of
> learning about how TZ work in UNIX.
>

You went for USE_TZ = True. That required a lost of work to adjust your
entire
codebase. I'm considering that a new feature for your project. If you wanted
to upgrade as fast as possible, you could have kept the legacy behavior with
USE_TZ = False. That was the default for projects upgraded from 1.3.

Similarly, changing the project layout is a good thing, but then he
> spent a day learning about package layouts - not a bad thing by any
> means, but in the 1.3 -> 1.4 release there are 75 similarly complex
> enhancements/deprecations/incompatibilities - it just takes some time
> to go through if you have not come across that topic before, eg,
> clickjacking.


Right, this was another complicated change you couldn't skip.

1.5 and 1.6 must have been easier than 1.4. 1.7 may be more difficult
because
of migrations and app-loading. If anyone has improvements to suggest for the
documentation, please do!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mU23ag9N4vnuajfFsOdtYKBp1RttE3WqG%2B36BUXo6yQ4g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-03 Thread Tom Evans 
On Tue, Sep 2, 2014 at 2:53 PM, Aymeric Augustin
 wrote:
> 2014-09-02 15:33 GMT+02:00 Tom Evans :
>
>> this story was scored
>> at 8 points, it took a junior developer much longer than 8 points and
>> wasn't finished in a single sprint - and 1.3->1.4 was *easy*
>
>
> I don't know how much a point or a sprint is worth in this context :-/

Sorry, context is important! "A sprint" ~ 9 work days, points vary
wildly from sprint to sprint unfortunately - 12 is about a sprints
worth of work.

> 

Good steps.

>
> Did your junior developer follow a similar process? If he did, what took
> them
> so long?
>
> What I want to understand is how much of an advantage I have when it comes
> to
> upgrading Django. What do I know that isn't written down and makes it so
> hard
> for others?

I think it was more distraction by topics he had not come across. We
set him off by saying "look at the release notes, go thru each change
in turn, see if we are affected and what we need to fix it".

The problem then was that he was new. He hadn't had to deal too much
with TZ support, so adding TZ support to our project meant a day of
learning about how TZ work in UNIX.

Similarly, changing the project layout is a good thing, but then he
spent a day learning about package layouts - not a bad thing by any
means, but in the 1.3 -> 1.4 release there are 75 similarly complex
enhancements/deprecations/incompatibilities - it just takes some time
to go through if you have not come across that topic before, eg,
clickjacking.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2B%2Bzdz0GK2jvJJ5tH1zLAi3agc3JNFhN4nWEXBBaLPATw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-02 Thread shmengie 
Pkl, 

You shouldn't have to upgrade all your old sites to the latest version of 
Django, unless you want maintain current methods. 

This is why we like virtual environments, much.

The core team has made a nice set of security releases available via pip.  
Django 1.4.14, Django 1.5.9, Django 1.6.6, and Django 1.7

# See package version in your environment
$\> pip freeze

# This command should get fetch security updates w/out breaking eco-system.
$\> pip install Django --upgrade


# Worst case scenario you'll have to un-install current version and Install a 
specific version.
$\> pip uninstall Django
$\> pip install Django==1.4.14 

More work to upgrade a bunch of virtuaenv's, but not as much as keeping all 
sites current, don't break your eco-system, unless there's a reason.

It's wise to choose a balance best practices that suit your eco-system.

I'd thank the team for issuing security releases for old versions.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1ddd5bbb-b2e6-4791-a92f-96bf997a9e1a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-02 Thread Aymeric Augustin 
2014-09-02 15:33 GMT+02:00 Tom Evans :

> this story was scored
> at 8 points, it took a junior developer much longer than 8 points and
> wasn't finished in a single sprint - and 1.3->1.4 was *easy*
>

I don't know how much a point or a sprint is worth in this context :-/

I know that upgrading is very painful if you do it without following a
strict
process. Shotgun debugging is awful in this context. Here's the process I
would recommend.

1. Upgrade your dependencies. If possible (depending on how many you have)
check that they support the target Django version. Otherwise, you'll notice
at
step 3.

2. Read through the release notes. At each bullet point, do a quick check
for
affected code with "Find all in project" and fix it. It doesn't matter if
you
miss something at this stage. Provided you read every item, you'll make the
connection if you hit the problem. This is tedious, but if you do it another
way, you'll waste more time.

3. Run your test suite under -Wall or even -Werror until it's clean. Fix
stuff
you've missed until it passes.

Don't forget that, if your project didn't run cleanly under the previous
Django version, you need to start from the previous release notes and clear
deprecation warnings from your own code.

Did your junior developer follow a similar process? If he did, what took
them
so long?

What I want to understand is how much of an advantage I have when it comes
to
upgrading Django. What do I know that isn't written down and makes it so
hard
for others?

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANE-7mV0tbQHf57B1H6YA6sSC95k-3GA97zbaCKCkX800P1b4g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-02 Thread Tom Evans 
On Mon, Sep 1, 2014 at 9:45 PM, Pkl  wrote:
>
> Hello,
>
> I once was once lured to an ideal of long-term stability and
> retrocompatibility, by nice docs like this one :
> https://docs.djangoproject.com/en/dev/misc/api-stability/
>
> But for some years, stuffs have actually been getting worse and worse, with
> each django release bringing its little crowd of nightmares.
>

Our solution is to not chase releases. Django 1.4 is marked as "Long
Term Stable", all our projects use Django 1.4 and no more; if/when a
newer stable release is proposed we will consider moving our projects
to that. We do then have some problems with 3rd party apps that do not
support 1.4..

There was a thread a few weeks ago that discussed (perhaps
tangentially) what the next LTS release branch will be, where it was
brought up that whatever release it is, the API will actually have to
be stable(!), probably ruling out 1.7. So, LTS will probably be 1.4
for some time yet.

On Tue, Sep 2, 2014 at 1:48 PM, Aymeric Augustin
 wrote:
> I'm not sure I understand completely where your frustration comes from. The
> tone of your email seems disproportionate with the effort needed to replace
> all instances of "django.conf.urls.defaults" with "django.conf.urls". That
> takes about ten minutes, if you count the time to locate the information in
> the release notes and make the commit.

It's not really, though is it? You encounter the problem because
you've upgraded django version; this will not be the only thing that
is broken, and so fixing this one thing will not enable you to run
your test suites.. it can take several iterations before you know that
it is fixed.

If you track django development and have used django for several
years, then these changes are relatively easy. If you are new to using
django, it is not so obvious and easy; as an example, we had an old
site running on django 1.3 that we wanted to move to 1.4 so that we
are using the same version of django throughout; this story was scored
at 8 points, it took a junior developer much longer than 8 points and
wasn't finished in a single sprint - and 1.3->1.4 was *easy*! We
prefer to spend our points on our own features (and senior developers
are expensive :)


Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BW2E0UO4kZZBwGHf%3DbA4%2BH4-u95T3jtGkUVX03ZOzQuQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-02 Thread Yo-Yo Ma 
With as many new frameworks as there are out there, with the gains in 
popularity seen in Go and Node, my thinking is, move quickly, break (some very 
small) things, or die slowly. As was said, if your favorite lib doesn't work 
with 1.6 or 1.7, either use a prior version, or spend some time contributing 
the 10 minutes to 2 hours that most of these changes would take at most. You'll 
feel good about it. You'll feel less frustrated, and you'll understand, 
thereafter, what it means to be an active part of the community. I too used to 
be a rail bird. It gets you "exactly nowhere", in the words of Russell K.M., 
and it makes those who contribute feel unappreciated.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ec509e6c-c72c-43f8-8d2d-eebc87347958%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-02 Thread Aymeric Augustin 
Hello,

While I generally agree with what other core devs have said regarding the
examples you raised, I still find your feedback interesting because it
reminds us that we iterate too quickly for at least some parts of the
community.

It's unavoidable that some people will find the pace of Django's evolution
too fast and others will find it too slow. We're trying to ease the pain of
the former group by documenting thoroughly the changes in each release.

I'm not sure I understand completely where your frustration comes from. The
tone of your email seems disproportionate with the effort needed to replace
all instances of "django.conf.urls.defaults" with "django.conf.urls". That
takes about ten minutes, if you count the time to locate the information in
the release notes and make the commit. Is it a case of death by a thousand
cuts? Perhaps you could explain how you proceed when you need to upgrade
Django in your projects? Are the release notes somehow lacking?

Overall, I don't think we're going to change our pace — especially when it
comes to security-related issues — but we can try to improve our
communication around deprecations and removals.

-- 
Aymeric.

2014-09-01 22:45 GMT+02:00 Pkl :

>
> Hello,
>
> I once was once lured to an ideal of long-term stability and
> retrocompatibility, by nice docs like this one :
> https://docs.djangoproject.com/en/dev/misc/api-stability/
>
> But for some years, stuffs have actually been getting worse and worse,
> with each django release bringing its little crowd of nightmares.
>
> In random order, I stumbled upon:
> - removal of django.conf.urls.defaults
> - removal of markup contrib lib (adios built-in RST support)
> - removal of request.raw_post_data (thus breaking about all existing
> webservice libs)
> - lots of changes in mandatory settings, like allowed_hosts or DB conf
> format,
> - future removal of the very basic "mimetype" argument from django objects
> (how many apps impacted ? quite MANY I guess)
> - future removal of request.REQUEST (aka "we're not all consenting adults,
> let's remove all tools that people could misuse")
>
> For example, I had great fun installing django + djangocms + zinnia +
> cms_plugin_zinnia ; the probability of having no version mismatch between
> all these components being more or less zero.
>
> And the future looks grimmer again, having a look at
> https://docs.djangoproject.com/en/dev/internals/deprecation/.
>
> I don't get this relentlessness at breaking, for the sake of purity, all
> stuffs that are not actively maintained, yet could be perfectly working.
>
> What is easier, in a deeply dynamic language like python, than keeping
> retrocompatibility ? Why not keep "raw_post_data" as an undocumented alias
> for "body" ? Why not keep "mimetype" as a silent alias for "content_type" ?
> What harm did the "markup" support in django, since no one expects such
> extensions to be 100% secure anyway (no software on earth is) ? If code
> "purity" is wanted, then why not have all these monkey-patched from a
> builtin, but removable, "django.compatibility" util ? Why not wait for 2.0,
> to remove all compatibility aliases in one single shot ?
>
> The one killing features of django is its app ecosystem - being able to
> plug blogs, authentication systems, webservice generators, and all other
> kinds of applications, into this common framework, and benefit from the
> common auto-admin. And since all these apps don't evolve at the same rythm,
> retrocompatibility is IMHO a NECESSITY for django, not a LUXURY. Or isn't
> it ?
>
> Please don't kill the ecosystem, please respect the promises of
> api-stability that were once made.
>
> regards,
> Pkl
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/6abfd1ee-0056-41c1-b079-f89f2f463cf5%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit

Re: Please don't kill the ecosystem

2014-09-02 Thread Florian Apolloner 
Hi Pkl,

On Monday, September 1, 2014 10:45:30 PM UTC+2, Pkl wrote:
>
> In random order, I stumbled upon:
> - removal of django.conf.urls.defaults
> - removal of markup contrib lib (adios built-in RST support)
> - removal of request.raw_post_data (thus breaking about all existing 
> webservice libs)
> - lots of changes in mandatory settings, like allowed_hosts or DB conf 
> format, 
> - future removal of the very basic "mimetype" argument from django objects 
> (how many apps impacted ? quite MANY I guess)
> - future removal of request.REQUEST (aka "we're not all consenting adults, 
> let's remove all tools that people could misuse")
>

At least three of those are security related, if you read the api stability 
docs you will see that we will not consider api stability if this affects a 
security issue. If that creates an issue for you, then by all means Django 
is not for you. As for the other changes, I'd say they are debatable, but 
caused serious confusion for users (I am helping out on IRC quite a bit, so 
I was really glad to see them changed). At the speed we release Django, 
this gives you roughly two years to change your code, I think this is 
reasonable enough to ask for.

For example, I had great fun installing django + djangocms + zinnia + 
> cms_plugin_zinnia ; the probability of having no version mismatch between 
> all these components being more or less zero.
>

Oh, how so? They say to support Django 1.6 -- I doubt any of the above 
stated issues caused breakage there.
 

> I don't get this relentlessness at breaking, for the sake of purity, all 
> stuffs that are not actively maintained, yet could be perfectly working. 
>

If we would break for purity, we'd be breaking a lot more…
 

> What is easier, in a deeply dynamic language like python, than keeping 
> retrocompatibility ? Why not keep "raw_post_data" as an undocumented alias 
> for "body" ? Why not keep "mimetype" as a silent alias for "content_type" ?
>

For what? By the time we remove the feature from Django all apps should 
work with new Django versions. The older Django versions we are dropping 
support for by not keeping raw_post_data are no longer maintained and have 
__known__ and sometimes __easily__ exploitable security issues.

What harm did the "markup" support in django, since no one expects such 
> extensions to be 100% secure anyway (no software on earth is) ? 
>

No software on earth is 100% secure, but supporting something which has 
quite a few security implications and having to release security releases 
just for this component quite often isn't going to help. In the end users 
expect us to deliver secure code for known security issues in the third 
party libraries, this is not something we want to do.
 

> If code "purity" is wanted, then why not have all these monkey-patched 
> from a builtin, but removable, "django.compatibility" util ? Why not wait 
> for 2.0, to remove all compatibility aliases in one single shot ?
>

You feel up to this? This should work perfectly fine as 3rd party app, just 
don't expect us to maintain compatibility for releases which aren't even 
under security maintenance anymore.

The one killing features of django is its app ecosystem - being able to 
> plug blogs, authentication systems, webservice generators, and all other 
> kinds of applications, into this common framework, and benefit from the 
> common auto-admin. And since all these apps don't evolve at the same rythm, 
> retrocompatibility is IMHO a NECESSITY for django, not a LUXURY. Or isn't 
> it ?
>

Dunno, I usually never had problems. That said I only use relatively 
maintained apps, cause I want to be able to get bugs fixed "soonish".

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ac4cd2d9-ed9b-4f9b-bd87-68ff5838ebcd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-01 Thread Marc Tamlyn 
Apologies if this does not directly address your points, but I recently
took over a Django site still running 1.1. I brought it fully up to 1.6 in
under 2 hours work, including rearranging the project structure, updating
settings, URL tags, import paths, switching to pip from buildout etc.
Considering the version of Django this was written for is five years old I
consider this an impressive achievement in backwards compatibility.

On another note, given Django's slow release cycle, deprecated features are
visible if you run with warnings for between 18 months and 2 years before
they stop working. I don't think it is unreasonable to expect a small
amount of maintenance within these time frames.

As the Django ecosystem is aging, it is sadly inevitable that some
unmaintained projects will stop working. The only way Django could avoid
this is complete stagnation which would kill Django much more than the
small maintenance burden it imposes.

Django remains one of the most stable web frameworks out there, but it must
also continue to evolve or risk being IE6.
On 2 Sep 2014 01:29, "Curtis Maloney"  wrote:

> Mostly for clarity and historical purpose, I'd like to expand on the
> points you listed.
>
> Do not take this in any way as a dismissal of the issues you've raised.
>
> On 2 September 2014 06:45, Pkl  wrote:
>
>>
>> Hello,
>>
>> I once was once lured to an ideal of long-term stability and
>> retrocompatibility, by nice docs like this one :
>> https://docs.djangoproject.com/en/dev/misc/api-stability/
>>
>> But for some years, stuffs have actually been getting worse and worse,
>> with each django release bringing its little crowd of nightmares.
>>
>> In random order, I stumbled upon:
>> - removal of django.conf.urls.defaults
>>
> This was moved into django.conf.urls ...
>
>
>> - removal of markup contrib lib (adios built-in RST support)
>>
> From memory, this was due to a security issue with the Markdown library
> used.  I, too , was hit by this, and ended up vendoring the tag library
> into my project.
>
>
>> - removal of request.raw_post_data (thus breaking about all existing
>> webservice libs)
>>
> It was renamed to request.body, not removed.  This is because it
> represents the request body, no matter the HTTP method used.
>
>
>> - lots of changes in mandatory settings, like allowed_hosts or DB conf
>> format,
>>
> ALOWED_HOSTS was added as a security measure.
> The only significant DB config format I can recall was adding multi-DB
> support.  Was there another?
>
> - future removal of the very basic "mimetype" argument from django objects
>> (how many apps impacted ? quite MANY I guess)
>>
> IIRC this was renamed to content-type, since "mime type" is, in fact, for
> email.
>
>
>> - future removal of request.REQUEST (aka "we're not all consenting
>> adults, let's remove all tools that people could misuse")
>>
> I remember the discussion for this.  Primarily it was because it did cause
> security issues, as well as being unclear as to which was looked up first-
> GET or POST.
>
> As someone who has been using Django since 0.91... I've seen  a lot of
> change over the years, some more drastic than others.
>
> I don't get this relentlessness at breaking, for the sake of purity, all
>> stuffs that are not actively maintained, yet could be perfectly working.
>>
>
> Whilst it could be argued that, for instance, request.body and
> requset.REQUEST were removed for "purity", it can also be said they were
> removed for clarity (to avoid confusion) and safety (to avoid real security
> issues).
>
> As for breaking unmaintained 3rd party apps... sorry, that's how the world
> works.  If you want that project to keep up to date with Django, either
> incentivise the devlopers to do so, or contribute to them yourself.  Such
> is open source.
>
>
>> What is easier, in a deeply dynamic language like python, than keeping
>> retrocompatibility ?
>>
> I do like this idea... it should be possible to monkey-path in at least
> some of these ideas.
> Open a ticket, CC me, and I'll happily work with you on a PR.
> If nothing else, we can make it a 3rd party app.
>
>
>> Why not keep "raw_post_data" as an undocumented alias for "body" ? Why
>> not keep "mimetype" as a silent alias for "content_type" ?
>>
> mimetype _was_ an alias for content_type for, IIRC, much longer than the
> usual 2-release deprecation cycle.
>
>
>> What harm did the "markup" support in django, since no one expects such
>> extensions to be 100% secure anyway (no software on earth is) ? If code
>> "purity" is wanted, then why not have all these monkey-patched from a
>> builtin, but removable, "django.compatibility" util ? Why not wait for 2.0,
>> to remove all compatibility aliases in one single shot ?
>>
> The current trend with contrib apps is to remove them if they'd benefit
> from a more frequent release cycle - one not tied to that of core.
> Howerver, I was a little surprised to see

Re: Please don't kill the ecosystem

2014-09-01 Thread Curtis Maloney 
Mostly for clarity and historical purpose, I'd like to expand on the points
you listed.

Do not take this in any way as a dismissal of the issues you've raised.

On 2 September 2014 06:45, Pkl  wrote:

>
> Hello,
>
> I once was once lured to an ideal of long-term stability and
> retrocompatibility, by nice docs like this one :
> https://docs.djangoproject.com/en/dev/misc/api-stability/
>
> But for some years, stuffs have actually been getting worse and worse,
> with each django release bringing its little crowd of nightmares.
>
> In random order, I stumbled upon:
> - removal of django.conf.urls.defaults
>
This was moved into django.conf.urls ...


> - removal of markup contrib lib (adios built-in RST support)
>
>From memory, this was due to a security issue with the Markdown library
used.  I, too , was hit by this, and ended up vendoring the tag library
into my project.


> - removal of request.raw_post_data (thus breaking about all existing
> webservice libs)
>
It was renamed to request.body, not removed.  This is because it represents
the request body, no matter the HTTP method used.


> - lots of changes in mandatory settings, like allowed_hosts or DB conf
> format,
>
ALOWED_HOSTS was added as a security measure.
The only significant DB config format I can recall was adding multi-DB
support.  Was there another?

- future removal of the very basic "mimetype" argument from django objects
> (how many apps impacted ? quite MANY I guess)
>
IIRC this was renamed to content-type, since "mime type" is, in fact, for
email.


> - future removal of request.REQUEST (aka "we're not all consenting adults,
> let's remove all tools that people could misuse")
>
I remember the discussion for this.  Primarily it was because it did cause
security issues, as well as being unclear as to which was looked up first-
GET or POST.

As someone who has been using Django since 0.91... I've seen  a lot of
change over the years, some more drastic than others.

I don't get this relentlessness at breaking, for the sake of purity, all
> stuffs that are not actively maintained, yet could be perfectly working.
>

Whilst it could be argued that, for instance, request.body and
requset.REQUEST were removed for "purity", it can also be said they were
removed for clarity (to avoid confusion) and safety (to avoid real security
issues).

As for breaking unmaintained 3rd party apps... sorry, that's how the world
works.  If you want that project to keep up to date with Django, either
incentivise the devlopers to do so, or contribute to them yourself.  Such
is open source.


> What is easier, in a deeply dynamic language like python, than keeping
> retrocompatibility ?
>
I do like this idea... it should be possible to monkey-path in at least
some of these ideas.
Open a ticket, CC me, and I'll happily work with you on a PR.
If nothing else, we can make it a 3rd party app.


> Why not keep "raw_post_data" as an undocumented alias for "body" ? Why not
> keep "mimetype" as a silent alias for "content_type" ?
>
mimetype _was_ an alias for content_type for, IIRC, much longer than the
usual 2-release deprecation cycle.


> What harm did the "markup" support in django, since no one expects such
> extensions to be 100% secure anyway (no software on earth is) ? If code
> "purity" is wanted, then why not have all these monkey-patched from a
> builtin, but removable, "django.compatibility" util ? Why not wait for 2.0,
> to remove all compatibility aliases in one single shot ?
>
The current trend with contrib apps is to remove them if they'd benefit
from a more frequent release cycle - one not tied to that of core.
Howerver, I was a little surprised to see django.contrib.markup wasn't
moved to a django-maintained 3rd party app like some others were.  Perhaps
open a ticket for that?


>
> The one killing features of django is its app ecosystem - being able to
> plug blogs, authentication systems, webservice generators, and all other
> kinds of applications, into this common framework, and benefit from the
> common auto-admin. And since all these apps don't evolve at the same rythm,
> retrocompatibility is IMHO a NECESSITY for django, not a LUXURY. Or isn't
> it ?
>


>
> Please don't kill the ecosystem, please respect the promises of
> api-stability that were once made.
>
These promises have been kept.  There is a LOT of code that could be
dropped or rewritten immediately if it weren't for the 2-relese deprecation
cycle.

One of my PRs was rejected specifically because I hadn't implemented a
deprecation path.  It would likely speed up template rendering, which I
think everyone agrees is desirable.

--
Curtis

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group

Re: Please don't kill the ecosystem

2014-09-01 Thread Tim Graham 
Pkl, if you're interested, I'd encourage you to be more involved in the 
development process including this mailing list when changes are proposed. 
I'm afraid raising a whole bunch of old issues that have already been 
discussed and decided isn't that helpful.

Our deprecation policy (as you probably know) is to maintain backwards 
compatibility (where at all possible) for two releases. Keeping these shims 
indefinitely, however, would add a lot of complexity and hinder our ability 
to evolve the framework. I don't believe we've diverged from our API 
stability policy regarding the deprecation process.

As you may know, we have added an experimental "Long Term Support" release 
(currently 1.4) for projects that don't wish to update to the newest 
versions of Django as quickly as we put them out.

We've also decided that "2.0" will simply be the release following 1.9. It 
won't be a big backwards incompatible release with sweeping changes.

If you have other suggestions to address your concerns let us know. Out of 
respect for the many volunteers that contribute their time to Django, 
please consider toning down the grim tone. :-)

Tim

On Monday, September 1, 2014 5:50:25 PM UTC-4, Collin Anderson wrote:
>
> I agree that there have been a lot of backwards-incompatible changes over 
> the last few years clean up the API. We've really been in "get stuff done" 
> mode.
>
> I do however think it's a not good idea to break all compatibility in one 
> single shot. Python 3 did this and 6 years later it's still preventing many 
> of us from even starting to use it. Spreading out the backwards 
> incompatibilities over many releases is a very good idea.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0fbe4ac0-317b-4a34-8e0e-118b7f39738e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

2014-09-01 Thread Collin Anderson 
I agree that there have been a lot of backwards-incompatible changes over 
the last few years clean up the API. We've really been in "get stuff done" 
mode.

I do however think it's a not good idea to break all compatibility in one 
single shot. Python 3 did this and 6 years later it's still preventing many 
of us from even starting to use it. Spreading out the backwards 
incompatibilities over many releases is a very good idea.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e107ef5f-9127-4cda-951d-b2441be9a1bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.