Re: Binary naming for Django Related Packages
On Wednesday, January 18, 2017 10:04:24 AM IOhannes m zmölnig wrote: > On 2017-01-18 07:46, Scott Kitterman wrote: > > +··named·django_packagename·upstream.··These·are·then·packaged·as > > +··python3-django-package·and > > please use "package" vs "packagename" consistently. > e.g. an upstream named "django_packagename" should be packaged as > "python3-django-packagename". > > It's kind of obvious, but I think the policy should be precise. > > (and probably use "" or "$packagename" or something else to > mark it as variable) Thanks. I went with $name to reduce the number of times we use the word package in the paragraph. Based on your feedback and the lack of other feedback, here's what I've committed to the VCS for the next upload (rfcdiff html attached). Scott KTitle: Diff: python-policy.txt.old - python-policy.txt.new python-policy.txt.old python-policy.txt.new skipping to change at line 516 skipping to change at line 516 Appendix B, `Packaging Tools'). For example, if Python 3.3, 3.4, and Appendix B, `Packaging Tools'). For example, if Python 3.3, 3.4, and 3.5 are supported, the Python statement 3.5 are supported, the Python statement import foo import foo should import the module when the program interpreter is any of should import the module when the program interpreter is any of `/usr/bin/python3.3', `/usr/bin/python3.4', and `/usr/bin/python3.5'. `/usr/bin/python3.3', `/usr/bin/python3.4', and `/usr/bin/python3.5'. This requirement also applies to extension modules; binaries for all This requirement also applies to extension modules; binaries for all the supported Python versions should be included in a single package. the supported Python versions should be included in a single package. As a special exception to the `python3-' and `python-' binary naming Packages intended for use with Django (`python3-django'/ policy, Python modules intended for use with Django (`python3-django'/ `python-django') are installed in the same namespace as other python `python-django') should add django to their binary package names to packages for a variety of reasons. Many such packages are named make it clear they are intended for use with Django and not general django_$name upstream. These are then packaged as purpose Python modules, i.e. `python3-django-' and `python-django-' `python3-django-$name' and `python-django-$name'. This makes it clear respectively. that they are intended for use with Django and not general purpose Python modules. Debian maintainers are encouraged to work with their upstreams to support consistent use of this approach. 3.4. Specifying Supported Versions 3.4. Specifying Supported Versions -- -- The `debian/control' source paragraph may contain optional fields to The `debian/control' source paragraph may contain optional fields to specify the versions of Python the package supports. specify the versions of Python the package supports. The optional `X-Python3-Version' field specifies the versions of The optional `X-Python3-Version' field specifies the versions of Python 3 supported. When not specified, it defaults to all currently Python 3 supported. When not specified, it defaults to all currently supported Python 3 versions. supported Python 3 versions. End of changes. 1 change blocks. 6 lines changed or deleted 8 lines changed or added This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/
Re: Binary naming for Django Related Packages
On 2017-01-18 07:46, Scott Kitterman wrote: > +··named·django_packagename·upstream.··These·are·then·packaged·as > +··python3-django-package·and please use "package" vs "packagename" consistently. e.g. an upstream named "django_packagename" should be packaged as "python3-django-packagename". It's kind of obvious, but I think the policy should be precise. (and probably use "" or "$packagename" or something else to mark it as variable) gfmadr IOhannes
Re: Binary naming for Django Related Packages
On Wednesday, December 07, 2016 11:43:29 AM Raphael Hertzog wrote: > Hi, > > On Wed, 30 Nov 2016, Scott Kitterman wrote: > > Raphael, do you think that the upstream Django project might be willing to > > make some kind of best practices for naming third party django packages? > > If they did that, then that would give us a basis for Debian maintainers > > talking to their upstreams about moving to django_. > > They already partly do that, see: > https://docs.djangoproject.com/en/1.10/intro/reusable-apps/#packaging-your-a > pp > > They recommend a "django-" prefix in the PyPi package name. But they say > nothing about the Python module name and the sample just bundles a "polls" > module in a "django-polls" package. > > Thus I posted this to gather their feedback on the need to recommend the > prefix on the name of the module too: > https://groups.google.com/forum/#!topic/django-developers/f8yNRkn6Fpo Thanks. Since not everyone liked my first attempt at this, I softened it a bit (please see the attached). We're going to have at least one more python- defaults upload and I'd like to get this resolved. Scott K=== modified file 'debian/python-policy.sgml' --- debian/python-policy.sgml 2016-11-26 07:23:09 + +++ debian/python-policy.sgml 2017-01-18 06:39:18 + @@ -628,14 +628,16 @@ versions should be included in a single package. - As a special exception to the python3- and - python- binary naming policy, Python modules - intended for use with Django (python3-django/ - python-django) should add django to their binary - package names to make it clear they are intended for use with Django - and not general purpose Python modules, i.e. - python3-django- and - python-django- respectively. + Packages intended for use with Django (python3-django/ + python-django) are installed in the same namespace as + other python packages for a variety of reasons. Many such packages are + named django_packagename upstream. These are then packaged as + python3-django-package and + python-django-package. + This makes it clear that they are intended for use with Django + and not general purpose Python modules. Debian maintainers are + encouraged to work with their upstreams to support consistent use of + this approach. Specifying Supported Versions
Re: Binary naming for Django Related Packages
Hi, On Wed, 30 Nov 2016, Scott Kitterman wrote: > Raphael, do you think that the upstream Django project might be willing to > make some kind of best practices for naming third party django packages? If > they did that, then that would give us a basis for Debian maintainers talking > to their upstreams about moving to django_. They already partly do that, see: https://docs.djangoproject.com/en/1.10/intro/reusable-apps/#packaging-your-app They recommend a "django-" prefix in the PyPi package name. But they say nothing about the Python module name and the sample just bundles a "polls" module in a "django-polls" package. Thus I posted this to gather their feedback on the need to recommend the prefix on the name of the module too: https://groups.google.com/forum/#!topic/django-developers/f8yNRkn6Fpo Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: http://www.freexian.com/services/debian-lts.html Learn to master Debian: http://debian-handbook.info/get/
Re: Binary naming for Django Related Packages
On 2016-12-03 17:01:45 +0100 (+0100), Thomas Goirand wrote: [...] > Because of problems when doing imports in Python3 (in a venv, the system > module wont be loaded if it's there and there's already something in the > venv), we should attempt to discourage upstream to use namespaced > modules. This indeed could prevent from running unit tests. That's what > has been discovered in the OpenStack world, and now all the oslo libs > aren't using namespace (though we've kept the dot for the egg-names). To clarify, the main issue encountered there was a conflict over namespace-level init when some modules were editable installs. Historical details of the decision are outlined at: https://specs.openstack.org/openstack/oslo-specs/specs/kilo/drop-namespace-packages.html#problem-description > -- Jeremy Stanley
Re: Binary naming for Django Related Packages
On 11/28/2016 05:11 PM, Scott Kitterman wrote: > I've recently done some Django related packaging for the first time and > noticed that we have organically (as far as I can tell) grown a slightly > different naming convention for such packages. Instead of python*-foo, we > use > python*-django-foo. > > I think this is a reasonable approach and followed it in the new packages > I've > recently done. > > I decided to check and see how common the approach is. Here's what I found > in > Sid: > > Start with django: 7 > Start w/django, not transitional: 2 > Start with django3: 0 > > Start with python-django (excluding -doc): 136 > Start with python3-django: 84 > > I think it would make sense to add this to the Python policy so how we're > doing it is documented. I am attaching a proposed diff. I made it a should > because there are two non-DPMT packages that don't follow this rule and I > think it's late in the cycle to be adding to must policy requirements. > > Please let me know what you think. I'm open to suggestions on wording. I'd > like to get this done in the next week and do a python-defaults upload with > this and a few minor (non-policy) changes that are pending. > > Scott K > > @@ -534,6 +534,13 @@ > This requirement also applies to extension modules; binaries for all > the supported Python versions should be included in a single package. > > + As a special exception to the `python3-' and `python-' binary naming > + policy, Python modules intended for use with Django (`python3-django'/ > + `python-django') should add django to their binary package names to > + make it clear they are intended for use with Django and not general > + purpose Python modules, i.e. `python3-django-' and `python-django-' > + respectively. IMO, what should drive the binary package name is what upstream sets as egg name, so that we don't have to do pydists-overrides. The current global Python policy is to use the Python module names (ie: what one would write when doing an import), which IMO is wrong, because we got to do so many substitutions to have the Depends correct. If our policy was to use egg-name for package names, we wouldn't have any substitution to calculate. Anyway, I don't see why Django modules should be an exception to any rule we choose. If upstream is missing the "django-" prefix, then we should suggest it. Thomas Goirand (zigo)
Re: Binary naming for Django Related Packages
On 11/28/2016 05:30 PM, Barry Warsaw wrote: > On Nov 28, 2016, at 11:11 AM, Scott Kitterman wrote: > >> @@ -534,6 +534,13 @@ >> This requirement also applies to extension modules; binaries for all >> the supported Python versions should be included in a single package. >> >> + As a special exception to the `python3-' and `python-' binary naming >> + policy, Python modules intended for use with Django (`python3-django'/ >> + `python-django') should add django to their binary package names to >> + make it clear they are intended for use with Django and not general >> + purpose Python modules, i.e. `python3-django-' and `python-django-' >> + respectively. > > +1 but I have a question since I'm not a hardcore Django developer. > > In many cases we have namespace packages, e.g. zope.*, flufl.*, etc. Usually > these will be called python-., e.g. python-flufl.i18n. > > Is there any risk of having confusing names because of a conflict between a > 3rd party Django module and a Django subpackage? e.g. python3-django-foo > vs. python3-django.foo. > > I'm sure it's a non-issue in practice. > > Cheers, > -Barry Because of problems when doing imports in Python3 (in a venv, the system module wont be loaded if it's there and there's already something in the venv), we should attempt to discourage upstream to use namespaced modules. This indeed could prevent from running unit tests. That's what has been discovered in the OpenStack world, and now all the oslo libs aren't using namespace (though we've kept the dot for the egg-names). Cheers, Thomas Goirand (zigo)
Re: Binary naming for Django Related Packages
On Tuesday, November 29, 2016 02:40:06 PM Piotr Ożarowski wrote: > [Scott Kitterman, 2016-11-29] > > > Piotr: Is there some language that acknowledges the situation as unusual, > > even if it doesn't fully bless it that you'd be comfortable with in > > policy so we can at least document current practice? > > if module name is foo, name of the binary package should be python3-foo, > not python3-bar-foo. There is no change needed on the upstream side or > in django/__init__.py - it's just about naming binary packages the way > our policy recommends. Adjusting policy to document good changes we came > up in practice is good. Documenting bad ones is not. > > That said, I don't have a veto vote so I will just have to deal with it. I can see your point. I can see Raphael's too. I do think that upstream third parties using django_foo represents some kind of best practice that should be documented and encouraged (but not directly by Debian). Raphael, do you think that the upstream Django project might be willing to make some kind of best practices for naming third party django packages? If they did that, then that would give us a basis for Debian maintainers talking to their upstreams about moving to django_. Over time, that would result in python-django-foo being the correct name without any kind of Python policy exception (and we'd limit this to being temporary). How's that? Scott K
Re: Binary naming for Django Related Packages
[Scott Kitterman, 2016-11-29] > Piotr: Is there some language that acknowledges the situation as unusual, > even > if it doesn't fully bless it that you'd be comfortable with in policy so we > can at least document current practice? if module name is foo, name of the binary package should be python3-foo, not python3-bar-foo. There is no change needed on the upstream side or in django/__init__.py - it's just about naming binary packages the way our policy recommends. Adjusting policy to document good changes we came up in practice is good. Documenting bad ones is not. That said, I don't have a veto vote so I will just have to deal with it. -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
Re: Binary naming for Django Related Packages
On Tuesday, November 29, 2016 01:52:07 PM Raphael Hertzog wrote:
> On Mon, 28 Nov 2016, Scott Kitterman wrote:
> > > > Please let me know what you think. I'm open to suggestions on
> > > > wording.
> > > > I'd like to get this done in the next week and do a python-defaults
> > > > upload with this and a few minor (non-policy) changes that are
> > > > pending.
>
> +1 from me. I'm actually the one who started this convention when I
> packaged the first Django extension. When I look for available Django
> extension, I like to be able to rely on the prefix.
>
> > > [²] sys.path.append('/usr/lib/python3/django-packages/') in
> > >
> > > django/__init__.py if django import always prepends other imports
> > > (python3-django- namespace would be tolerable then, I guess)
> >
> > I'm not one of the python-django uploaders, so we'd need their feedback on
> > [2]. I think something like that is a reasonable compromise if they are
> > willing to support it.
>
> I certainly don't want to introduce this Debian-specific difference, no.
> Django applications/extensions are meant to be managed via "pip" and they
> must be available in the global namespace. I would not be surprised that
> some of the extensions actually rely on being available globally...
>
> I don't see any benefit to this change. The global namespace pollution
> already exists at the upstream level, while we have to handle potential
> conflicts, it's not up to us to preventively curate the namespace when
> upstream has not followed the best practice (i.e. the "django_" prefix
> in the module name).
Thanks for the feedback. I think that eliminates Piotr's options 2.
Personally, I think policy is at its best documenting current practice rather
than to drive it, that's why I made the initial proposal. There's an
exception to the usual rule that is virtually universally applied, so I
believe we ought to document it.
Piotr: Is there some language that acknowledges the situation as unusual, even
if it doesn't fully bless it that you'd be comfortable with in policy so we
can at least document current practice?
Scott K
Re: Binary naming for Django Related Packages
On Mon, 28 Nov 2016, Scott Kitterman wrote:
> > > Please let me know what you think. I'm open to suggestions on wording.
> > > I'd like to get this done in the next week and do a python-defaults
> > > upload with this and a few minor (non-policy) changes that are pending.
+1 from me. I'm actually the one who started this convention when I
packaged the first Django extension. When I look for available Django
extension, I like to be able to rely on the prefix.
> > [²] sys.path.append('/usr/lib/python3/django-packages/') in
> > django/__init__.py if django import always prepends other imports
> > (python3-django- namespace would be tolerable then, I guess)
>
> I'm not one of the python-django uploaders, so we'd need their feedback on
> [2]. I think something like that is a reasonable compromise if they are
> willing to support it.
I certainly don't want to introduce this Debian-specific difference, no.
Django applications/extensions are meant to be managed via "pip" and they
must be available in the global namespace. I would not be surprised that
some of the extensions actually rely on being available globally...
I don't see any benefit to this change. The global namespace pollution
already exists at the upstream level, while we have to handle potential
conflicts, it's not up to us to preventively curate the namespace when
upstream has not followed the best practice (i.e. the "django_" prefix
in the module name).
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/
Re: Binary naming for Django Related Packages
On Mon, 28 Nov 2016, Piotr Ożarowski wrote: > [Barry Warsaw, 2016-11-28] > > Is there any risk of having confusing names because of a conflict between a > > 3rd party Django module and a Django subpackage? e.g. python3-django-foo > > vs. python3-django.foo. > > > > I'm sure it's a non-issue in practice. > > this is a huge issue IMHO beacause Django submodules use global > namespace and thus any unique django submodule name takes not so unique > Python module name (i.e. they're installed under > /usr/lib/python3/dist-packages/ now, not under > /usr/lib/python3/dist-packages/django) This is true but it's still a non-issue in practice because that kind of conflict is usually detected and thus avoided at the pypi level. And checking for file conflict is part of the job of the packager and we have QA tools doing that kind of work too. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: http://www.freexian.com/services/debian-lts.html Learn to master Debian: http://debian-handbook.info/get/
Re: Binary naming for Django Related Packages
On Monday, November 28, 2016 05:50:24 PM Piotr Ożarowski wrote:
> [Scott Kitterman, 2016-11-28]
>
> > I've recently done some Django related packaging for the first time and
> > noticed that we have organically (as far as I can tell) grown a slightly
> > different naming convention for such packages. Instead of python*-foo, we
> > use python*-django-foo.
> >
> > I think this is a reasonable approach and followed it in the new packages
> > I've recently done.
> >
> > I decided to check and see how common the approach is. Here's what I
> > found in Sid:
> >
> > Start with django: 7
> > Start w/django, not transitional: 2
> > Start with django3: 0
> >
> > Start with python-django (excluding -doc): 136
> > Start with python3-django: 84
> >
> > I think it would make sense to add this to the Python policy so how we're
> > doing it is documented. I am attaching a proposed diff. I made it a
> > should because there are two non-DPMT packages that don't follow this
> > rule and I think it's late in the cycle to be adding to must policy
> > requirements.
> >
> > Please let me know what you think. I'm open to suggestions on wording.
> > I'd like to get this done in the next week and do a python-defaults
> > upload with this and a few minor (non-policy) changes that are pending.
>
> -1 from me
>
>
> If Django packages have no use outside Django¹, they should be moved out²
> of public dist-packages IMO. If they are useful, "-django" part is
> misleading.
>
> [¹] dash suggest they're not in django namespace, otherwise binary
> package name would be python3-django.foo
> (or python3-django.ext.foo, like in flask?)
> [²] sys.path.append('/usr/lib/python3/django-packages/') in
> django/__init__.py if django import always prepends other imports
> (python3-django- namespace would be tolerable then, I guess)
Some, but not all of them, are shipped as django_foo, so for those, I think
python*-django-foo is actually correct.
>From a django perspective, it makes a difference. As an example, django-
python-hstore (which I just packaged with python*-django-hstore binaries) is
django_hstore. When added to Django INSTALLED_APPS, it's added as
django_hstore. The django. namespace is for things shipped with Django, not
third-party packages.
Asking upstreams to rename to use django_ where they don't will complicate
things for upgrades, so I don't think that's a great idea.
I'm not one of the python-django uploaders, so we'd need their feedback on
[2]. I think something like that is a reasonable compromise if they are
willing to support it.
Thanks,
Scott K
Re: Binary naming for Django Related Packages
[Barry Warsaw, 2016-11-28] > Is there any risk of having confusing names because of a conflict between a > 3rd party Django module and a Django subpackage? e.g. python3-django-foo > vs. python3-django.foo. > > I'm sure it's a non-issue in practice. this is a huge issue IMHO beacause Django submodules use global namespace and thus any unique django submodule name takes not so unique Python module name (i.e. they're installed under /usr/lib/python3/dist-packages/ now, not under /usr/lib/python3/dist-packages/django) -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
Re: Binary naming for Django Related Packages
[Scott Kitterman, 2016-11-28]
> I've recently done some Django related packaging for the first time and
> noticed that we have organically (as far as I can tell) grown a slightly
> different naming convention for such packages. Instead of python*-foo, we
> use
> python*-django-foo.
>
> I think this is a reasonable approach and followed it in the new packages
> I've
> recently done.
>
> I decided to check and see how common the approach is. Here's what I found
> in
> Sid:
>
> Start with django: 7
> Start w/django, not transitional: 2
> Start with django3: 0
>
> Start with python-django (excluding -doc): 136
> Start with python3-django: 84
>
> I think it would make sense to add this to the Python policy so how we're
> doing it is documented. I am attaching a proposed diff. I made it a should
> because there are two non-DPMT packages that don't follow this rule and I
> think it's late in the cycle to be adding to must policy requirements.
>
> Please let me know what you think. I'm open to suggestions on wording. I'd
> like to get this done in the next week and do a python-defaults upload with
> this and a few minor (non-policy) changes that are pending.
-1 from me
If Django packages have no use outside Django¹, they should be moved out²
of public dist-packages IMO. If they are useful, "-django" part is misleading.
[¹] dash suggest they're not in django namespace, otherwise binary
package name would be python3-django.foo
(or python3-django.ext.foo, like in flask?)
[²] sys.path.append('/usr/lib/python3/django-packages/') in
django/__init__.py if django import always prepends other imports
(python3-django- namespace would be tolerable then, I guess)
--
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl www.griffith.cc www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
Re: Binary naming for Django Related Packages
+1 On 28/11/2016 17:11, Scott Kitterman wrote: Snark on #debian-python
Re: Binary naming for Django Related Packages
On Nov 28, 2016, at 11:11 AM, Scott Kitterman wrote: >@@ -534,6 +534,13 @@ > This requirement also applies to extension modules; binaries for all > the supported Python versions should be included in a single package. > >+ As a special exception to the `python3-' and `python-' binary naming >+ policy, Python modules intended for use with Django (`python3-django'/ >+ `python-django') should add django to their binary package names to >+ make it clear they are intended for use with Django and not general >+ purpose Python modules, i.e. `python3-django-' and `python-django-' >+ respectively. +1 but I have a question since I'm not a hardcore Django developer. In many cases we have namespace packages, e.g. zope.*, flufl.*, etc. Usually these will be called python-., e.g. python-flufl.i18n. Is there any risk of having confusing names because of a conflict between a 3rd party Django module and a Django subpackage? e.g. python3-django-foo vs. python3-django.foo. I'm sure it's a non-issue in practice. Cheers, -Barry pgpPkLIxR9pNA.pgp Description: OpenPGP digital signature
