Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Nathaniel Smith]

On Aug 21, 2016 9:18 AM, "Nick Coghlan"  wrote:
>
[...]
> By contrast, I *know* Linux distros are in the habit of pulling
> release tarballs from PyPI and feeding them directly into their
> release pipelines, so the potential for unanticipated breakage seems
> much higher there, and more likely to fall on community distros rather
> than on commercial Python redistributors (simply because there are
> more volunteers working on Linux based tooling than there are on
> WIndows developer tools).

But those release pipelines today have no problem handling the ~10% of
sdists that are zips. For my personal projects actually I've always
released .zip sdists exclusively (just because it seemed like a more
universally convenient format than .tar.gz), and those packages have shown
up in distros and no one has ever complained. Are any distros really
hardcoding an assumption that .tar.gz is that only possible sdist format?

(I'm agnostic on the overall question of whether to prefer .zip or .tar.gz
as the standard -- both seem reasonable with their own tradeoffs.)

-n
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposed new Distutils API for compiler flag detection (Issue26689)

[Ralf Gommers]

On Mon, Aug 22, 2016 at 6:10 AM, Donald Stufft  wrote:

> We’re reaching a point where *some* projects are announcing the end of
> Python 2 support as of a certain date, but let us not forget that Python
> 2.7 is still an order of magnitude more than any other version of Python in
> terms of downloads from PyPI.
>

Even in 5 years from now, when almost all projects have dropped support for
Python 2.7, the reasoning remains the same. Projects will then support 3 or
4 Python 3.x versions, so still any new API added to distutils cannot be
used by those projects for 3-4 years. It does not make much sense to add
new things to distutils, with the exception maybe of something needed
particularly for a new Python version (like MSVC 2015 support in Python
3.5).

On top of that there are technical reasons (don't want to test combinations
of python+setuptools that both change per release) and organizational ones
(distutils maintenance is terrible, many simple bugfix patches don't get
merged for ages, setuptools at least fixes regressions quite fast).

I'm not sure if there's an official policy on adding new things to
distutils, but if not then this request is a good time to make one.
Assuming of course that the setuptools devs are willing to merge features
like the one from Sylvain.

Ralf



>
> On Aug 21, 2016, at 2:08 PM, Sylvain Corlay 
> wrote:
>
> Although we are reaching a tipping point where a lot of projects are
> announcing the end of Python 2 support as of a certain date.
>
> Whatever is in the latest version of Python 3 when it will be considered a
> sane decision to have a Python 3-only library will be considered standard.
>
> On Sun, Aug 21, 2016 at 5:28 PM, Donald Stufft  wrote:
>
>>
>> On Aug 21, 2016, at 5:18 AM, Sylvain Corlay 
>> wrote:
>>
>> With this reasoning, nothing should ever be added to the standard library.
>>
>>
>>
>> Packaging is a bit different than other things because the network effect
>> is much more prominent. There’s no real way to say, install a backport if
>> you need one, you just have to kind of wait until every has upgraded which
>> is unlikely other bits of the standard library. In addition, people writing
>> projects in Python that are designed to be distributed, they tend to need
>> to work across many versions of Python, while someone writing a project for
>> themselves only need to worry about whatever version of Python they are
>> deploying to. So while the new statistics module is, even without a
>> backport, immediately useful to people developing their own projects for a
>> recent version of Python, something in distutils is not useful for package
>> authors until it is the *minimum* version of Python they support.
>>
>> This generally makes the reward for changing distutils very small,
>> particularly with the 3.x split because very few authors are willing to
>> drop 2.7 much less go straight to 3.6 (or whatever) and for people making
>> their own, internal projects, distutils isn’t generally used a whole lot
>> there either.
>>
>> —
>> Donald Stufft
>>
>>
>>
>>
>
>
> —
> Donald Stufft
>
>
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Robert Collins]

On 22 Aug 2016 4:18 AM, "Nick Coghlan"  wrote:
>
>
> Once we have a universal default, unilaterally changing that default
> gets easier, rather than harder - the main precedent being set here is
> that we *don't want* the default format to be platform dependent, we
> want there to be just one default.

Sorry for the separate reply, phone typing.

Monocultures are less flexible, so I'm assuming once we have Just One
format that bit rot in tooling will creep in and it will be harder and
harder to change, not easier.

-Rob
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Robert Collins]

On 22 Aug 2016 4:18 AM, "Nick Coghlan"  wrote:
>
> On 21 August 2016 at 18:21, Robert Collins 
wrote:
> > tl;dr: I think standardising on .tar.gz would be a rather shortsighted
> > thing to do, given how many Windows users Python has and how much of a
> > different supporting .zip makes for workflow on that platform - with
> > no negative impacts on any other platform.
>
> Once we have a universal default, unilaterally changing that default
> gets easier, rather than harder - the main precedent being set here is
> that we *don't want* the default format to be platform dependent, we
> want there to be just one default.
>
> My core assumption in recommending starting with tar.gz as that
> universal default is that Windows users will mostly be interacting
> with sdists through tooling that is closely linked to or at least
> originated in the Python ecosystem (easy_install, pip, PyPM, conda,
> Enthought Canopy, buildout, Christoph Gohlke's Windows installers),
> and that double-clicking a Python sdist in an Explorer window is not
> something most folks are in the habit of doing.

It is certainly wrong for me, when I'm using Windows, and when I've
shoulder surfed others also. E.g. at pycon sprints.

> By contrast, I *know* Linux distros are in the habit of pulling
> release tarballs from PyPI and feeding them directly into their
> release pipelines, so the potential for unanticipated breakage seems
> much higher there, and more likely to fall on community distros rather
> than on commercial Python redistributors (simply because there are
> more volunteers working on Linux based tooling than there are on
> WIndows developer tools).

Those pipelines are mature, poly format capable and centralized, containing
the damage.

-Rob
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposed new Distutils API for compiler flag detection (Issue26689)

[Donald Stufft]

We’re reaching a point where *some* projects are announcing the end of Python 2 
support as of a certain date, but let us not forget that Python 2.7 is still an 
order of magnitude more than any other version of Python in terms of downloads 
from PyPI.

> On Aug 21, 2016, at 2:08 PM, Sylvain Corlay  wrote:
> 
> Although we are reaching a tipping point where a lot of projects are 
> announcing the end of Python 2 support as of a certain date.
> 
> Whatever is in the latest version of Python 3 when it will be considered a 
> sane decision to have a Python 3-only library will be considered standard.
> 
> On Sun, Aug 21, 2016 at 5:28 PM, Donald Stufft  > wrote:
> 
>> On Aug 21, 2016, at 5:18 AM, Sylvain Corlay > > wrote:
>> 
>> With this reasoning, nothing should ever be added to the standard library.
> 
> 
> Packaging is a bit different than other things because the network effect is 
> much more prominent. There’s no real way to say, install a backport if you 
> need one, you just have to kind of wait until every has upgraded which is 
> unlikely other bits of the standard library. In addition, people writing 
> projects in Python that are designed to be distributed, they tend to need to 
> work across many versions of Python, while someone writing a project for 
> themselves only need to worry about whatever version of Python they are 
> deploying to. So while the new statistics module is, even without a backport, 
> immediately useful to people developing their own projects for a recent 
> version of Python, something in distutils is not useful for package authors 
> until it is the *minimum* version of Python they support.
> 
> This generally makes the reward for changing distutils very small, 
> particularly with the 3.x split because very few authors are willing to drop 
> 2.7 much less go straight to 3.6 (or whatever) and for people making their 
> own, internal projects, distutils isn’t generally used a whole lot there 
> either.
> 
> —
> Donald Stufft
> 
> 
> 
> 


—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposed new Distutils API for compiler flag detection (Issue26689)

[Sylvain Corlay]

Although we are reaching a tipping point where a lot of projects are
announcing the end of Python 2 support as of a certain date.

Whatever is in the latest version of Python 3 when it will be considered a
sane decision to have a Python 3-only library will be considered standard.

On Sun, Aug 21, 2016 at 5:28 PM, Donald Stufft  wrote:

>
> On Aug 21, 2016, at 5:18 AM, Sylvain Corlay 
> wrote:
>
> With this reasoning, nothing should ever be added to the standard library.
>
>
>
> Packaging is a bit different than other things because the network effect
> is much more prominent. There’s no real way to say, install a backport if
> you need one, you just have to kind of wait until every has upgraded which
> is unlikely other bits of the standard library. In addition, people writing
> projects in Python that are designed to be distributed, they tend to need
> to work across many versions of Python, while someone writing a project for
> themselves only need to worry about whatever version of Python they are
> deploying to. So while the new statistics module is, even without a
> backport, immediately useful to people developing their own projects for a
> recent version of Python, something in distutils is not useful for package
> authors until it is the *minimum* version of Python they support.
>
> This generally makes the reward for changing distutils very small,
> particularly with the 3.x split because very few authors are willing to
> drop 2.7 much less go straight to 3.6 (or whatever) and for people making
> their own, internal projects, distutils isn’t generally used a whole lot
> there either.
>
> —
> Donald Stufft
>
>
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Nick Coghlan]

On 22 August 2016 at 02:38, Thomas Kluyver  wrote:
> I think shelling out is a reasonable approach for certain operations if the
> tool you're shelling out to is designed for it. When we write command-line
> programs, we often think about users typing in commands at a terminal, and
> design things that are flexible and somewhat smart. If shelling out is to be
> an API, the commands need to be predictable and dumb.

And even then providing a decent error reporting API can be quite
difficult compared to the in-Python error reporting options with full
exception support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Thomas Kluyver]

On Sun, Aug 21, 2016, at 05:19 PM, Jim Fulton wrote:
> I'm open to shelling out, but pessimistic that it will turn out well.
> I started with that approach initially with easy_install and it fell
> apart quickly.  But when we get into it... who knows?

I think shelling out is a reasonable approach for certain operations
if the tool you're shelling out to is designed for it. When we write
command-line programs, we often think about users typing in commands
at a terminal, and design things that are flexible and somewhat
smart. If shelling out is to be an API, the commands need to be
predictable and dumb.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Jim Fulton]

On Sun, Aug 21, 2016 at 12:29 PM, Donald Stufft  wrote:

>
> On Aug 21, 2016, at 12:19 PM, Jim Fulton  wrote:
>
> I'm open to shelling out, but pessimistic that it will turn out well. I
> started with that approach initially with easy_install and it fell apart
> quickly.  But when we get into it... who knows?
>
>
>
> Shelling out is currently the only exposed “API” that pip has, we’re not
> opposed to adding extra APIs though. Our current approach has been to wait
> and see for people to come out with specific use cases they have for an API
> and then work together to figure out what API we can create that satisfies
> that. Thus far we’ve accomplished this by creating new libraries that
> aren’t pip and moving functionality out of pip (and setuptools) and into
> those libraries, and then making pip/setuptools consume those. This has
> generally worked pretty well I think, as it’s easier to be careful not to
> accidentally expose some terrible internal details of pip as public API
> when it’s a new, carefully designed thing, and we can make working with
> those libraries better than it is to simply expose some part of pip. We
> generally pair this along with defining things in PEPs so that these new
> libraries don’t become the new distutils/setuptools/pip (e.g.,
> implementation defined standards) which should ideally allow anyone to
> create a from scratch implementation and have it interopt just fine.
>

Sounds reasonable. (I'd seen similar statements before, which I'd alluded
to in another message.) Thanks.

Jim

-- 
Jim Fulton
http://jimfulton.info
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Donald Stufft]

> On Aug 21, 2016, at 12:19 PM, Jim Fulton  wrote:
> 
> I'm open to shelling out, but pessimistic that it will turn out well. I 
> started with that approach initially with easy_install and it fell apart 
> quickly.  But when we get into it... who knows?


Shelling out is currently the only exposed “API” that pip has, we’re not 
opposed to adding extra APIs though. Our current approach has been to wait and 
see for people to come out with specific use cases they have for an API and 
then work together to figure out what API we can create that satisfies that. 
Thus far we’ve accomplished this by creating new libraries that aren’t pip and 
moving functionality out of pip (and setuptools) and into those libraries, and 
then making pip/setuptools consume those. This has generally worked pretty well 
I think, as it’s easier to be careful not to accidentally expose some terrible 
internal details of pip as public API when it’s a new, carefully designed 
thing, and we can make working with those libraries better than it is to simply 
expose some part of pip. We generally pair this along with defining things in 
PEPs so that these new libraries don’t become the new distutils/setuptools/pip 
(e.g., implementation defined standards) which should ideally allow anyone to 
create a from scratch implementation and have it interopt just fine.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Jim Fulton]

On Sat, Aug 20, 2016 at 6:31 PM, Wes Turner  wrote:
...

> I may be optimistic, but should adding buildout support for wheel be more
> complicated than shelling out to pip with the correct, uhm, --prefix etc?
>

I'm open to shelling out, but pessimistic that it will turn out well. I
started with that approach initially with easy_install and it fell apart
quickly.  But when we get into it... who knows?

Jim

-- 
Jim Fulton
http://jimfulton.info
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Jim Fulton]

On Sat, Aug 20, 2016 at 11:07 PM, Nick Coghlan  wrote:

> On 21 August 2016 at 05:46, Jim Fulton  wrote:
> > On Sat, Aug 20, 2016 at 3:02 PM, Nick Coghlan 
> wrote:
> >> > I have the impression that uninstalling things can be
> >> > problematic, but maybe that's been fixed.
> >>
> >> Uninstallation is fine, as we *do* have a full file manifest after a
> >> component has been installed.
> >
> > So, if package A and B install the same file, X, and then A is
> uninstalled,
> > is X still there?  If then B is uninstalled, does X go away?  Does this
> > machinery depend on whether X has the the same contents in A and B?
>
> If A and B install to the same path, B will overwrite A's version, and
> if either is uninstalled, they'll delete whichever version is
> currently there. Since "the same path" in this case usually refers to
> either a script or a Python module, the unhandled conflict is really
> at install time - we'd want something closer to the RPM/deb style "A
> file already exists at that destination, so we're not installing the
> requested package at all", rather than a Windows-style reference
> counting system.
>
> As Daniel notes, this kind of check is actually already possible when
> installing from a wheel file today, it just requires someone with the
> roundtuits to add it to pip (perhaps via a new vendorable package for
> working with Python installation metadata and detecting conflicts
> between what's already installed and a wheel being considered for
> installation).
>

The most common source of conflicts, I expect, are namespace package
__init__.py files. You wouldn't want to reject installing a package with a
conflicting __init__.py file in a namespace package.  I realize that PEP420
and the demise of Python 2 should make this case go away, someday, although
PEP420 doesn't prohibit __init__.py files in namespace packages.

OK, I'm done beating this horse. It was dead a while ago. :)

Jim

-- 
Jim Fulton
http://jimfulton.info
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposed new Distutils API for compiler flag detection (Issue26689)

[Donald Stufft]

> On Aug 21, 2016, at 5:18 AM, Sylvain Corlay  wrote:
> 
> With this reasoning, nothing should ever be added to the standard library.


Packaging is a bit different than other things because the network effect is 
much more prominent. There’s no real way to say, install a backport if you need 
one, you just have to kind of wait until every has upgraded which is unlikely 
other bits of the standard library. In addition, people writing projects in 
Python that are designed to be distributed, they tend to need to work across 
many versions of Python, while someone writing a project for themselves only 
need to worry about whatever version of Python they are deploying to. So while 
the new statistics module is, even without a backport, immediately useful to 
people developing their own projects for a recent version of Python, something 
in distutils is not useful for package authors until it is the *minimum* 
version of Python they support.

This generally makes the reward for changing distutils very small, particularly 
with the 3.x split because very few authors are willing to drop 2.7 much less 
go straight to 3.6 (or whatever) and for people making their own, internal 
projects, distutils isn’t generally used a whole lot there either.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Daniel Holth]

Targz is about 3/4 the size of zip for a bag of Python dists I tested. Zip
inside a second zip would provide the same compression. No word on the
Weissman score.

Tar isn't exactly a single format. For Unicode try POSIX-1.2001 aka pax
format tar. Python defaults to gnutar. Zip has good Unicode support.

Exactly how much simpler is exactly one file format?

On Sun, Aug 21, 2016, 06:57 Paul Moore  wrote:

> On 21 August 2016 at 09:21, Robert Collins 
> wrote:
> >
> > tl;dr: I think standardising on .tar.gz would be a rather shortsighted
> > thing to do, given how many Windows users Python has and how much of a
> > different supporting .zip makes for workflow on that platform - with
> > no negative impacts on any other platform.
>
> One thing that has (IIRC) come up in a pip bug in the past - how do
> tar and zip format fare in terms of Unicode support? IIRC, older
> versions of (I think) tar format don't include an encoding, nor do
> they mandate UTF-8, so they have the potential to break when used
> cross-platform. Sorry, I can't recall exact details.
>
> I think it's important that whatever format we mandate works with full
> Unicode filenames, and the available user tools support that.
>
> Paul
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Nathaniel Smith]

On Sun, Aug 21, 2016 at 3:51 AM, Paul Moore  wrote:
> On 20 August 2016 at 22:31, Nathaniel Smith  wrote:
>> Wheels are a pretty simple and straightforward format. They've got some
>> metadata, and then the are a set of directories with labels attached: "this
>> directory needs to go on sys.path", "this directory has scripts that should
>> have shebang fixups applied and then go on $PATH", etc. (The spec is not
>> long: https://www.python.org/dev/peps/pep-0427/)
>>
>> My first guess at how buildout could handle wheels elegantly would be:
>>
>> - we have one piece of code that abstracts the operation of "please give me
>> a wheel for package X version Y that works on this system".
>
> This can be abstracted at the moment as "pip wheel ".
> That will deliver a wheel to the current directory which is exactly
> what is described above.

Or possibly "pip wheel --no-deps " or something similar.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Paul Moore]

On 21 August 2016 at 09:21, Robert Collins  wrote:
>
> tl;dr: I think standardising on .tar.gz would be a rather shortsighted
> thing to do, given how many Windows users Python has and how much of a
> different supporting .zip makes for workflow on that platform - with
> no negative impacts on any other platform.

One thing that has (IIRC) come up in a pip bug in the past - how do
tar and zip format fare in terms of Unicode support? IIRC, older
versions of (I think) tar format don't include an encoding, nor do
they mandate UTF-8, so they have the potential to break when used
cross-platform. Sorry, I can't recall exact details.

I think it's important that whatever format we mandate works with full
Unicode filenames, and the available user tools support that.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Paul Moore]

On 20 August 2016 at 22:31, Nathaniel Smith  wrote:
> Wheels are a pretty simple and straightforward format. They've got some
> metadata, and then the are a set of directories with labels attached: "this
> directory needs to go on sys.path", "this directory has scripts that should
> have shebang fixups applied and then go on $PATH", etc. (The spec is not
> long: https://www.python.org/dev/peps/pep-0427/)
>
> My first guess at how buildout could handle wheels elegantly would be:
>
> - we have one piece of code that abstracts the operation of "please give me
> a wheel for package X version Y that works on this system".

This can be abstracted at the moment as "pip wheel ".
That will deliver a wheel to the current directory which is exactly
what is described above. It probably doesn't meet buildout's
repeatability requirements (as it could invoke a build step if there's
no published wheel, and build steps can do anything they like -
--only-binary would be a way to avoid that, at the cost of failing if
a project didn't publish a wheel) but it would likely work as a
prototype.

I don't think there's anything inherent in buildout's isolation model
of using sys.path entries for each dependency, that is incompatible
with pip/wheels. If buildout unpacks eggs, it can do the same with
wheels. If it uses zipped eggs, then it can *probably* do the same
with wheels, although there's no "zip safe" mechanism to declare a
wheel as safe for this use.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposed new Distutils API for compiler flag detection (Issue26689)

[Sylvain Corlay]

On Sun, Aug 21, 2016 at 8:40 AM, Nathaniel Smith  wrote:

> > Having the `has_flag` in a different location from `has_function` would
> be weird in my opinion.
>
> I think the point though is that in your proposal, has_flag is in
> distutils 3.6, but has_function is in distutils 2.6, 2.7, 3.2, 3.3, 3.4,
> 3.5, 3.6. So they're still in different places in practice, and if your new
> feature is only in distutils 3.6 then almost no one will be able to use it
> within the next 3+ years. Otoh putting it in setuptools makes it uniformly
> usable for everyone immediately.
>
> -n
>

With this reasoning, nothing should ever be added to the standard library.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Deprecating little used file types/extensions on PyPI?

[Robert Collins]

On 21 August 2016 at 15:22, Nick Coghlan  wrote:
> On 21 August 2016 at 08:40, Chris Barker - NOAA Federal
>  wrote:
>>>
>>> If we're going through all this trouble, isn't it better just to jump to 
>>> .zip files like every other distribution format in existence?
>>
>> Yes. :-)
>
> zip is popular for *user facing* distribution formats, where the file
> format is exposed directly to end users, and end user applications
> (e.g. zipimport, ODF).
>
> That's not where tarballs shine, so it's not where they show up: tar
> is a backend infrastructure format, used for tool-to-tool
> communication, rather than user-to-user.
>
> The trajectory of distutils-sig for the past few years has been
> towards a world where sdists are primarily a tool-to-tool format for
> data interchange between publishing tools (e.g. twine), installation
> tools (e.g. pip) and redistribution tools (e.g. conda, RPM, deb,
> buildout), with wheels (and potentially eggs, or an equally zipimport
> friendly future derivative) being the preferred formats for working
> directly with Python software in a destination environment (whether
> that's a system integrator's build farm, a production software
> deployment, or a Python user's local system).

Jar's are zip's (~= wheels). *source* jars are also zips. (~= sdists).
I'm struggling to think of any two ways that .tar.gz is better than
.zip. The compression is ~=. .tar.lzma and friends can be radically
smaller whereas .zip has per-element compression, so thats one way,
but .tar.gz has no other benefits (it has other differences, but none
are relevant in the context of the sdist use cases AFAICT).

As far as the switching consts Donald raises, here's my take:
 - we're going to break a lot of peoples workflow no matter what: 10%
of a lot is a lot.
 - all setuptools in the while /can/ produce .zip's AIUI, so the
issues w.r.t. the long tail of Linux distros is entirely
uninteresting: the folk depending on the default behaviour will need
to be explict - but thats going to be the case for *everyone* [because
setuptools on windows is inconsistent with setuptools on Linux]: we
are basically telling everyone they have to hardcode to .zip until the
long tail of existing installs *everywhere* has gone away.
 - unzip and friends are as available on Linux distros as tar is.

tl;dr: I think standardising on .tar.gz would be a rather shortsighted
thing to do, given how many Windows users Python has and how much of a
different supporting .zip makes for workflow on that platform - with
no negative impacts on any other platform.

-Rob
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposed new Distutils API for compiler flag detection (Issue26689)

[Nathaniel Smith]

On Aug 20, 2016 3:09 PM, "Sylvain Corlay"  wrote:
>
> Hi Ralf,
>
> On Sat, Aug 20, 2016 at 11:42 PM, Ralf Gommers 
wrote:
>>>
>>>
>>> On python-dev and in the bug tracker, Sylvain has understandably asked
for a review with an eye to adding this new feature to Python 3.6 whose
feature code cutoff is scheduled for a few weeks from now.  As release
manager, I am not opposed in general to adding new features to Distutils
but I think we should be very cautious about modifying or adding new
Distutils APIs, given that many third-party distribution authors want to
support their packages on multiple versions.  So I want to make sure that
there is some agreement that adding this new API starting with 3.6 is a
good thing to do rather than having it go in under the radar.
>>
>>
>> I'd rather see that kind of thing added to setuptools. We're already
having to deal with setuptools as a moving target, so if distutils becomes
one again as well that means more testing with combinations of different
Python and setuptools versions. Imho distutils changes should be bugfix and
essential maintenance only.
>>
>
>
> Having the `has_flag` in a different location from `has_function` would
be weird in my opinion.

I think the point though is that in your proposal, has_flag is in distutils
3.6, but has_function is in distutils 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6. So
they're still in different places in practice, and if your new feature is
only in distutils 3.6 then almost no one will be able to use it within the
next 3+ years. Otoh putting it in setuptools makes it uniformly usable for
everyone immediately.

-n
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig