Re: [Distutils] Conditionless setup.py

[David Mertz]

Me too. YAML is *so much* more widely used, and the complicated edge cases
can simply be ignored for this requirement.

Maybe it's just because I've never heard of whatever improper behavior the
author engaged in, but I don't think a data format needs to answer to the
actions of its creator(s). If G. Klyne or C. Newman were to do something
dreadful I wouldn't want dates to stop following ISO-8601.[*]

[*] I don't even know the 8601 authors' first names, and assume they are
good and honorable people. Just making an analogy.

On Aug 25, 2017 5:46 PM, "xoviat"  wrote:

> I personally do not understand the aversion to YAML. I mean yes, the
> specification is more complicated, but it's also more popular and the YAML
> files will not be complex enough for a C library to help that much. And
> since it's more popular, people might even prefer specifying package
> metadata in a pyproject.yaml. pip could even cache a wheel of the pyyaml
> package between builds that could be imported at build time with a
> zipimporter rather than vendoring the package. And as a plus it's not named
> after an alleged sexist.
>
> Honestly this is not an issue that interests me very much but this rant is
> because I was surprised that toml was chosen when I first found out about
> it.
>
> 2017-08-25 18:16 GMT-05:00 Nathaniel Smith :
>
>> On Fri, Aug 25, 2017 at 1:00 PM, Jeremy Stanley 
>> wrote:
>> > (The
>> > community around it is sensitive to gender diversity issues and
>> > wants to avoid acquiring more of a "brogrammer" image, so some of us
>> > worry that any conspicuous TOML files checked into revision control
>> > repositories could be seen as a tacit endorsement of the author's
>> > alleged behavior at GH a few years ago.)
>>
>> I was one of the folks championing TOML during the original
>> discussions, and this is an issue that also worried me a lot. In case
>> it's a useful data point: I actually contacted several of the main
>> rust/cargo developers, since they were the major users of TOML and are
>> also well known to be sensitive to these issues, to ask if they've had
>> any issues with this, and they said that they haven't heard any
>> complaints.
>>
>> Obviously there's a difference between "no-one complained" and "no-one
>> was bothered", and I suspect the community's existing reputation may
>> affect how this is interpreted as well, but... maybe useful as a data
>> point.
>>
>> Between this and the way the TOML spec appears to have been abandoned
>> at v0.4 (with the admonition "you should assume that is is unstable
>> and act accordingly") I've wondered if we should fork it, rename it
>> "the obvious minimal language", and release our own 1.0.
>>
>> -n
>>
>> --
>> Nathaniel J. Smith -- https://vorpus.org
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] When can we kill Python 2.6 support?

[David Mertz]

Kill it with fire!

On Sep 2, 2016 2:06 PM, "Donald Stufft"  wrote:

> The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort of
> been
> where they've been at for a while now. I would like to think about what we
> need
> to be to start considering Python 2.6 as "too old" to support. In pip we
> generally follow a usage based deprecation/removal of supported Pythons
> but we
> don't have any real guidelines for when something is at a low enough usage
> to
> consider it no longer supported and we instead just sort of wait until
> someone
> makes a case that it's "low enough".
>
> This issue tends to impact more than just pip, because once pip drops
> support
> for something people tend to start dropping it across the entire ecosystem
> and
> use pip's no longer supporting it as justification for doing so.
>
> I would like to take a look at Python 2.6 and try and figure out if we're
> at a
> point that we can deprecate and drop it, and if not what is such a point.
>
> Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8)
> for
> downloading from PyPI I see the usage is ~3% of downloads are via Python
> 2.6.
> The only thing lower than Python 2.6 that is still supported is Python 3.3.
>
> Python 2.6 itself has been EOL since 2013-10-29 which is now just about 3
> years
> ago. It's SSL module is not generally secure and requires the use of
> additional
> installed modules to get it to be so. I believe the only place to get a
> Python 2.6 that is "supported" is through the Enterprise-y Linux
> Distributions
> like RHEL/CentOS/etc.
>
> Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3
> years
> is enough to start deprecating and dropping 2.6? If not what sort of
> threshold
> do we think is enough? It'd be nice to get the albatross of Python 2.6
> support
> off from around our necks but I'm not sure how others feel. Obviously all
> of
> the existing versions of all of the tooling will still be fully functional
> so
> Python 2.6 users will simply need to not upgrade their tooling to continue
> to
> work, *but* it also means that they will be left out of new packaging
> features
> (and likewise, people can't rely on them if they still wish to support
> 2.6).
>
> Thoughts?
>
> —
> Donald Stufft
>
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[David Mertz]

I found I really did typically have that problem that Paul describes pretty
often until I switched to using predominantly conda.  I would always make
symlinks for pip2 and pip3 (and maybe for pip3.3 vs. pip3.4) to make sure
things went the right places.

I suppose this problem was largely because I didn't really use virtualenv
much, and now that I'm a conda person the environments come "for free" with
the installation.

On Sat, Nov 7, 2015 at 3:41 PM, Paul Moore <p.f.mo...@gmail.com> wrote:

> On 7 November 2015 at 22:21, Antoine Pitrou <solip...@pitrou.net> wrote:
> > The actual question is: which problem are you trying to solve *that
> > current users are actually experiencing*?
>
> Typically, people using "pip" to install stuff, and finding it gets
> installed into the "wrong" Python installation (i.e., not the one they
> expected). I'm not clear myself on how this happens, but it seems to
> be common on some Linux distros (and I think on OSX as well) where
> system and user-installed Pythons get confused.
>
> Whether removing the pip command in favour of explicitly using the
> name of the python you want to install into is a reasonable solution,
> or an over-reaction, is what we're trying to establish. But it is a
> very real problem and we see a fair number of bug reports based on it.
>
> Paul
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
*David Mertz, Ph.D.*
*Senior Software Engineer and Senior Trainer*
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[David Mertz]

On Nov 7, 2015 7:30 PM, "Nathaniel Smith"  wrote:
> alternative approach would be to totally decouple the host python used
> to execute pip from the target python that pip acts upon, on the
> grounds that these are logically distinct things. (As a thought
> experiment you can even imagine a package manager called, say, 'cpip',
> which is a standalone program written in C or some other
> non-Python-language, but that happens to know how to manipulate Python
> environments.

This is a great idea! It exists, and it is spelled "conda". Well, it's
written in some particular version of Python, but the package and
environment management is completely decoupled. You can in install R
packages or Julia packages or different Python versions.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Making pip and PyPI work with conda packages

[David Mertz]

On Tue, May 19, 2015 at 4:11 PM, Chris Barker chris.bar...@noaa.gov wrote:

 On Tue, May 19, 2015 at 3:09 PM, Paul Moore p.f.mo...@gmail.com wrote:
 conda -- a fully open source package management system
 Anaconda -- a python and other stuff distribution produced by
Continuum.

 How Continuum does or doesn't publish the recipes it used to build
 Anaconda doesn't really have anything to do with conda-the-technology.

True.  Also though, in answer to the a question here, I asked a Continuum
colleague on the conda team.  It seems that Anaconda was built using a
proprietary system before conda-build and conda-recipes was opened, so not
all recipes have made it over to the Free side of the fence yet.  But
y'know, gh:conda-build *is*  a public repository, anyone could add more.


 I will note that most recipes seem to consist of either 'python setup.py
install' or './configure; make; make install'.


 sure -- but those aren't the ones we want ;-)

Understood.


 see if you can find the wxPython one, while you are at it :-)
   --  though I suspect that was built from the official executable,
rather than re-built from scratch.

In the case of pyyaml, this is actually what's behind the wall


 #!/bin/bash
 patch -p0 EOF
 --- setup.cfg~ 2011-05-29 22:31:18.0 -0500
 +++ setup.cfg 2012-07-10 20:33:50.0 -0500
 @@ -4,10 +4,10 @@
 [build_ext]
 # List of directories to search for 'yaml.h' (separated by ':').
 -#include_dirs=/usr/local/include:../../include
 +include_dirs=$PREFIX/include
 # List of directories to search for 'libyaml.a' (separated by ':').
 -#library_dirs=/usr/local/lib:../../lib
 +library_dirs=$PREFIX/lib
 # An alternative compiler to build the extention.
 #compiler=mingw32
 EOF
 $PYTHON setup.py install

It's not *only* the 'setup.py install', but it's not *that* much mystery
either.  wxPython I can't seem to find, not sure what I'm missing.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Making pip and PyPI work with conda packages

[David Mertz]

It is certainly not our intention at Continuum to keep build recipes
private.  I have just come on board at the company, but I'll add it to my
TODO list to work on making sure that those are better updated and
maintained at https://github.com/conda/conda-recipes.

On Tue, May 19, 2015 at 3:09 PM, Paul Moore p.f.mo...@gmail.com wrote:

 On 19 May 2015 at 22:29, Chris Barker chris.bar...@noaa.gov wrote:
  As far as I can tell, Continuum does not publish the build scripts used
 to
  build all the stuff in Anaconda.

 So, for example the process for building the pyyaml package available
 via conda is private? (I want to say proprietary, but there's a lot
 of implications in that term that I don't intend...) That seems like a
 rather striking downside to conda that I wasn't aware of. Hopefully,
 I'm misunderstanding something :-)

 Paul
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 https://mail.python.org/mailman/listinfo/distutils-sig




-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Making pip and PyPI work with conda packages

[David Mertz]

I will note that most recipes seem to consist of either 'python setup.py
install' or './configure; make; make install'.  So there is quite likely
actually little significant work that has failed to have been published.
But I'm not sure of pyyaml off the top of my head, and how that is built.

On Tue, May 19, 2015 at 3:23 PM, David Mertz dme...@continuum.io wrote:

 It is certainly not our intention at Continuum to keep build recipes
 private.  I have just come on board at the company, but I'll add it to my
 TODO list to work on making sure that those are better updated and
 maintained at https://github.com/conda/conda-recipes.

 On Tue, May 19, 2015 at 3:09 PM, Paul Moore p.f.mo...@gmail.com wrote:

 On 19 May 2015 at 22:29, Chris Barker chris.bar...@noaa.gov wrote:
  As far as I can tell, Continuum does not publish the build scripts used
 to
  build all the stuff in Anaconda.

 So, for example the process for building the pyyaml package available
 via conda is private? (I want to say proprietary, but there's a lot
 of implications in that term that I don't intend...) That seems like a
 rather striking downside to conda that I wasn't aware of. Hopefully,
 I'm misunderstanding something :-)

 Paul
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 https://mail.python.org/mailman/listinfo/distutils-sig




 --
 The dead increasingly dominate and strangle both the living and the
 not-yet born.  Vampiric capital and undead corporate persons abuse
 the lives and control the thoughts of homo faber. Ideas, once born,
 become abortifacients against new conceptions.




-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Dynamic linking between Python modules (was: Beyond wheels 1.0: helping downstream, FHS and more)

[David Mertz]

This pertains more to the other thread I started, but I'm sort of becoming
convinced--especially by Paul Moore's suggestion there--that the better
approach is to grow conda (the tool) rather than shoehorn conda packages
into pip.  Getting pip to recognize the archive format of conda would be
easy enough alone, but that really doesn't cover the fact that 'conda ~=
pip+virtualenv', and pip alone simply should not try to grow that latter
aspect itself.  Plus pip is not going to be fully language agnostic, for
various reasons, but including the fact that apt-get and yum and homebrew
and ports already exist.

So it might make sense to actually allow folks to push conda to budding web
developers, if conda allowed installation (and environment management) of
sdist packages on PyPI.  So perhaps it would be good if *this* worked:

  % pip install conda
  % conda install scientific_stuff
  % conda install --sdist django_widget   # we know to look on PyPI

Maybe that flag is mis-named, or could be omitted altogether.  But there's
no conceptual reason that conda couldn't build an sdist fetched from PyPI
into a platform specific binary matching the current user machine (and do
all the metadata dependency and environment stuff the conda tool does).

On Mon, May 18, 2015 at 3:17 AM, Paul Moore p.f.mo...@gmail.com wrote:

 On 17 May 2015 at 23:50, Chris Barker chris.bar...@noaa.gov wrote:
  I guess the key thing here for me is that I don't see pushing conda to
  budding web developers -- but what if web developers have the need for a
 bit
  of the scipy stack? or???
 
  We really don't have a good solution for those folks.

 Agreed. My personal use case is as a general programmer (mostly
 sysadmin and automation type of work) with some strong interest in
 business data analysis and a side interest in stats.

 For that sort of scenario, some of the scipy stack (specifically
 matplotlib and pandas and their dependencies) is really useful. But
 conda is *not* what I'd use for day to day work, so being able to
 install via pip is important to me. It should be noted that installing
 via pip *is* possible - via some of the relevant projects having
 published wheels, and the rest being available via Christoph Gohlke's
 site either as wheels or as wininsts that I can convert. But that's
 not a seamless process, so it's not something I'd be too happy
 explaining to a colleague should I want to share the workload for that
 type of thing.

 Paul
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 https://mail.python.org/mailman/listinfo/distutils-sig




-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Dynamic linking between Python modules (was: Beyond wheels 1.0: helping downstream, FHS and more)

[David Mertz]

I don't really see any reason conda couldn't support bdist wheels also.
But yes, basically the idea is that we'd like users to be able to rely
entirely on conda as their packaging (and environment configuration) system
if they choose to.  It may be impolitic to say so, but I think conda can
and should replace pip for a large class of users.  That is, it should be
possible for users to use pip exactly once (as in the line I show above),
and use conda forever thereafter.

Since conda does a lot more (programming language independence,
environments), perhaps it really does make a lot more sense for conda to be
one package manager to rule them all much more than trying to make a pip
that does so.

But y'know, the truth is I'm trying to figure out the best path here.  I
want to get better interoperability between conda packages and the rest of
the Python ecosystem, but there are stakeholders involved both in the
distutils community and within Continuum (where I now work).

On Mon, May 18, 2015 at 11:21 AM, Paul Moore p.f.mo...@gmail.com wrote:

 On 18 May 2015 at 18:50, David Mertz dme...@continuum.io wrote:
% pip install conda
% conda install scientific_stuff
% conda install --sdist django_widget   # we know to look on PyPI

 But that doesn't give (Windows, mainly) users a solution for things
 that need a C compiler, but aren't provided as conda packages.

 My honest view is that unless conda is intending to replace pip and
 wheel totally, you cannot assume that people will be happy to use
 conda alongside pip (or indeed, use any pair of independent packaging
 tools together - people typically want one unified solution). And if
 the scientific community stops working towards providing wheels for
 people without compilers because you can use conda, there is going
 to be a proportion of the Python community that will lose out on some
 great tools as a result.

 Paul




-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Making pip and PyPI work with conda packages

[David Mertz]

I've just started monitoring this SIG to get a sense of the issues and
status of things.  I've also just started working for Continuum Analytics.

Continuum has a great desire to make 'pip' work with conda packages.
Obviously, we love for users to choose the Anaconda Python distribution but
many will not for a variety of reasons (many good reasons).

However, we would like for users of other distros still to be able to
benefit from our creation of binary packages for many platforms in the
conda format.  As has been discussed in recent threads on dependency
solving, the way conda provides metadata apart from entire packages makes
much of that work easier.  But even aside from that, there are simply a
large number of well-tested packages (not only for Python, it is true, so
that's possibly a wrinkle in the task) we have generated in conda format.

It is true that right now, a user can in principle type:

  % pip install conda
  % conda install some_conda_package

But that creates two separate systems for tracking what's installed and
what dependencies are resolved; and many users will not want to convert
completely to conda after that step.

What would be better as a user experience would be to let users do this:

  % pip install --upgrade pip
  % pip install some_conda_package

Whether that second command ultimately downloads code from pyip.python.org
or from repo.continuum.io is probably less important for a user experience
perspective.  Continuum is very happy to upload all of our conda packages
to PyPI if this would improve this user experience.  Obviously, the idea
here would be that the user would be able to type 'pip list' and friends
afterward, and have knowledge of what was installed, even as conda packages.

I'm hoping members of the SIG can help me understand both the technical and
social obstacles that need to be overcome before this can happen.

Yours, David...
-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig