Re: [Distutils] The Python Packaging Ecosystem (of Nick) Support for other programming languages

2017-04-06 Thread Nathaniel Smith 
On Thu, Apr 6, 2017 at 10:26 PM, Thomas Güttler
 wrote:
> Am 06.04.2017 um 17:22 schrieb Nathaniel Smith:
>> On Apr 6, 2017 7:32 AM, "Thomas Güttler" > > wrote:
>>
>> Dear Nick and other distutils listeners,
>>
>> Nick wrote this about seven months ago:
>>
>> 
>> http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html
>>  
>> 
>>
>> I love Python and I use it daily.
>>
>> On the other hand there are other interesting programming languages out 
>> there.
>>
>> Why not do "thinking in sets" here and see python just as one item in 
>> the list of a languages?
>>
>> Let's dream: All languages should be supported in the ever best 
>> packaging solution of the future.
>>
>> What do you think?
>>
>>
>> This is basically what conda attempts to do. It's nice in some ways, but 
>> does also have limitations.
>>
>> In any case this isn't a very useful thing to post here?
>
> For me it was very useful. I was not aware of the paypal post about python 
> packaging. Yes, it was useful.

My point was it's wasting the time of the many many people who read
this list, who are trying to move Python packaging forward for the
millions of people who use Python. Justifying that by saying your
message was useful to *you* is... stunningly self-centered. Did your
post make *Python* better? If not, maybe think twice next time before
posting?

>> Distutils and pip and pypi aren't going anywhere,
>
> That's new to me. I guess I misunderstood what you said (I am not a native 
> speaker). I understood "There is no progress, and won't be in the future". 
> That's new to me.
> I saw several new pip versions in the past. I thought there was progress.

My apologies for using an unclear idiom. My sentence means: "they are
not going to disappear, or be replaced by something radically
different".

>>.. and the folks here are all at their limit trying to keep them from falling 
>>over, so taking up time with these super vague blue sky ideas is a bit rude.
>
> What is the problem if "falling over" happens?
>
> Is there no easier solution then going at its limit?
>
> Why is having blue sky ideas rude? AFAIK the word "rude" means "offensively 
> impolite or bad-mannered."
>
> I personally like the tongue spoken at the linux-kernel mailing list (or 
> systemd). Yes, the people there are impolite and bad-mannered.
> People speak you what they think and feel. Why not? I think being polite in 
> tech related discussions slows down progress.
>
> Look at my signature. Tell me what's wrong: Hit me with arguments.

If you don't care how your words effect others then I'm not really
interested in talking to you, except to urge you to reconsider that.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The Python Packaging Ecosystem (of Nick) Support for other programming languages

2017-04-06 Thread Thomas Güttler 
Am 06.04.2017 um 17:22 schrieb Nathaniel Smith:
> On Apr 6, 2017 7:32 AM, "Thomas Güttler"  > wrote:
> 
> Dear Nick and other distutils listeners,
> 
> Nick wrote this about seven months ago:
> 
> 
> http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html
>  
> 
> 
> I love Python and I use it daily.
> 
> On the other hand there are other interesting programming languages out 
> there.
> 
> Why not do "thinking in sets" here and see python just as one item in the 
> list of a languages?
> 
> Let's dream: All languages should be supported in the ever best packaging 
> solution of the future.
> 
> What do you think?
> 
> 
> This is basically what conda attempts to do. It's nice in some ways, but does 
> also have limitations.
> 
> In any case this isn't​ a very useful thing to post here? 

For me it was very useful. I was not aware of the paypal post about python 
packaging. Yes, it was useful.

> Distutils and pip and pypi aren't going anywhere,

That's new to me. I guess I misunderstood what you said (I am not a native 
speaker). I understood "There is no progress, and won't be in the future". 
That's new to me.
I saw several new pip versions in the past. I thought there was progress.

>.. and the folks here are all at their limit trying to keep them from falling 
>over, so taking up time with these super vague blue sky ideas is a bit rude.

What is the problem if "falling over" happens?

Is there no easier solution then going at its limit?

Why is having blue sky ideas rude? AFAIK the word "rude" means "offensively 
impolite or bad-mannered."

I personally like the tongue spoken at the linux-kernel mailing list (or 
systemd). Yes, the people there are impolite and bad-mannered.
People speak you what they think and feel. Why not? I think being polite in 
tech related discussions slows down progress.

Look at my signature. Tell me what's wrong: Hit me with arguments.


Regards,
  Thomas Güttler


-- 
I am looking for feedback for my personal programming guidelines:
https://github.com/guettli/programming-guidelines
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

2017-04-06 Thread Wes Turner 
On Thursday, April 6, 2017, Chris Barker  wrote:

> On Wed, Apr 5, 2017 at 6:41 PM, Nick Coghlan  > wrote:
>
>
>> PayPal Engineering put together a decent write-up of their path
>> towards adopting that model last year:
>> https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/
>
>
> Thanks for that link.
>

+1


>
> We're a much smaller shop, but have had pretty much the same experience --
> also really good to see them mention miniconda at the end -- I think that's
> a much better way to go for most folks than the whole Anaconda pile.
>
> It's definitely a reasonable way to go for organisational
>> infrastructure, but even conda doesn't cover all the potential use
>> cases that are out there
>
>
> of course not -- nothing does, but I would add to the contents of that
> post:
>
> The conda-forge project is a Major boon to the conda infrastructure --
> there is now a robust way for the community to expand the available number
> of packages (and keep up more recent versions). And anyone can take
> advantage of that infrastructure for their own (selfish?) needs:
>
> Chances are, there will be a package or two that you rely on that is not
> in conda defaults (maintained by Continuum) or currently in conda-forge. So
> you can pip-install those few -- but what if they aren't on PyPi either? or
> are hard to compile and install with ugly dependencies?  You can contribute
> build recipes to conda-forge, and then have it for you, and all your users,
> and the rest of the world to access. Much better than hand maintaining
> stuff yourself.
>

Someone still needs to commit to maintaining the conda package; otherwise
who knows whether this is the latest  stable  release?


>
> My pain point now is still full multi-platform support. conda has package
> versions that are platform independent, but it can still be hard to get
> everything built  in the same version on all platforms, so it does get a
> bit ugly.
>

Docker images are reproducible and archivable:

https://github.com/ContinuumIO/docker-images
-
https://github.com/ContinuumIO/docker-images/blob/master/miniconda3/Dockerfile
-
https://github.com/ContinuumIO/docker-images/blob/master/anaconda3/Dockerfile

https://github.com/jupyter/docker-stacks
-
https://github.com/jupyter/docker-stacks/blob/master/README.md#visual-overview
-
https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile
  - a diff miniconda setup

-
https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile.ppc64le
  - ARMv would be cool too (e.g. for raspberry pi)

https://github.com/Kaggle/docker-python
- https://github.com/Kaggle/docker-python/blob/master/Dockerfile
  - everything and the kitchen sink


What platforms does conda-forge auto-build for?
- [x] x86[-64]
- [ ] linux-armv7l
  - https://github.com/conda-forge/conda-forge.github.io/issues/269
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] CentOS5 is EOL, impact on manylinux1?

2017-04-06 Thread Nathaniel Smith 
On Wed, Apr 5, 2017 at 2:37 AM, Nick Coghlan  wrote:
> On 5 April 2017 at 15:59, Nathaniel Smith  wrote:
>> On Tue, Apr 4, 2017 at 10:10 PM, Nick Coghlan  wrote:
>>> It may also be worth getting in touch with the CentOS CI folks about
>>> those aspects: https://wiki.centos.org/QaWiki/CI/GettingStarted
>>
>> That page seems to say that for CentOS 7 they only support x86_64?
>
> Sorry, I should have spelled out that chain of logic more clearly:
>
> - ppc64le machines aren't that easy to come by, especially in public
> cloud services
> - a build approach based on ppc64le VMs running on x86_64 hosts is
> likely to be easier to manage
> - that means the desired CI service feature is "run arbitrary VMs",
> rather than "bare metal ppc64le systems"
>
> You can kinda sorta do that to a limited degree in Travis, but our
> experience was that the CentOS Vagrant boxes didn't work by default,
> so we asked the CentOS CI maintainers what our options might be over
> there.
>
> And as it turns out, not only are we able to get bare metal machines
> to run our VMs on (rather than messing about with nested virt
> support), but the CentOS boxes are pre-cached on the local network, so
> they should be pretty quick to download (we haven't actually got our
> CI up and running yet, so I won't be able to vouch for that personally
> until we do).

I think bare metal access only matters for running x86 with hardware
accelerated virtualization? If we want to emulate a totally different
architecture like ppc64le then IIUC qemu does that as a regular
user-space program. You definitely can run qemu in this mode on
travis-ci, e.g. check out all the virtual architectures that rust
builds on:

   https://travis-ci.org/rust-lang/rust/

So I think travis-ci and centos-ci are equivalent on this axis – if we
want to go the qemu route, then either works, and if we don't, then
neither works?

For me the big question is whether emulation is actually a good idea.
When rust announced their plans I remember seeing some skepticism
about whether one can really trust emulated machines for this kind of
use case, though I can't find it again now... but it's definitely true
that all the major distributions go to great lengths to use real
hardware in their build farms, despite the obvious drawbacks (not just
in terms of maintenance, but also the ongoing pain of using tiny
little arm and mips machines that take dozens of hours to build
things). They know a whole lot more about this than I do so I assume
they have *some* reason :-).

It might be useful to get in touch with some of the distro's ppc64le
wranglers to get their opinion, if anyone knows any of them...

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] CentOS5 is EOL, impact on manylinux1?

2017-04-06 Thread Bruno Alexandre Rosa 
Please correct me if I'm wrong: so, that means - in theory - we can use CentOS 
CI for ppc64le machine and migrate away from Travis?

Kind Regards,
Bruno Rosa

-Original Message-
From: Nick Coghlan [mailto:ncogh...@gmail.com] 
Sent: quarta-feira, 5 de abril de 2017 06:38
To: Nathaniel Smith 
Cc: Bruno Alexandre Rosa ; distutils sig 

Subject: Re: [Distutils] CentOS5 is EOL, impact on manylinux1?

On 5 April 2017 at 15:59, Nathaniel Smith  wrote:
> On Tue, Apr 4, 2017 at 10:10 PM, Nick Coghlan  wrote:
>> It may also be worth getting in touch with the CentOS CI folks about 
>> those aspects: https://wiki.centos.org/QaWiki/CI/GettingStarted
>
> That page seems to say that for CentOS 7 they only support x86_64?

Sorry, I should have spelled out that chain of logic more clearly:

- ppc64le machines aren't that easy to come by, especially in public cloud 
services
- a build approach based on ppc64le VMs running on x86_64 hosts is likely to be 
easier to manage
- that means the desired CI service feature is "run arbitrary VMs", rather than 
"bare metal ppc64le systems"

You can kinda sorta do that to a limited degree in Travis, but our experience 
was that the CentOS Vagrant boxes didn't work by default, so we asked the 
CentOS CI maintainers what our options might be over there.

And as it turns out, not only are we able to get bare metal machines to run our 
VMs on (rather than messing about with nested virt support), but the CentOS 
boxes are pre-cached on the local network, so they should be pretty quick to 
download (we haven't actually got our CI up and running yet, so I won't be able 
to vouch for that personally until we do).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

2017-04-06 Thread Chris Barker 
On Wed, Apr 5, 2017 at 6:41 PM, Nick Coghlan  wrote:


> PayPal Engineering put together a decent write-up of their path
> towards adopting that model last year:
> https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/


Thanks for that link.

We're a much smaller shop, but have had pretty much the same experience --
also really good to see them mention miniconda at the end -- I think that's
a much better way to go for most folks than the whole Anaconda pile.

It's definitely a reasonable way to go for organisational
> infrastructure, but even conda doesn't cover all the potential use
> cases that are out there


of course not -- nothing does, but I would add to the contents of that post:

The conda-forge project is a Major boon to the conda infrastructure --
there is now a robust way for the community to expand the available number
of packages (and keep up more recent versions). And anyone can take
advantage of that infrastructure for their own (selfish?) needs:

Chances are, there will be a package or two that you rely on that is not in
conda defaults (maintained by Continuum) or currently in conda-forge. So
you can pip-install those few -- but what if they aren't on PyPi either? or
are hard to compile and install with ugly dependencies?  You can contribute
build recipes to conda-forge, and then have it for you, and all your users,
and the rest of the world to access. Much better than hand maintaining
stuff yourself.

My pain point now is still full multi-platform support. conda has package
versions that are platform independent, but it can still be hard to get
everything built  in the same version on all platforms, so it does get a
bit ugly.

But no other solution makes that better anyway.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The Python Packaging Ecosystem (of Nick) Support for other programming languages

2017-04-06 Thread Nick Coghlan 
On 7 April 2017 at 00:32, Thomas Güttler  wrote:
> Dear Nick and other distutils listeners,
>
> Nick wrote this about seven months ago:
>
> http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html
>
> I love Python and I use it daily.
>
> On the other hand there are other interesting programming languages out
> there.
>
> Why not do "thinking in sets" here and see python just as one item in the
> list of a languages?
>
> Let's dream: All languages should be supported in the ever best packaging
> solution of the future.
>
> What do you think?

Package management that is both language and platform independent is
precisely the role conda fills, but as I explain in the post, that
doesn't eliminate the need for a language specific plugin manager for
Python runtimes (which is the role pip fills), and nor does it
eliminate the need for platform specific component managers (which is
the role filled by tools like apt-get, yum/dnf, OneGet, etc).

The language independent packaging problem is *way* out of scope for
distutils-sig as an entity though - while it's a genuine use case,
there are other more appropriate communities for discussing it (such
as those related to conda development).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The Python Packaging Ecosystem (of Nick) Support for other programming languages

2017-04-06 Thread Nathaniel Smith 
On Apr 6, 2017 7:32 AM, "Thomas Güttler" 
wrote:

Dear Nick and other distutils listeners,

Nick wrote this about seven months ago:

http://www.curiousefficiency.org/posts/2016/09/python-packag
ing-ecosystem.html

I love Python and I use it daily.

On the other hand there are other interesting programming languages out
there.

Why not do "thinking in sets" here and see python just as one item in the
list of a languages?

Let's dream: All languages should be supported in the ever best packaging
solution of the future.

What do you think?


This is basically what conda attempts to do. It's nice in some ways, but
does also have limitations.

In any case this isn't​ a very useful thing to post here? Distutils and pip
and pypi aren't going anywhere, and the folks here are all at their limit
trying to keep them from falling over, so taking up time with these super
vague blue sky ideas is a bit rude.

-n
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The Python Packaging Ecosystem (of Nick) Support for other programming languages

2017-04-06 Thread Robert Collins 
I wrote this 5 years ago, and its largely still true as far as I can
tell of the surrounding systems. Snappy-core for instance, isn't
targeting MacOS X or Windows (last I checked anyhow ...)

https://rbtcollins.wordpress.com/2012/08/27/why-platform-specific-package-systems-exist-and-wont-go-away/

-Rob

On 7 April 2017 at 02:32, Thomas Güttler  wrote:
> Dear Nick and other distutils listeners,
>
> Nick wrote this about seven months ago:
>
> http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html
>
> I love Python and I use it daily.
>
> On the other hand there are other interesting programming languages out
> there.
>
> Why not do "thinking in sets" here and see python just as one item in the
> list of a languages?
>
> Let's dream: All languages should be supported in the ever best packaging
> solution of the future.
>
> What do you think?
>
> Regards,
>   Thomas
>
>
>
> --
> Thomas Guettler http://www.thomas-guettler.de/
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] The Python Packaging Ecosystem (of Nick) Support for other programming languages

2017-04-06 Thread Thomas Güttler 

Dear Nick and other distutils listeners,

Nick wrote this about seven months ago:

http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html

I love Python and I use it daily.

On the other hand there are other interesting programming languages out there.

Why not do "thinking in sets" here and see python just as one item in the list 
of a languages?

Let's dream: All languages should be supported in the ever best packaging 
solution of the future.

What do you think?

Regards,
  Thomas



--
Thomas Guettler http://www.thomas-guettler.de/
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Versioned trove classifiers for Django

2017-04-06 Thread James Bennett 
Bumping this because Django 1.11 is out, so 'Framework :: Django :: 1.11'
would be a useful thing to have.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig