[Distutils] Re: Should an sdist/MANIFEST.in include docs and tests?

[Bert JW Regeer]

Speaking as a maintainer of various different packages for the Pylons project, 
we include the following in our sdists:

- source code for the package
- tests for the package
- documentation for the package

and of course the license/history/changelog/everything you'd theoretically need 
to create a fork (minus .git). Our sdists are pretty big as a result.

In our wheels we ship:

- source code for the package/software

And nothing else, tests are not included in the wheel. Some people do ship 
tests with their wheel, but we try not to, to keep wheel sizes small.

It comes down to personal preference, we tend to think that source dist means 
exactly that, a source distribution.

Bert

> On Sep 8, 2018, at 17:08, Segev Finer  wrote:
> 
> What should really be included in an sdist via MANIFEST.in? Besides the 
> obvious files required for a functioning package (Anything not caught by the 
> default rules required for the package to function that is) and obviously 
> LICENSE.txt and similar.
> 
> A package's source tree, more often than not, includes other files such as 
> documentation, tests, examples, a random assortment of other text files, etc. 
> Should docs & tests, in particular, be included in an sdist via MANIFEST.in? 
> Should other files be added too? Or maybe the sdist should be kept to a 
> minimum?
> 
> This is not clearly discussed in the packaging guide: 
> https://packaging.python.org/guides/distributing-packages-using-setuptools/#manifest-in
>  
> .
>  The sampleproject (https://github.com/pypa/sampleproject 
> ) does seem to include tests (Well a 
> no-op test) and doesn't include them in the sdist.
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/BJYGZIMYFA6EQD2OGQ6LCR7YFJL3EE74/



signature.asc
Description: Message signed with OpenPGP
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/KLW3SIHL6SXB7BESDS7NRU7RV5AF3BY5/

[Distutils] Re: Distlib vs Packaging (Was: disable building wheel for a package)

[Bert JW Regeer]

> On Sep 20, 2018, at 12:11, Tzu-ping Chung  wrote:
> 
> 
> On 21 Sep 2018, at 02:01, Bert JW Regeer  <mailto:xiste...@0x58.com>> wrote:
> 
>> 
>> 
>>> On Sep 19, 2018, at 23:22, Chris Jerdonek >> <mailto:chris.jerdo...@gmail.com>> wrote:
>>> 
>>> Thus, it's looking like things could be on track to split the user and 
>>> maintainer base in two, with pip bearing the legacy burden and perhaps not 
>>> seeing the improvements. Are we okay with that future?
>> 
>> This'll be a sad day. pip is still used as an installer by other build 
>> system where using pipenv is simply not a possibility.
> 
> I am not quite sure I understand why you’d think so. pip has been bearing the 
> legacy burden for years, and if this is the future (not saying it is), it 
> would more like just another day in the office for pip users, since nothing 
> is changing.

pip not seeing any improvements is something I think will be sad. I don't use 
pipenv, but use poetry which uses pip behind the scenes to do installation. I 
also use flit. For either of those cases I would think it sad that pipenv 
splits from pip, and then developers of alternate tooling around building 
packages (but not installing) don't get new improvements because "pip is 
legacy".

pipenv doesn't work in various scenarios, and trying to shoehorn it into those 
scenarios is just wrong especially since it wasn't designed to do those things.--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/3JKW3DJYVAGENQNQRLEAKKJXQC2QXIZF/

[Distutils] Re: Distlib vs Packaging (Was: disable building wheel for a package)

[Bert JW Regeer]

> On Sep 19, 2018, at 23:22, Chris Jerdonek  wrote:
> 
> Thus, it's looking like things could be on track to split the user and 
> maintainer base in two, with pip bearing the legacy burden and perhaps not 
> seeing the improvements. Are we okay with that future?

This'll be a sad day. pip is still used as an installer by other build system 
where using pipenv is simply not a possibility.--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/TDB7OETMEAFRVCASVF3GWF527OH7JSK2/

[Distutils] Re: Distlib vs Packaging (Was: disable building wheel for a package)

[Bert JW Regeer]

> On Sep 20, 2018, at 16:30, Dan Ryan  wrote:
> 
> Pipenv also uses pip as mentioned several times in the thread, and 
> (reiterating here) the entire point of the conversation is about how both can 
> work together on changes. That is the thrust of the whole discussion. We are 
> actively using pip via its internals and pips developers (who _actively 
> develop pip_) would like us to an alternate approach. 
> 
> The discussion is about how to find one and then contribute it back to pip. 
> Nobody is discontinuing work on pip, nobody is splitting from pip, and I 
> would prefer if we could refrain from trying to spread this kind of 
> inaccurate picture.

Wait, what? How did my apparently misunderstanding of what "it's looking like 
things could be on track to split the user and maintainer base in two" and me 
explaining why I don't think all new innovation should go into pipenv suddenly 
turn into "spread this kind of inaccurate picture". 

> I know we have had unproductive conversations on the issue tracker, please 
> don’t bring them to the mailing list. 

This isn't about you, has absolutely NOTHING to do with you, don't make it 
about you. I am trying to contribute my thoughts back to the discussion which 
only is of peripherally concerned about pipenv, but is about the future of 
pip/package installation, and a comment that was made regarding pip becoming 
"legacy".

You made me feel incredibly unwelcome to pipenv, I will no longer actively 
attempt to contribute back to that community. I have gone out of my way to stay 
away from any PyPA projects because of the actions and behaviours you showed on 
the pipenv tracker, and have actively encouraged others to do the same and look 
at other open source projects instead. Let us be crystal clear here, the way 
you and Kenneth have shown your colours on the pipenv issue tracker is a real 
shame and is turning off many potential contributors and good feedback to help 
improve pipenv.

This post, right here, has re-iterated that view.

Don't contact me again.

> 
> Dan Ryan // pipenv maintainer
> gh: @techalchemy
> 
> On Sep 20, 2018, at 2:29 PM, Bert JW Regeer  <mailto:xiste...@0x58.com>> wrote:
> 
>> 
>> 
>>> On Sep 20, 2018, at 12:11, Tzu-ping Chung >> <mailto:uranu...@gmail.com>> wrote:
>>> 
>>> 
>>> On 21 Sep 2018, at 02:01, Bert JW Regeer >> <mailto:xiste...@0x58.com>> wrote:
>>> 
>>>> 
>>>> 
>>>>> On Sep 19, 2018, at 23:22, Chris Jerdonek >>>> <mailto:chris.jerdo...@gmail.com>> wrote:
>>>>> 
>>>>> Thus, it's looking like things could be on track to split the user and 
>>>>> maintainer base in two, with pip bearing the legacy burden and perhaps 
>>>>> not seeing the improvements. Are we okay with that future?
>>>> 
>>>> This'll be a sad day. pip is still used as an installer by other build 
>>>> system where using pipenv is simply not a possibility.
>>> 
>>> I am not quite sure I understand why you’d think so. pip has been bearing 
>>> the legacy burden for years, and if this is the future (not saying it is), 
>>> it would more like just another day in the office for pip users, since 
>>> nothing is changing.
>> 
>> pip not seeing any improvements is something I think will be sad. I don't 
>> use pipenv, but use poetry which uses pip behind the scenes to do 
>> installation. I also use flit. For either of those cases I would think it 
>> sad that pipenv splits from pip, and then developers of alternate tooling 
>> around building packages (but not installing) don't get new improvements 
>> because "pip is legacy".
>> 
>> pipenv doesn't work in various scenarios, and trying to shoehorn it into 
>> those scenarios is just wrong especially since it wasn't designed to do 
>> those things.

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/BC7DTEX7VGBZ5NEAXQH2TRF7EXF3PMH3/

[Distutils] Re: Distlib vs Packaging (Was: disable building wheel fora package)

[Bert JW Regeer]

fic, like… I don’t know, pyproject? But that is taken 
> :p (This is intentionally rhetoric to touch on the 
> we-should-use-pyproject-for-this camp. To be clear: I am not in that camp, 
> that’s likely a bad idea unless we rethink the whole application-library 
> distinction Python packaging makes.)
> 
>  
> 
> TP
> 
>  
> 
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
> 
>  
> 
> From: Paul Moore <mailto:p.f.mo...@gmail.com>
> Sent: 21 September 2018 20:16
> To: Nick Coghlan <mailto:ncogh...@gmail.com>
> Cc: Michael Merickel <mailto:mmeri...@gmail.com>; Bert JW Regeer 
> <mailto:xiste...@0x58.com>; Distutils <mailto:distutils-sig@python.org>
> Subject: [Distutils] Re: Distlib vs Packaging (Was: disable building wheel 
> fora package)
> 
>  
> 
> On Fri, 21 Sep 2018 at 11:41, Nick Coghlan  <mailto:ncogh...@gmail.com>> wrote:
> 
> > 
> 
> > On Fri, 21 Sep 2018 at 05:47, Donald Stufft  > <mailto:don...@stufft.io>> wrote:
> 
> > > On Sep 20, 2018, at 3:35 PM, Paul Moore  > > <mailto:p.f.mo...@gmail.com>> wrote:
> 
> > > I don't think anyone's even spoken to the pip maintainers (yet?) about
> 
> > > supporting the pipfile format
> 
> > >
> 
> > > That comes from me, I initially wrote the Pipfile as a proof of concept / 
> > > sketch of an API for replacing the requirements.txt format, which Kenneth 
> > > took and created pipenv from. At some point I plan on trying to push 
> > > support for those ideas back into pip (not the virtual environment 
> > > management bits though). That’s obviously my personal goal though, and 
> > > doesn’t represent an agreed upon direction for pip.
> 
> > 
> 
> > And it's one where I think there are a couple of different levels of
> 
> > support that are worth considering:
> 
> > 
> 
> > Q. Should pip support installing from Pipfile.lock files as well as
> 
> > requirements.txt files?
> 
> > 
> 
> > A. Once the lock file format stabilises, absolutely, since this is
> 
> > squarely in pip's "component installer" wheelhouse.
> 
> > 
> 
> > Q. Should "pip install" support saving the installed components to a
> 
> > Pipfile, and then regenerating Pipfile.lock?
> 
> > 
> 
> > A. This is far less of a clearcut decision, as managing updates to a
> 
> > file that's intended to be checked in to source control is where I
> 
> > draw the line between "component installer" and
> 
> > "application/environment dependency manager".
> 
>  
> 
> Speaking as a pip developer:
> 
>  
> 
> Where's there a good summary of the pipfile format, the pipfile.lock
> 
> format, and their relationship and contrast with requirements.txt? I
> 
> don't view https://github.com/pypa/pipfile <https://github.com/pypa/pipfile> 
> as a "good summary",
> 
> because it explicitly states that pipfile is intended to *replace*
> 
> requirements.txt, and I disagree strongly with that.
> 
>  
> 
> Also, pipfile is human-readable, but pipfile.lock isn't. As far as I
> 
> know, pipfile.lock is currently generated solely by pipfile - before
> 
> pip consumes pipfile.lock, I'd like to see that format discussed and
> 
> agreed as a formal interop standard that any tools wanting to pass
> 
> data to pip (for the use case the standard describes) can use. One
> 
> obvious thing I'd like to consider is changing the name to something
> 
> less tool-specific - requirements.lock maybe?
> 
>  
> 
> As far as the pipfile format is concerned, I see that more as pipenv's
> 
> human readable input file that is used to *generate* the lock file,
> 
> and I don't see it as something pip should consume directly, as that
> 
> would mean pip overlapping in functionality with pipenv.
> 
>  
> 
> If I'm misunderstanding the relationship between pip and pipenv, or
> 
> between pipenv and pipfile, I'm happy to be corrected. But can I
> 
> suggest that the best way to do so would be to amend the project pages
> 
> that are giving me the impressions I have above, and pointing me at
> 
> the corrected versions? That way, we can make sure that any
> 
> misinformation is corrected at source...
> 
>  
> 
> Paul
> 
>  
> 
> PS Full disclosure - I've tried to use pipenv in a couple of local
> 
> projects, because of the hype about it being the "great new thing" and
> 
> found it basically of no use for my requirements/workflow. So

[Distutils] Re: Distlib vs Packaging (Was: disable building wheel for a package)

[Bert JW Regeer]

> On Sep 20, 2018, at 12:42, Donald Stufft  wrote:
> 
> 
> 
>> On Sep 20, 2018, at 2:29 PM, Bert JW Regeer > <mailto:xiste...@0x58.com>> wrote:
>> 
>> pip not seeing any improvements is something I think will be sad. I don't 
>> use pipenv, but use poetry which uses pip behind the scenes to do 
>> installation. I also use flit. For either of those cases I would think it 
>> sad that pipenv splits from pip, and then developers of alternate tooling 
>> around building packages (but not installing) don't get new improvements 
>> because "pip is legacy".
>> 
>> pipenv doesn't work in various scenarios, and trying to shoehorn it into 
>> those scenarios is just wrong especially since it wasn't designed to do 
>> those things.
> 
> FWIW, I don’t think that anyone is trying to sunset pip.

That's great, however that is not how the previous emails read to me. I'm glad 
that is sorted.--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/UWVZDM3CHNZSJL7M6HREJQ4QKDJK2EKM/

[Distutils] Building local versions with pip wheel

[Bert JW Regeer]

Hey all,

Patching 3rd party repositories comes up every so often at $WORK and one of the 
things we do is build a local version that is generally some released version + 
a couple of local patches.

We follow the usual version scheme of using the public version identifier + 
local version label.

The steps are as follows:

python setup.py egg_info -b "+local" sdist bdist_wheel
twine upload dist/*.whl

We only really care about the wheel though, and for all of our other 
dependencies we basically just run:

pip wheel 

and upload it to our local pypi instance.

This also allows us to directly build from Github repositories.

With for example:

pip wheel git+https://github.com/org/somerepo.git@ourpatchset#egg=somerepo 
https://github.com/org/somerepo.git@ourpatchset#egg=somerepo>

However we would like to add the build number so that when a new release is 
provided upstream we can easily update our local repository (or preferably we 
have upstreamed our patch in the meantime) and build a new release with our 
patches.

Is there some way to influence the egg info for the build when using pip wheel? 
Some thing like:

pip wheel 
git+https://github.com/org/somerepo.git@ourpatchset#egg=somerepo_info= 
https://github.com/org/somerepo.git@ourpatchset#egg=somerepo_info=>"-b 
+local"

Or is there a better method for dealing with this scenario?

Thanks,
Bert JW Regeer--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/Q73WK3J4ERVSCVVSYPIBUSOGTBR7XYWX/

[Distutils] Re: Building local versions with pip wheel

[Bert JW Regeer]

> On Jan 4, 2019, at 12:36, Nick Coghlan  wrote:
> 
> On Fri, 4 Jan 2019 at 16:50, Nick Coghlan  wrote:
>> 
>> On Thu, 3 Jan 2019 at 11:13, Bert JW Regeer  wrote:
>>> Is there some way to influence the egg info for the build when using pip 
>>> wheel? Some thing like:
>>> 
>>> pip wheel 
>>> git+https://github.com/org/somerepo.git@ourpatchset#egg=somerepo_info="-b
>>>  +local"
>>> 
>>> Or is there a better method for dealing with this scenario?
>> 
>> The way auditwheel approaches this is as a "post-process the already
>> built wheel" problem, rather than as a "modify the wheel build
>> process" problem:
>> 
>> 1. Build the original wheel archive however you'd normally do so
>> 2. Unpack the original wheel archive
>> 3. Make any desired changes to wheel contents and/or metadata
>> 4. Repack the modified wheel (potentially with an updated RECORD file
>> and modified filename)
>> 
>> auditwheel's own utilities for doing that kind of thing are here:
>> https://github.com/pypa/auditwheel/blob/master/auditwheel/wheeltools.py
> 
> It's also worth noting that distlib offers a helper API for modifying
> wheel archives without needing to write your own archive unpacking and
> repacking code:
> https://distlib.readthedocs.io/en/latest/tutorial.html#modifying-wheels
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/ZRKT4NRO6A2Q63DUCI3HISJO4WE4WLFJ/


Thanks for this information Nick, this is very helpful.

The other question I have is related to automated version number changes when 
using CI to do builds. At the moment we use Jenkins and we use the Jenkins 
build number to use egg_info with -b as well.

Modifying the wheel after the fact just to add the version number seems a 
little silly, especially since we are likely building from source in the first 
place, but there is no way (other than --global-options, but that disables all 
wheel support in pip) to tell pip wheel to increase the version number.

This really precludes us from having all of the features that pip wheel brings, 
such as isolated build environments and other such goodies (or even PEP-518/517 
support).

Would it make sense to add an optional extra_version or something along those 
lines to PEP-517's config_settings that is to be used by build backends to 
append to the existing version identifier?

I can automate things like bumpversion, but setup tools has supported passing 
daily builds/tags for a while now and it feels weird that we are going 
backwards with the new interfaces where it is made more difficult to do that in 
an easy/sane manner.

Especially in the case of continuous integration/deployment where there is no 
explicit version bump anymore and the code is always moving forward. The 
version number of or code base has been perpetually stuck at 1.0 with the only 
thing increasing is the build number making it: 1.0.. A new 
commit per build just to increase the version (thereby triggering CI again... 
causing fun loops ;-), especially for branch builds where a developer may still 
be making more changes and the build number just needs to increase so that each 
build of a branch has a unique version number.

I will admit that I have yet to test if pip passes through the environment 
un-modified, because in that case I may just end up using the environment 
variable in setup.py directly.

Bert JW Regeer
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/3TRGSCVLL6DE5JPLBNXJ43NEMKIDKDOG/

[Distutils] Re: psycopg2/psycopg2-binary

[Bert JW Regeer]

> On Apr 6, 2019, at 23:10, Nathaniel Smith  wrote:
> 
> On Sat, Apr 6, 2019 at 2:14 PM Bert JW Regeer  wrote:
>> 
>> Hey all,
>> 
>> You may have seen some hub hub around psycopg2 and no longer shipping binary 
>> wheels by default [1][2] (and in fact using psycopg2-binary if you want 
>> wheels), and I wanted to bring it up here because it demonstrates a problem 
>> area with the current state of packaging in Python:
>> 
>> There is no good way for a new package to specify that it provides what 
>> another package would provide, and setuptools currently checks that all 
>> distributions are found before running the console scripts (so if a console 
>> script has a setup.py that depends on psycopg2 and you install 
>> psycopg2-binary it fails) [3]
>> 
>> So currently if you pip install psycopg2-binary and then install a project 
>> that uses psycopg2 as a dependency it will install psycopg2 over top of 
>> psycopg2-binary.
>> 
>> The author of psycopg2 stopped distributing binaries for psycopg2 because of 
>> issues with the version of SSL that Python was compiled for/what was used by 
>> psycopg2 and it causing all kinds of issues.
>> 
>> I don't have a proposal or a fix, but this is going to be an issue not just 
>> for psycopg2 but also for other projects that potentially distribute wheels 
>> that are built against a different version of OpenSSL.
>> 
>> I see two things that should get some thought:
>> 
>> 1. How to have a package provide for another package (there are keywords but 
>> AFAIK they are currently ignored by pip)
>> 2. How to handle/deal with shared libraries that are not versioned
> 
> The psycopg2 authors originally misdiagnosed the problem, and haven't
> updated their docs since the problem was diagnosed further, so a lot
> of people are confused about this whole psycopg2-binary thing :-(
> 
> There is no problem with shipping openssl in wheels. Lots of projects
> do it fine. The reason psycopg2 is having problems is because of an
> easily-fixable bug in psycopg2:
> 
> - Old versions of OpenSSL need some annoying configuration applied to
> make them thread-safe
> - libpq (which psycopg2 uses) normally does this configuration in one way
> - the Python ssl module also normally does this configuration in a different 
> way
> - If libpq and the stdlib ssl module are both linked against the
> *same* copy of openssl, then they can end up fighting with each other
> - So the psycopg2 code has a special hack to unconditionally disable
> libpq's thread-safety code, because the psycopg2 developers assumed
> that psycopg2 and the stdlib ssl would *always* share the same copy of
> openssl, and the stdlib ssl module would take care of the
> thread-safety stuff
> - Then they started distributing psycopg2 binaries with their own copy
> of openssl in them, and of course they got crashes, because they've
> turned off thread-safety, and now that they have their own copy of
> openssl, no-one else is fixing it for them
> 
> So all they need to do to fix their wheels is either:
> 
> - somehow disable this patch in their wheel builds:
> https://github.com/psycopg/psycopg2/commit/a59704cf93e8594dfe59cf12d416e82a816953a4
> 
> Or else:
> 
> - switch to building their wheels against a newer version of openssl,
> since newer versions of openssl are now thread-safe by default (thank
> goodness)
> 
> Either way, it's totally fixable, and only the psycopg2 devs can fix it.
> 
> They've known this since July, but said they don't have energy to fix
> it: https://github.com/psycopg/psycopg2/issues/543#issuecomment-408352209
> 
> I sympathize with that, but I wish they would tell people "hey, this
> is a bug in psycopg2, we need help", instead of blaming python
> packaging and trying to force all their downstreams to do wacky stuff
> with dependencies to work around their bug.
> 
> It would indeed be nice if Python packages had better support for
> Provides:, but psycopg2 is not really a good motivating use case.

No, it is not a good motivating use case, but it is an issue that exists 
_today_ hence why I brought it up.

> 
> -n
> 
> -- 
> Nathaniel J. Smith -- https://vorpus.org <https://vorpus.org/>

Thank you for your information Nathaniel. It is a shame that this fix was not 
used so that they could continue to ship psycopg2 wheels. Sadly the maintainers 
have made more work for themselves by not following up on your suggestions :-/

Bert

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/SME74E4PCAUCQXGRQ362HSXHL5JRNN3T/

[Distutils] Fwd: BitBounc BS Was: psycopg2/psycopg2-binary

[Bert JW Regeer]

Is there a contact for the list owner so that people like this can get removed?

> Begin forwarded message:
> 
> From: eys...@python-org.link
> Subject: Re: [Distutils] psycopg2/psycopg2-binary
> Date: April 6, 2019 at 15:34:29 MDT
> To: xiste...@0x58.com
> 
> 
>
>
> Thanks for emailing me! No, I haven’t been hacked :)
> 
> I signed up for a spam filtering service called BitBounce. To deliver your 
> email to my inbox, please click the link below and pay the small Bitcoin fee. 
> Thanks!
> 
> $0.05 to deliver your email.
> 
> We’ve never met — I’ll pay your fee.
> 
>  
> I know you — Add me to your whitelist.
> 
>  
> 
>
>
> BitBounce  is powered by
>  the Credo 
>  cryptocurrency
> 
> I’m from a business — 
> what are my delivery options 
>    
> BitBounce and Credo are
> transacted through CredoEx 
> 
> Made by Turing Technology Inc. in San Mateo, California Sign Up for BitBounce 
> 
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/B64AMGYPMJYGWUMOWP7ZYIJ67NXBFVXQ/

[Distutils] psycopg2/psycopg2-binary

[Bert JW Regeer]

Hey all,

You may have seen some hub hub around psycopg2 and no longer shipping binary 
wheels by default [1][2] (and in fact using psycopg2-binary if you want 
wheels), and I wanted to bring it up here because it demonstrates a problem 
area with the current state of packaging in Python:

There is no good way for a new package to specify that it provides what another 
package would provide, and setuptools currently checks that all distributions 
are found before running the console scripts (so if a console script has a 
setup.py that depends on psycopg2 and you install psycopg2-binary it fails) [3]

So currently if you pip install psycopg2-binary and then install a project that 
uses psycopg2 as a dependency it will install psycopg2 over top of 
psycopg2-binary.

The author of psycopg2 stopped distributing binaries for psycopg2 because of 
issues with the version of SSL that Python was compiled for/what was used by 
psycopg2 and it causing all kinds of issues.

I don't have a proposal or a fix, but this is going to be an issue not just for 
psycopg2 but also for other projects that potentially distribute wheels that 
are built against a different version of OpenSSL.

I see two things that should get some thought:

1. How to have a package provide for another package (there are keywords but 
AFAIK they are currently ignored by pip)
2. How to handle/deal with shared libraries that are not versioned

Thanks,
Bert

[1]: https://github.com/psycopg/psycopg2/issues/674 

[2]: https://github.com/psycopg/psycopg2/issues/883 

[3]: https://github.com/zalando/patroni/issues/1021#issuecomment-480202590 


--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/NCTYRASPRQ2JALB3HLA3KOFW7BGV2LQE/

[Distutils] Not at the sprints -- editable installs?

[Bert JW Regeer]

Hey all,

I am not at the sprints and I am not sure where I can follow the discussion and 
or see what people are coming up with.

Is there any feedback or information on editable installs using PEP517/PEP518 
projects?

Bert
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/W35DQNXT7VD7GQL7D5OSRCXCKDVMDRCO/

[Distutils] Re: Linux binary wheels?

[Bert JW Regeer]

The nice thing with the new macaroons is that theoretically I can provide 
someone my key to upload packages on my behalf for a singular PyPI project. 
This way I could allow a third-party service to backfill binary wheels for 
other platforms once I've released a version to PyPI.

Bert

> On Aug 20, 2019, at 12:25, Tzu-ping Chung  wrote:
> 
> 
> On 20 Aug 2019, at 23:47, Nick Timkovich  > wrote:
> 
>> On Tue, Aug 20, 2019, at 5:05 AM Matthew Brett > > wrote:
>> ...  Unless you meant wheels for non-Intel platforms, in which case, please 
>> do say more about you need.
>> 
>> Minor tangent: I've seen some people use https://www.piwheels.org/ 
>>  for Raspberry Pi (ARM 6/7), but could the ARM 
>> binaries be uploaded to PyPI?
>> 
>> I think I'm conflating the wheel building spec (is manylinux amd64 specific, 
>> or as long as the libraries are on any architecture?), toolchains, 
>> environment (sounds like Piwheels provides a platform to build them on), and 
>> package hosting (can PyPI host arbitrary archs?) in that one sentence.
> 
> This issue may be of relevant: https://github.com/pypa/warehouse/issues/3668 
> 
> 
> And there are even more layers to this problem. Wheels on piwheels are 
> currently maintained by RPi folks; if they are going into PyPI, either 
> package maintainers need to take over uploading (and even building) them, or 
> PyPI needs a way to allow (qualified) people to upload stuffs for packages 
> they don’t own. And maintainers might decide that ARM is not their supported 
> platform anyway, and get us back to where we started.
> 
>> --
>> Distutils-SIG mailing list -- distutils-sig@python.org 
>> 
>> To unsubscribe send an email to distutils-sig-le...@python.org 
>> 
>> https://mail.python.org/mailman3/lists/distutils-sig.python.org/ 
>> 
>> Message archived at 
>> https://mail.python.org/archives/list/distutils-sig@python.org/message/OXSUW73EO5DTUO34EFURN3KHCDAKNS4Z/
>>  
>> 
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/HYGOQ6EGLZHNN6HR3KY6XUGI2ZZQX2SI/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/O5CXREPQQJ5SGHKUCHFNRFJQS7RUS2LM/

[Distutils] Re: Questionable package in PyPi, maybe malicious

[Bert JW Regeer]

The package contains two files, and basically prints out:

"You probably meant to install flask-migrate"

It is meant to make sure that no-one else squats on the name and uses it to 
exploit unsuspecting users that install the wrong package.

Bert JW Regeer

> On Mar 21, 2020, at 01:09, Evagelos  wrote:
> 
> Hi there,
> 
> I searched PyPi for "flaskmigrate" and this came up:
> flaskmigrate - A package to prevent exploit
> 
> Seems strange to me so I wanted to make you aware, I was advised to report in 
> this mailing list.
> 
> Regards,
> Evagelos
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/D5EISUDCPU6DMWG2CJK6RK6TZSJXSWD5/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/O4OM744BSRP5YY3F3XISHDIU5LCQBVH5/

[Distutils] Re: Questionable package in PyPi, maybe malicious

[Bert JW Regeer]

The package contains two files, and basically prints out:

"You probably meant to install flask-migrate"

It is meant to make sure that no-one else squats on the name and uses it to 
exploit unsuspecting users that install the wrong package.

Bert JW Regeer

> On Mar 21, 2020, at 01:09, Evagelos  wrote:
> 
> Hi there,
> 
> I searched PyPi for "flaskmigrate" and this came up:
> flaskmigrate - A package to prevent exploit
> 
> Seems strange to me so I wanted to make you aware, I was advised to report in 
> this mailing list.
> 
> Regards,
> Evagelos
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/D5EISUDCPU6DMWG2CJK6RK6TZSJXSWD5/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/TJ3ZXZ77N7WKF5VY54C7S7L656A4QSSU/

[Distutils] Re: Best way for a project to provide an optional C module

[Bert JW Regeer]

By splitting it into two different packages you end up with the same situation 
that currently plagues psycopg2/psycopg2-binary whereby if you depend on 
psycopg2 you can't easily swap in psycopg2-binary and vice-versa as the two 
don't satisfy the same dependency.

Number 3 is kind of what sqlalchemy does, and then provide wheels for a huge 
variety of platforms to allow people to install the package without needing a 
compiler themselves.

Bert

> On Aug 2, 2020, at 04:42, Daniele Varrazzo  wrote:
> 
> Hello,
> 
> packaging psycopg3, I'm wondering what is the best way to provide an
> optional optimisation module.
> 
> 1) provide a separate psycopg3-c distribution
> 2) provide an extra psycopg3[c]
> 3) try building the extension and fail quietly.
> 
> 1) seems the cleanest approach: the psycopg3 distribution would have
> no build-time external dependency (Cython, -dev packages, a compiler)
> and psycopg3-c can fail hard if some of these dependencies are
> missing. I am currently trying this approach, finding some problems in
> working out a good files layout to have two setup.py in the same git
> repository.
> 
> 2) would be nice but I don't see a way to identify the extra requested
> at build time to implement a build_ext command such that if "c" is not
> the extra then don't do anything. it seems that extra are thought for
> a different use case, not for optional build-time parts
> 
> 3) would give me endless headaches to work out why something failed,
> differentiate failures for missing dependencies from real errors, and
> dealing with user reports.
> 
> What would be your advice? Press on with 1 or a different approach?
> Examples are welcome (the only one I have in mind is PyYAML doing 3).
> 
> 
> -- Daniele
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/archives/list/distutils-sig@python.org/message/GEPU27VYHQ54BSAYDAK7ZPBFXAMVJIII/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/4YQOQ3EBKZE243ZZYWFGCB3TJCQDQJMX/

[Distutils] Re: pip install very slow

[Bert JW Regeer]

Isn't your data folder mentioned here in your setup.py?

> On Jun 25, 2020, at 12:12, Stuart McGraw  wrote:
> 
> package_data={'mypackage': ['data/*.csv', 'tmpl/*.jinja',]},

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/archives/list/distutils-sig@python.org/message/RQ4VV3Y4JQ5P2OCFY542N3OVQLNJO3TZ/