> On Apr 6, 2019, at 23:10, Nathaniel Smith <n...@pobox.com> wrote: > > On Sat, Apr 6, 2019 at 2:14 PM Bert JW Regeer <xiste...@0x58.com> wrote: >> >> Hey all, >> >> You may have seen some hub hub around psycopg2 and no longer shipping binary >> wheels by default [1][2] (and in fact using psycopg2-binary if you want >> wheels), and I wanted to bring it up here because it demonstrates a problem >> area with the current state of packaging in Python: >> >> There is no good way for a new package to specify that it provides what >> another package would provide, and setuptools currently checks that all >> distributions are found before running the console scripts (so if a console >> script has a setup.py that depends on psycopg2 and you install >> psycopg2-binary it fails) [3] >> >> So currently if you pip install psycopg2-binary and then install a project >> that uses psycopg2 as a dependency it will install psycopg2 over top of >> psycopg2-binary. >> >> The author of psycopg2 stopped distributing binaries for psycopg2 because of >> issues with the version of SSL that Python was compiled for/what was used by >> psycopg2 and it causing all kinds of issues. >> >> I don't have a proposal or a fix, but this is going to be an issue not just >> for psycopg2 but also for other projects that potentially distribute wheels >> that are built against a different version of OpenSSL. >> >> I see two things that should get some thought: >> >> 1. How to have a package provide for another package (there are keywords but >> AFAIK they are currently ignored by pip) >> 2. How to handle/deal with shared libraries that are not versioned > > The psycopg2 authors originally misdiagnosed the problem, and haven't > updated their docs since the problem was diagnosed further, so a lot > of people are confused about this whole psycopg2-binary thing :-( > > There is no problem with shipping openssl in wheels. Lots of projects > do it fine. The reason psycopg2 is having problems is because of an > easily-fixable bug in psycopg2: > > - Old versions of OpenSSL need some annoying configuration applied to > make them thread-safe > - libpq (which psycopg2 uses) normally does this configuration in one way > - the Python ssl module also normally does this configuration in a different > way > - If libpq and the stdlib ssl module are both linked against the > *same* copy of openssl, then they can end up fighting with each other > - So the psycopg2 code has a special hack to unconditionally disable > libpq's thread-safety code, because the psycopg2 developers assumed > that psycopg2 and the stdlib ssl would *always* share the same copy of > openssl, and the stdlib ssl module would take care of the > thread-safety stuff > - Then they started distributing psycopg2 binaries with their own copy > of openssl in them, and of course they got crashes, because they've > turned off thread-safety, and now that they have their own copy of > openssl, no-one else is fixing it for them > > So all they need to do to fix their wheels is either: > > - somehow disable this patch in their wheel builds: > https://github.com/psycopg/psycopg2/commit/a59704cf93e8594dfe59cf12d416e82a816953a4 > > Or else: > > - switch to building their wheels against a newer version of openssl, > since newer versions of openssl are now thread-safe by default (thank > goodness) > > Either way, it's totally fixable, and only the psycopg2 devs can fix it. > > They've known this since July, but said they don't have energy to fix > it: https://github.com/psycopg/psycopg2/issues/543#issuecomment-408352209 > > I sympathize with that, but I wish they would tell people "hey, this > is a bug in psycopg2, we need help", instead of blaming python > packaging and trying to force all their downstreams to do wacky stuff > with dependencies to work around their bug. > > It would indeed be nice if Python packages had better support for > Provides:, but psycopg2 is not really a good motivating use case.
No, it is not a good motivating use case, but it is an issue that exists _today_ hence why I brought it up. > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org <https://vorpus.org/> Thank you for your information Nathaniel. It is a shame that this fix was not used so that they could continue to ship psycopg2 wheels. Sadly the maintainers have made more work for themselves by not following up on your suggestions :-/ Bert
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/SME74E4PCAUCQXGRQ362HSXHL5JRNN3T/
