Re: Helping hands needed to upgrade scipy
Hi Drew, Am Wed, Jan 18, 2023 at 02:45:31AM +0100 schrieb Drew Parsons: > On 2023-01-17 18:12, Andreas Tille wrote: > > Hi, > > > > I upgraded the experimental branch of scipy to contain the submodules > > upstream includes. I'm *not* convinced that we need them all - > > hopefully we can get rid of boost (but upstream has some patches here) > > and scipy-mathjax. > > > > For the moment I've tried to build the experimental branch and ended > > up with some issue where cython can't be found (should be cython3 for > > sure)[1]. > > > > Unfortunately my spare time is occupied now by other more urgent tasks. > > I'd be happy if someone else could review / pick up / continue from > > here. > > > The problem was not meson as such, but that upstream is only half using > meson. They must have decided they don't trust the way meson handles > cython, and set up their own custom build tools for it, see > https://github.com/scipy/scipy/pull/15407 > > That is, they hard-coded the cython executable name in > scipy/_build_utils/cythoner.py > > They actually did the same already in the old setuptools build > (tools/cythonize.py), but we didn't notice since they had the breaking > cython call wrapped in a try block which inspected a Cython class from > inside python. I raised an Issue about whether cythoner.py should do the > same in https://github.com/scipy/scipy/issues/17808 > We could argue the fault is debian's for not providing a cython executable > (it does have -2 and -3 options for choosing the python version to work > with). Thanks for the clarification. > I figure also we should use python3-mesonpy, since it's what upstream uses. > Best not to diverge too far from their build method I think. I'm perfectly fine with this approach. That's why I initially pushed meson-python to new. I've just dput meson-python_0.12.0-2_source to let it migrate to testing. My attempt to use plain meson (as per upstream readme) was just to shorten the time while meson-python was in new queue. > I've pushed commits updating cythoner.py to point cythoner.py at cython3. > The main build now completes. Have to clean up test handling next: > "FileNotFoundError: [Errno 2] No such file or directory: > '/projects/python/build/scipy/.coveragerc' Nice. Hope my preparation (of meson-python and the multi-source tarball) helped to move forward. Please double check whether we really need the boost code copy. Kind regards Andreas. -- http://fam-tille.de
Re: Helping hands needed to upgrade scipy
On 2023-01-17 18:12, Andreas Tille wrote: Hi, I upgraded the experimental branch of scipy to contain the submodules upstream includes. I'm *not* convinced that we need them all - hopefully we can get rid of boost (but upstream has some patches here) and scipy-mathjax. For the moment I've tried to build the experimental branch and ended up with some issue where cython can't be found (should be cython3 for sure)[1]. Unfortunately my spare time is occupied now by other more urgent tasks. I'd be happy if someone else could review / pick up / continue from here. The problem was not meson as such, but that upstream is only half using meson. They must have decided they don't trust the way meson handles cython, and set up their own custom build tools for it, see https://github.com/scipy/scipy/pull/15407 That is, they hard-coded the cython executable name in scipy/_build_utils/cythoner.py They actually did the same already in the old setuptools build (tools/cythonize.py), but we didn't notice since they had the breaking cython call wrapped in a try block which inspected a Cython class from inside python. I raised an Issue about whether cythoner.py should do the same in https://github.com/scipy/scipy/issues/17808 We could argue the fault is debian's for not providing a cython executable (it does have -2 and -3 options for choosing the python version to work with). I figure also we should use python3-mesonpy, since it's what upstream uses. Best not to diverge too far from their build method I think. I've pushed commits updating cythoner.py to point cythoner.py at cython3. The main build now completes. Have to clean up test handling next: "FileNotFoundError: [Errno 2] No such file or directory: '/projects/python/build/scipy/.coveragerc' Drew
Re: Old generated binary dependencies after renaming psycopg
On January 17, 2023 11:01:45 PM UTC, Tomasz Rybak wrote:
>On Tue, 2023-01-17 at 09:20 +, Julian Gilbey wrote:
>> On Tue, Jan 17, 2023 at 09:08:01AM +0100, Tomasz Rybak wrote:
>> > Hello.
>> > After fixing #1016031 "psycopg3: binary package name should be python3-
>> > psycopg"
>> > (I renamed package names, full changes:
>> > * python3-psycopg3 -> python3-psycopg
>> > * python3-psycopg3-pool -> python3-psycopg-pool
>> > * python-psycopg3-doc -> python3-psycopg-doc)
>> > I tried to rebuild reverse dependencies,
>> > i.e. pgcli and python3-pgspecial.
>> > Rebuild went without problems, new packages are the same
>> > as old ones, but their binary packages still depend on python3-
>> > psycopg3,
>> > even though they build-depend on python3-psycopg.
>>
>> Nope, pgcli does not build-depend on it, rather it explicitly
>> specifies Depends: python3-psycopg3. Likewise, python-pgspecial
>> specifies the same Depends (though it also has a Build-Depends:
>> python3-psycopg3). These packages will need their dependencies
>> updating. (You might also consider making python3-psycopg Provides:
>> python3-psycopg3 and likewise for the other two binary packages for
>> bookworm.)
>>
>
>No, this is not the problem.
>I checked (rebuilt packages with) different variants - with and
>without python3-psycopg[3] as explicit dependency. In all the cases
>dependency for python3-psycopg3 comes from ${python3:Depends}
>via pgcli/debian/pgcli.substvars which contains
>python3:Depends=python3-cli-helpers, python3-click, python3-configobj,
>python3-pendulum, python3-pgspecial, python3-prompt-toolkit, python3-
>psycopg3, python3-pygments, python3-setproctitle, python3-sqlparse (>=
>0.3), python3:any.
>
>When python3-psycopg is in Depends in debian/control, it just gets
>added - so binary package's Depends contains it twice (once
>python3-psycopg3 from ${python3:Depends, once python3-psycopg
>mentioned explicitly; in case of pgcli the second one is versioned).
>
>Sample line from control file for pgcli, which contains both
>python3-psycopg and python3-psycopg3:
>Depends: python3-cli-helpers, python3-pendulum, python3-pgspecial (>=
>2), python3-pkg-resources, python3-prompt-toolkit (>= 3.0), python3-
>psycopg (>= 3.0.14), python3-sqlparse (>= 0.3), python3-tabulate,
>python3-terminaltables, python3-click, python3-configobj, python3-
>psycopg3, python3-pygments, python3-setproctitle, python3:any
>
>I tried to analyze dh_python3, but could not understand where exactly
>fills it in list of (additional) dependencies. They are generated
>somewhere between dh_python3, dhpython/pydist.py,
>and dhpython/depends.py (all files belong to package dh-python
>and are in /usr/share/dh-python).
>
>So - any tips how to fix it would be really helpful.
You'll need to add a py3dist-overrides file in /debian. The details are in the
dh-python or pybuild documentation. I don't recall where.
Scott K
Re: Old generated binary dependencies after renaming psycopg
On Tue, 2023-01-17 at 09:20 +, Julian Gilbey wrote:
> On Tue, Jan 17, 2023 at 09:08:01AM +0100, Tomasz Rybak wrote:
> > Hello.
> > After fixing #1016031 "psycopg3: binary package name should be python3-
> > psycopg"
> > (I renamed package names, full changes:
> > * python3-psycopg3 -> python3-psycopg
> > * python3-psycopg3-pool -> python3-psycopg-pool
> > * python-psycopg3-doc -> python3-psycopg-doc)
> > I tried to rebuild reverse dependencies,
> > i.e. pgcli and python3-pgspecial.
> > Rebuild went without problems, new packages are the same
> > as old ones, but their binary packages still depend on python3-
> > psycopg3,
> > even though they build-depend on python3-psycopg.
>
> Nope, pgcli does not build-depend on it, rather it explicitly
> specifies Depends: python3-psycopg3. Likewise, python-pgspecial
> specifies the same Depends (though it also has a Build-Depends:
> python3-psycopg3). These packages will need their dependencies
> updating. (You might also consider making python3-psycopg Provides:
> python3-psycopg3 and likewise for the other two binary packages for
> bookworm.)
>
No, this is not the problem.
I checked (rebuilt packages with) different variants - with and
without python3-psycopg[3] as explicit dependency. In all the cases
dependency for python3-psycopg3 comes from ${python3:Depends}
via pgcli/debian/pgcli.substvars which contains
python3:Depends=python3-cli-helpers, python3-click, python3-configobj,
python3-pendulum, python3-pgspecial, python3-prompt-toolkit, python3-
psycopg3, python3-pygments, python3-setproctitle, python3-sqlparse (>=
0.3), python3:any.
When python3-psycopg is in Depends in debian/control, it just gets
added - so binary package's Depends contains it twice (once
python3-psycopg3 from ${python3:Depends, once python3-psycopg
mentioned explicitly; in case of pgcli the second one is versioned).
Sample line from control file for pgcli, which contains both
python3-psycopg and python3-psycopg3:
Depends: python3-cli-helpers, python3-pendulum, python3-pgspecial (>=
2), python3-pkg-resources, python3-prompt-toolkit (>= 3.0), python3-
psycopg (>= 3.0.14), python3-sqlparse (>= 0.3), python3-tabulate,
python3-terminaltables, python3-click, python3-configobj, python3-
psycopg3, python3-pygments, python3-setproctitle, python3:any
I tried to analyze dh_python3, but could not understand where exactly
fills it in list of (additional) dependencies. They are generated
somewhere between dh_python3, dhpython/pydist.py,
and dhpython/depends.py (all files belong to package dh-python
and are in /usr/share/dh-python).
So - any tips how to fix it would be really helpful.
Best regards.
--
Tomasz Rybak, Debian Developer
GPG: A565 CE64 F866 A258 4DDC F9C7 ECB7 3E37 E887 AA8C
signature.asc
Description: This is a digitally signed message part
Re: twitterwatch - upload needed
Hi, On Tue, Jan 17, 2023 at 6:16 PM Malik wrote: > Hello Emmanuel, > > currently upstream-author has no plan to update or release any further > releases for this project [1] > Ok > > > [1] https://gitlab.com/chaica/twitterwatch/-/issues/1 > > kind regards, > Malik > If you leave the package as is it now, when a new release exists, d/watch won't note it. So, I'd change the d/watch to the Gitlab repository. Also Homepage link. Cheers, Emmanuel
Re: How to create multi-source tarball with different submodules for scipy
On Mon, 16 Jan 2023 at 16:06, Andreas Tille wrote: > > Hi, > > I tried to create a multi-source tarball for scipy in its experimental > branch[1]. Upstream includes a set of git submodules in its build > process. I intended to merge all these submodules in a single > scipy_1.10.0.orig-submodules.tar.gz. This tarball is created with a > script[2] which makes sure that the exact directory structure as it is > used by upstream is conserved. This directory layout is needed in the > build process. Unfortunately `dpkg-source -x` extracts the content of > the submodules tarball into a subdirectory submodules/. > > Is there any trick to unpack this tarball right into the root? > Otherwise I need to do some symlinks workaround in d/rules to provide > all files where these are needed. When I packaged firmware-microbit-microbit I wasn't aware of any tricks, so I resorted to overriding dh_auto_build to move the extracted components into their expected paths before building: https://salsa.debian.org/python-team/packages/firmware-microbit-micropython/-/blob/debian/master/debian/rules Not sure if this useful, but I thought I'd pass it along in case it helps. Cheers, Nick
Re: twitterwatch - upload needed
Hello Emmanuel, currently upstream-author has no plan to update or release any further releases for this project [1] [1] https://gitlab.com/chaica/twitterwatch/-/issues/1 kind regards, Malik Am So., 15. Jan. 2023 um 13:28 Uhr schrieb Emmanuel Arias < eam...@yaerobi.com>: > Hi, > > On Sun, Jan 15, 2023 at 12:24:43PM +0100, Malik wrote: > > Hello Emannauel, > > > > Thank you for the suggestions. > > > > I was not sure how to handle the new homepage [1] since there are no > tags > > or releases maintained by the upstream maintainer, should I delete the > > watchfile? or can I watch the version in the setup.py ? > > Oh, I didn't note it. Perhaps it's a good reason to contact to upstream > and ask for them. Don't remove d/watch file. > > > > > And for the tests, should I mock the calls to the "internet" since those > > would be skipped if conceited to the internet > > Yes, or skip them. > > > > > > [1]: https://gitlab.com/chaica/twitterwatch/-/tags > > > > regards, > > Malik > > > > Cheers, > Emmanuel > -- Malik Mlitat
Re: Help for python-pynndescent needed
Hi Malik, Am Tue, Jan 17, 2023 at 07:53:32PM +0100 schrieb Malik: > did you tried to run those tests with a lower python version, you are > running tests with 3.10 [1]: > > = test session starts > == > platform linux -- Python 3.10.9, pytest-7.2.1, pluggy-1.0.0+repack -- > /usr/bin/python3.10 > > and the upstream is running those tests with max 3.9 : > > platform linux -- Python 3.9.16, pytest-7.2.0, pluggy-1.0.0 -- > /opt/hostedtoolcache/Python/3.9.16/x64/bin/python Lowering the Python3 version does not really help. If the tests do not run with the current default 3.11 than it will not migrate to testing. Kind regards Andreas. -- http://fam-tille.de
Re: Help for python-pynndescent needed
Hey Andreas, did you tried to run those tests with a lower python version, you are running tests with 3.10 [1]: = test session starts == platform linux -- Python 3.10.9, pytest-7.2.1, pluggy-1.0.0+repack -- /usr/bin/python3.10 and the upstream is running those tests with max 3.9 : platform linux -- Python 3.9.16, pytest-7.2.0, pluggy-1.0.0 -- /opt/hostedtoolcache/Python/3.9.16/x64/bin/python regards, Malik Am Di., 17. Jan. 2023 um 18:24 Uhr schrieb Andreas Tille : > Hi, > > any idea why the build-time test of python-pynndescent fails[1]? > > Any help would be welcome. > > Kind regards >Andreas. > > > [1] > https://salsa.debian.org/python-team/packages/python-pynndescent/-/jobs/3811021 > > -- > http://fam-tille.de > > -- Malik Mlitat
Re: Helping hands needed to upgrade scipy
Hi Andreas, I am an upstream author of software packages (fabio & pyFAI) which took the same build system as scipy (meson-python) and I can confirm you there are issues with the debian packaging: * finding cython (apparently it should be fixed upstream in meson) * choose the proper python: sid has currently py3.10 and 3.11 co-installed and I found out the wrong one was used when building: https://github.com/mesonbuild/meson/issues/11248 I hope this helps, Jerome On Tue, 17 Jan 2023 18:12:42 +0100 Andreas Tille wrote: > Hi, > > I upgraded the experimental branch of scipy to contain the submodules > upstream includes. I'm *not* convinced that we need them all - > hopefully we can get rid of boost (but upstream has some patches here) > and scipy-mathjax. > > For the moment I've tried to build the experimental branch and ended > up with some issue where cython can't be found (should be cython3 for > sure)[1]. > > Unfortunately my spare time is occupied now by other more urgent tasks. > I'd be happy if someone else could review / pick up / continue from > here. > > Kind regards > Andreas. > > > [1] https://salsa.debian.org/python-team/packages/scipy/-/jobs/3809903 > > -- > http://fam-tille.de >
Help for python-pynndescent needed
Hi, any idea why the build-time test of python-pynndescent fails[1]? Any help would be welcome. Kind regards Andreas. [1] https://salsa.debian.org/python-team/packages/python-pynndescent/-/jobs/3811021 -- http://fam-tille.de
Helping hands needed to upgrade scipy
Hi, I upgraded the experimental branch of scipy to contain the submodules upstream includes. I'm *not* convinced that we need them all - hopefully we can get rid of boost (but upstream has some patches here) and scipy-mathjax. For the moment I've tried to build the experimental branch and ended up with some issue where cython can't be found (should be cython3 for sure)[1]. Unfortunately my spare time is occupied now by other more urgent tasks. I'd be happy if someone else could review / pick up / continue from here. Kind regards Andreas. [1] https://salsa.debian.org/python-team/packages/scipy/-/jobs/3809903 -- http://fam-tille.de
Re: How to create multi-source tarball with different submodules for scipy
Am Tue, Jan 17, 2023 at 08:55:21AM +0800 schrieb Paul Wise: > On Mon, 2023-01-16 at 17:05 +0100, Andreas Tille wrote: > > > I intended to merge all these submodules in a single > > scipy_1.10.0.orig-submodules.tar.gz. > Any reason not to use one tarball per submodule? Yes, this would need manual intervention for any change of the modules (either we can get rid of one or upstream adds another one). Single tarballs also would not really solve the problem I was asking for. Kind regards Andreas. -- http://fam-tille.de
Bug#1029076: uwsgi-plugin-python3: built against non-default libpython3.11 / should always build against the defalt Python in testing
Package: uwsgi-plugin-python3 Version: 2.0.21-3+b1 Severity: grave Justification: renders package unusable X-Debbugs-Cc: debian-python@lists.debian.org -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Currently, the uWSGI Python 3 plugin is built against Python 3.11, and depends on libpython3.11. This is, to some extent, fine, as Python 3.11 is already in Debian. However, Python 3.10 is still the default Python in bookworm, and as it stands this will not change [1]. In practice, this means that without changing the interpreter and manually ensuring that the Python 3.11 environment is fully available, apps run through uWSGI do not work. So, the uWSGI plugin should in general always build against the default Python IMHO. - -nik [1] https://lists.debian.org/debian-python/2023/01/msg00010.html - -- System Information: Debian Release: bookworm/sid APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 6.0.0-6-amd64 (SMP w/8 CPU threads; PREEMPT) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=nb_NO.UTF-8, LC_CTYPE=nb_NO.UTF-8 (charmap=UTF-8), LANGUAGE=nb_NO:nb:no_NO:no:nn_NO:nn:da:sv:en Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages uwsgi-plugin-python3 depends on: ii libc6 2.36-8 ii libpython3.11 3.11.1-2 ii uwsgi-core 2.0.21-3+b1 uwsgi-plugin-python3 recommends no packages. Versions of packages uwsgi-plugin-python3 suggests: pn python3-uwsgidecorators - -- no debconf information -BEGIN PGP SIGNATURE- iMAEARYKAGgWIQSk6zxRYJYchegBkTEK5VTlRg4b3QUCY8aedTEaaHR0cHM6Ly93 d3cuZG9taW5pay1nZW9yZ2UuZGUvZ3BnLXBvbGljeS50eHQuYXNjGBxuYXR1cmVz aGFkb3dAZGViaWFuLm9yZwAKCRAK5VTlRg4b3ZDoAQCYW8oE4ZgiBKkgo1lge2Az 7/qTIXGHgKAAF5kmuGTB5QD+NiuAOboj6I6ZvxRZF4o1D3vXCBr1HkqYz+piZMQO Fgc= =Y+XX -END PGP SIGNATURE-
Re: Old generated binary dependencies after renaming psycopg
On Tue, Jan 17, 2023 at 09:08:01AM +0100, Tomasz Rybak wrote: > Hello. > After fixing #1016031 "psycopg3: binary package name should be python3- > psycopg" > (I renamed package names, full changes: > * python3-psycopg3 -> python3-psycopg > * python3-psycopg3-pool -> python3-psycopg-pool > * python-psycopg3-doc -> python3-psycopg-doc) > I tried to rebuild reverse dependencies, > i.e. pgcli and python3-pgspecial. > Rebuild went without problems, new packages are the same > as old ones, but their binary packages still depend on python3- > psycopg3, > even though they build-depend on python3-psycopg. Nope, pgcli does not build-depend on it, rather it explicitly specifies Depends: python3-psycopg3. Likewise, python-pgspecial specifies the same Depends (though it also has a Build-Depends: python3-psycopg3). These packages will need their dependencies updating. (You might also consider making python3-psycopg Provides: python3-psycopg3 and likewise for the other two binary packages for bookworm.) Best wishes, Julian
Old generated binary dependencies after renaming psycopg
Hello. After fixing #1016031 "psycopg3: binary package name should be python3- psycopg" (I renamed package names, full changes: * python3-psycopg3 -> python3-psycopg * python3-psycopg3-pool -> python3-psycopg-pool * python-psycopg3-doc -> python3-psycopg-doc) I tried to rebuild reverse dependencies, i.e. pgcli and python3-pgspecial. Rebuild went without problems, new packages are the same as old ones, but their binary packages still depend on python3- psycopg3, even though they build-depend on python3-psycopg. I analysed build process and this dependency comes from *.substvars file (e.g. pgcli/debian/pgcli.substvars) which is generated by dh-python. Apt search shows both python3-psycopg3 (in old version) and python3-psycopg, so I suspect the former gets taken in as binary dependency to be used. What can I do to fix it? Should I ask ftp-masters to remove old packages, or will the problem fix itself in the next few days? Best regards. -- Tomasz Rybak, Debian Developer GPG: A565 CE64 F866 A258 4DDC F9C7 ECB7 3E37 E887 AA8C signature.asc Description: This is a digitally signed message part
