Re: Bug#1043240: transition: pandas 1.5 -> 2.1
On 11.12.23 08:12, Matthias Klose wrote: On 10.12.23 14:06, Rebecca N. Palmer wrote: Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. I just nmued pyrle and sorted-nearest, having dependencies on cython3-legacy, letting the pyranges autopkg tests fail. Once this succeeds, pandas should be able to migrate.
Re: Bug#1043240: transition: pandas 1.5 -> 2.1
On 10.12.23 14:06, Rebecca N. Palmer wrote: I'd like to move forward with the pandas 1.5 -> 2.1 transition reasonably soon. Given that pandas 2.x is *not* required for Python 3.12 (but is required for Cython 3.0), should we wait for the Python 3.12 transition to be done first? These are broken by pandas 2.x and have a possible (but untested) fix in their bug - please test and apply it: dask(?) dials influxdb-python* python-altair python-feather-format python-upsetplot seaborn tqdm* (* = this package is currently also broken for a non-pandas reason, probably Python 3.12, that I don't have a fix for) These are broken by pandas 2.x and have no known-to-me fix: augur cnvkit dyda emperor esda mirtop pymatgen pyranges python-anndata python-biom-format python-cooler python-nanoget python-skbio python-ulmo q2-quality-control q2-demux q2-taxa q2-types q2templates sklearn-pandas Some generic things to try are pandas.util.testing -> pandas.testing, .iteritems() -> .items(), and if one exists, a more recent upstream version. Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. Is there a way to see the binNMUs which are still stuck in unstable, and don't migrate? Matthias
Re: Preparing for Python 3.12
python3-defaults, adding Python 3.12 as a supported version, is now in unstable. We don't expect to block other transitions while working on the first step of getting to 3.12. We probably will see some Python related build failures while binNMUing around 600 packages to add 3.12 as a supported version. If you see builds failing because of a missing 3.12 extension, please just wait a few days until all the binNMUs are done. For the progress, see (ignoring the 'unknown' status) https://release.debian.org/transitions/html/python3.12-add.html Matthias On 07.11.23 11:27, Matthias Klose wrote: Python 3.12 was released a month ago, and it's time to prepare for the update in unstable, first adding 3.12 as a supported version. There s a tracker for adding 3.12 as a supported version [1], also there are the first bug reports filed for issues related to 3.12 [2]. As usual, it's difficult to find about issues in higher stages before building packages in lower/earlier stages of the transition. Therefore we started again adding 3.12 in Ubuntu, and then filing and fixing issues in unstable before adding 3.12 in Debian unstable. This Ubuntu tracker can be seen at [3]. Note that i386 is only a partial architecture, and that Ubuntu doesn't run the tests on riscv64 during the build (so packages succeeding to build on riscv64 but not on the other architectures most likely show test failures instead of build failures). Ubuntu's update_excuses for python3-defaults also shows autopkg tests failing with 3.12 supported, although this information is a bit out of date, due to infrastructure issues for the autopkg testers. The plan is to make 3.12 supported in unstable at the end of November, or earlier if possible, so that other transitions aren't blocked by the addition of 3.12. Then planning for the defaults change in January. While this timeline is not that much needed for 3.12, it will be a good exercise for 3.13, so that we get 3.13 as the default into the trixie release. Matthias [1] https://bugs.debian.org/1055085 [2] https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.12;users=debian-pyt...@lists.debian.org [3] https://ubuntu-archive-team.ubuntu.com/transitions/html/python3.12-add-v2.html [4] https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#python3-defaults
Re: Python 3.9 for bullseye
On 11/9/20 10:19 AM, Matthias Klose wrote: > On 10/23/20 1:07 PM, Matthias Klose wrote: >> On 10/18/20 12:13 PM, Matthias Klose wrote: >>> Python 3.9 as a supported Python3 version is now in unstable, and all >>> binNMUs >>> are done (thanks to Graham for the work). Bug reports should be all filed >>> for >>> all known problems [1], and the current state of the 3.9 addition can be >>> seen at >>> [2] (a few of the "bad" are false packages with b-d n python3-all-dev, but >>> not >>> building for 3.9, bug reports also filed). >>> >>> The major outstanding issue is the pandas stack, all other problems are >>> found in >>> leaf packages (leaf in the sense of that no other package for the 3.9 >>> addition >>> is blocked). >>> >>> Please help fixing the remaining issues! >>> >>> Matthias >>> >>> [1] >>> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org >>> [2] https://release.debian.org/transitions/html/python3.9.html >> >> Going on with a test rebuild of 3.9 as default to file bug reports for more >> packages. The first stage1 packages for 3.9 as default [1] can be found at >> >> deb [trusted=yes] http://people.debian.org/~doko/tmp/python3.9 ./ >> deb [trusted=yes] http://people.debian.org/~ginggs/python3.9-repo ./ >> >> The first repo just having the python3-defaults packages from experimental. >> The >> second repo of course will be outdated very soon ... Bug reports for stage1 >> are >> filed, Graham is now running the test builds for stage2. >> >> The autopkg test results at [2] need checking. There's currently a britney >> bug >> which marks things as bad, and only gets it right after a week. Plus there's >> no >> way to select an "unrelated" package from unstable for a test, and have it >> marked as a successful test. So basically you need to wait until all the 3.9 >> related fixes migrate to testing for running a successful autopkg test. >> >> Matthias >> >> [1] https://release.debian.org/transitions/html/python3.9-default.html >> [2] https://tracker.debian.org/pkg/python3-defaults > > - python3-defaults now migrated to testing. The following packages were >removed from testing with fastened hints. python3-defaults 3.9.0-3 (supporting 3.8 and 3.9, defaulting to 3.9) is now in testing. python3-defaults 3.9.0-4 (supporting only 3.9, defaulting to 3.9) is now in unstable. After that version migrates to testing, we'll do the binNMUs to drop the extensions for 3.8 (this way we avoid testing against 3.8 again). python3-defaults 3.9.1-1 is expected next week with the upstream Python 3.9.1 release. We are not yet finished, still having the list list of RC issues at [1]. If you think that your package needs a break, because it is likely to break with partial upgrades, leave a message at https://bugs.debian.org/976655. I'll add these breaks as we had them for 3.7 (but didn't have them for 3.8). Maybe now is also the time to look at packages with outdated upstream versions https://qa.debian.org/developer.php?email=python-modules-team%40lists.alioth.debian.org lists 30-40% outdated source packages. https://qa.debian.org/developer.php?email=team%2Bpython%40tracker.debian.org looks a bit better. Matthias [1] https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org
Re: Python 3.9 for bullseye
On 11/11/20 3:27 AM, Michael Hudson-Doyle wrote: > On Mon, 9 Nov 2020 at 22:20, Matthias Klose wrote: > >> On 10/23/20 1:07 PM, Matthias Klose wrote: >>> On 10/18/20 12:13 PM, Matthias Klose wrote: >>>> Python 3.9 as a supported Python3 version is now in unstable, and all >> binNMUs >>>> are done (thanks to Graham for the work). Bug reports should be all >> filed for >>>> all known problems [1], and the current state of the 3.9 addition can >> be seen at >>>> [2] (a few of the "bad" are false packages with b-d n python3-all-dev, >> but not >>>> building for 3.9, bug reports also filed). >>>> >>>> The major outstanding issue is the pandas stack, all other problems are >> found in >>>> leaf packages (leaf in the sense of that no other package for the 3.9 >> addition >>>> is blocked). >>>> >>>> Please help fixing the remaining issues! >>>> >>>> Matthias >>>> >>>> [1] >>>> >> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org >>>> [2] https://release.debian.org/transitions/html/python3.9.html >>> >>> Going on with a test rebuild of 3.9 as default to file bug reports for >> more >>> packages. The first stage1 packages for 3.9 as default [1] can be found >> at >>> >>> deb [trusted=yes] http://people.debian.org/~doko/tmp/python3.9 ./ >>> deb [trusted=yes] http://people.debian.org/~ginggs/python3.9-repo ./ >>> >>> The first repo just having the python3-defaults packages from >> experimental. The >>> second repo of course will be outdated very soon ... Bug reports for >> stage1 are >>> filed, Graham is now running the test builds for stage2. >>> >>> The autopkg test results at [2] need checking. There's currently a >> britney bug >>> which marks things as bad, and only gets it right after a week. Plus >> there's no >>> way to select an "unrelated" package from unstable for a test, and have >> it >>> marked as a successful test. So basically you need to wait until all >> the 3.9 >>> related fixes migrate to testing for running a successful autopkg test. >>> >>> Matthias >>> >>> [1] https://release.debian.org/transitions/html/python3.9-default.html >>> [2] https://tracker.debian.org/pkg/python3-defaults >> >> - python3-defaults now migrated to testing. The following packages were >>removed from testing with fastened hints. >> >>numba >>python-executing >>python-fabio >>python-icecream >>python-molotov >>supysonic >> >> - The Python related ftbfs issues from the last archive test rebuild >>were user-tagged with 'python3.9', although I didn't make much >>effort to determine if these are ftbfs seen for 3.9 only, or for >>both 3.8 and 3.9. dh-python now supports building and testing for >>all supported python version before bailing out in case of errors, >>but this came too late for the test rebuild. >> >>issues for key packages (those with lots of dependencies) are: >> >> https://bugs.debian.org/973056 src:sphinx-tabs >> > > Fixed. > > >> https://bugs.debian.org/973057 src:python-py >> > > Fixed. > > >> https://bugs.debian.org/973061 src:nototools >> > > Pasted a trivial fix to the bug. I guess it could be NMUed (it's not a > Python team package). > > >> https://bugs.debian.org/973072 src:python-kubernetes >> > > I patched the failures (see the bug) but then the build hangs for me. > > >> https://bugs.debian.org/973087 src:python-fs >> > > Fixed, but I forgot to put the Closes: #xxx in the changelog. > > >> https://bugs.debian.org/973114 src:python-future >> https://bugs.debian.org/973121 src:cairocffi >> https://bugs.debian.org/973126 src:responses >> https://bugs.debian.org/973134 src:python-webob >> https://bugs.debian.org/973165 src:pyflakes >> > > Fixed. > > >> https://bugs.debian.org/973167 src:ufonormalizer >> https://bugs.debian.org/973168 src:pylint >> > > This looks confusing! Upstream is thinking about it but I'm not sure what > their ETA is. > > >> https://bugs.debian.org/973193 src:parso >> https://bugs.debian.org/973195 src:python-asyncssh >> https://bugs.debian.
Re: Python 3.9 for bullseye
On 10/23/20 1:07 PM, Matthias Klose wrote: > On 10/18/20 12:13 PM, Matthias Klose wrote: >> Python 3.9 as a supported Python3 version is now in unstable, and all binNMUs >> are done (thanks to Graham for the work). Bug reports should be all filed >> for >> all known problems [1], and the current state of the 3.9 addition can be >> seen at >> [2] (a few of the "bad" are false packages with b-d n python3-all-dev, but >> not >> building for 3.9, bug reports also filed). >> >> The major outstanding issue is the pandas stack, all other problems are >> found in >> leaf packages (leaf in the sense of that no other package for the 3.9 >> addition >> is blocked). >> >> Please help fixing the remaining issues! >> >> Matthias >> >> [1] >> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org >> [2] https://release.debian.org/transitions/html/python3.9.html > > Going on with a test rebuild of 3.9 as default to file bug reports for more > packages. The first stage1 packages for 3.9 as default [1] can be found at > > deb [trusted=yes] http://people.debian.org/~doko/tmp/python3.9 ./ > deb [trusted=yes] http://people.debian.org/~ginggs/python3.9-repo ./ > > The first repo just having the python3-defaults packages from experimental. > The > second repo of course will be outdated very soon ... Bug reports for stage1 > are > filed, Graham is now running the test builds for stage2. > > The autopkg test results at [2] need checking. There's currently a britney bug > which marks things as bad, and only gets it right after a week. Plus there's > no > way to select an "unrelated" package from unstable for a test, and have it > marked as a successful test. So basically you need to wait until all the 3.9 > related fixes migrate to testing for running a successful autopkg test. > > Matthias > > [1] https://release.debian.org/transitions/html/python3.9-default.html > [2] https://tracker.debian.org/pkg/python3-defaults - python3-defaults now migrated to testing. The following packages were removed from testing with fastened hints. numba python-executing python-fabio python-icecream python-molotov supysonic - The Python related ftbfs issues from the last archive test rebuild were user-tagged with 'python3.9', although I didn't make much effort to determine if these are ftbfs seen for 3.9 only, or for both 3.8 and 3.9. dh-python now supports building and testing for all supported python version before bailing out in case of errors, but this came too late for the test rebuild. issues for key packages (those with lots of dependencies) are: https://bugs.debian.org/973056 src:sphinx-tabs https://bugs.debian.org/973057 src:python-py https://bugs.debian.org/973061 src:nototools https://bugs.debian.org/973072 src:python-kubernetes https://bugs.debian.org/973087 src:python-fs https://bugs.debian.org/973114 src:python-future https://bugs.debian.org/973121 src:cairocffi https://bugs.debian.org/973126 src:responses https://bugs.debian.org/973134 src:python-webob https://bugs.debian.org/973165 src:pyflakes https://bugs.debian.org/973167 src:ufonormalizer https://bugs.debian.org/973168 src:pylint https://bugs.debian.org/973193 src:parso https://bugs.debian.org/973195 src:python-asyncssh https://bugs.debian.org/973239 src:python-fixtures For the other ftbfs, see [1]. Matthias [1] https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org
Re: Python 3.9 for bullseye
On 10/23/20 1:07 PM, Matthias Klose wrote: > On 10/18/20 12:13 PM, Matthias Klose wrote: >> Python 3.9 as a supported Python3 version is now in unstable, and all binNMUs >> are done (thanks to Graham for the work). Bug reports should be all filed >> for >> all known problems [1], and the current state of the 3.9 addition can be >> seen at >> [2] (a few of the "bad" are false packages with b-d n python3-all-dev, but >> not >> building for 3.9, bug reports also filed). >> >> The major outstanding issue is the pandas stack, all other problems are >> found in >> leaf packages (leaf in the sense of that no other package for the 3.9 >> addition >> is blocked). >> >> Please help fixing the remaining issues! >> >> Matthias >> >> [1] >> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org >> [2] https://release.debian.org/transitions/html/python3.9.html > > Going on with a test rebuild of 3.9 as default to file bug reports for more > packages. The first stage1 packages for 3.9 as default [1] can be found at > > deb [trusted=yes] http://people.debian.org/~doko/tmp/python3.9 ./ > deb [trusted=yes] http://people.debian.org/~ginggs/python3.9-repo ./ > > The first repo just having the python3-defaults packages from experimental. > The > second repo of course will be outdated very soon ... Bug reports for stage1 > are > filed, Graham is now running the test builds for stage2. > > The autopkg test results at [2] need checking. There's currently a britney bug > which marks things as bad, and only gets it right after a week. Plus there's > no > way to select an "unrelated" package from unstable for a test, and have it > marked as a successful test. So basically you need to wait until all the 3.9 > related fixes migrate to testing for running a successful autopkg test. > > Matthias > > [1] https://release.debian.org/transitions/html/python3.9-default.html > [2] https://tracker.debian.org/pkg/python3-defaults Update: - Graham finished the test rebuilds with 3.9 as the default, and the archive on p.d.o is updated. We don't plan to keep that up-to-date. Bug reports for ftbfs are filed. - Bug reports are also filed for all failing autopkg tests triggered by the python3-defaults upload. - Lucas made an archive-wide test rebuild for unstable, adding more RC ftbfs issues, mostly for python binary-indep (all) packages. These don't have the python3.9 user tag. Matthias
Re: Python 3.9 for bullseye
On 10/18/20 12:13 PM, Matthias Klose wrote: > Python 3.9 as a supported Python3 version is now in unstable, and all binNMUs > are done (thanks to Graham for the work). Bug reports should be all filed > for > all known problems [1], and the current state of the 3.9 addition can be seen > at > [2] (a few of the "bad" are false packages with b-d n python3-all-dev, but not > building for 3.9, bug reports also filed). > > The major outstanding issue is the pandas stack, all other problems are found > in > leaf packages (leaf in the sense of that no other package for the 3.9 addition > is blocked). > > Please help fixing the remaining issues! > > Matthias > > [1] > https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org > [2] https://release.debian.org/transitions/html/python3.9.html Going on with a test rebuild of 3.9 as default to file bug reports for more packages. The first stage1 packages for 3.9 as default [1] can be found at deb [trusted=yes] http://people.debian.org/~doko/tmp/python3.9 ./ deb [trusted=yes] http://people.debian.org/~ginggs/python3.9-repo ./ The first repo just having the python3-defaults packages from experimental. The second repo of course will be outdated very soon ... Bug reports for stage1 are filed, Graham is now running the test builds for stage2. The autopkg test results at [2] need checking. There's currently a britney bug which marks things as bad, and only gets it right after a week. Plus there's no way to select an "unrelated" package from unstable for a test, and have it marked as a successful test. So basically you need to wait until all the 3.9 related fixes migrate to testing for running a successful autopkg test. Matthias [1] https://release.debian.org/transitions/html/python3.9-default.html [2] https://tracker.debian.org/pkg/python3-defaults
Python 3.9 for bullseye
Python 3.9 as a supported Python3 version is now in unstable, and all binNMUs are done (thanks to Graham for the work). Bug reports should be all filed for all known problems [1], and the current state of the 3.9 addition can be seen at [2] (a few of the "bad" are false packages with b-d n python3-all-dev, but not building for 3.9, bug reports also filed). The major outstanding issue is the pandas stack, all other problems are found in leaf packages (leaf in the sense of that no other package for the 3.9 addition is blocked). Please help fixing the remaining issues! Matthias [1] https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org [2] https://release.debian.org/transitions/html/python3.9.html
Re: Timing of Python (3.9) upstream and Debian releases
On 7/6/20 9:04 PM, Matthias Klose wrote: > Starting with Python 3.8, Python upstream changed to a time based yearly > release > schedule, targeting the first release of a major Python version (3.x) for > October of each year. For the transition to 3.8: > > - we add 3.8 as supported in November > - made 3.8 the default in March > - dropped 3.7 in April > > That's a little bit late looking at the Debian release schedule, so a little > speedup would be needed if we want to target the current Python release for > the > current Debian release. For 3.8 Ubuntu started to prepare the switch a bit > earlier, using the results for a better experience in Debian: > > - added 3.8 in mid October > - made 3.8 the default in mid December > > Technically it would be possible to do the defaults change before the Debian > freeze, however there's usually a tail of follow-up upstream releases required > to support a new major Python version, unless you opt for actively backporting > changes in various packages. Having the confidence, that the switch is > feasible > in another distro certainly helps doing the transition in Debian, however it > adds a delay. I don't think it's feasible to do the transition in > experimental > first, or doing a large test rebuild, because it requires keeping the whole > python stack in sync with testing/unstable. > > So what I'm proposing here is to aim to support 3.9 as early as possible as a > supported Python3 version, starting with the 3.9 upstream release, and fixing > stuff on the go. Then decide in November, if we can do the defaults change to > 3.9, or if we drop 3.9 again, or ship with two supported Python3 versions. > > Please note this will be a re-occuring situation with Python 3.11 and > bullseye+1, so we should find out how to handle this on a regular basis, > assuming that Debian release schedules won't change much. 3.9 was released last week and migrated to testing. I did a test rebuild with 3.9 as a supported Python3 version, however not for unstable, but for the current Ubuntu development version. Of course there is some version skew, but it should give an idea what still needs work. Bugs are filed at https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9;users=debian-pyt...@lists.debian.org which definitely looks better than the addition of 3.8 last year. The PPAs used are based on groovy: https://launchpad.net/~pythoneers/+archive/ubuntu/python3.9 https://launchpad.net/~pythoneers/+archive/ubuntu/python3.9-fixes The first PPA just has no-change uploads, while the second one has some sourceful uploads. If you want to see some update in these PPAs, please ask Emmanuel Arias, Graham Inggs or myself. I'm planning to upload python3-defaults from experimental to unstable this Wednesday (Oct 14). We will see some days of instability as we are doing the binNMUs. Matthias
GCC and binutils plans for bullseye
Debian bullseye will be based on a gcc-10 package taken from the gcc-10 upstream branch, and binutils based on a binutils package taken from the 2.35 branch. I'm planning to make gcc-10 the default after gcc-10 (10.2.0) is available (upstream targets mid July). binutils will be updated before making the GCC switch. The GCC 10 switch involves some minor library transitions for D, gccgo, M2, which should be no-brainers. The gnat transition will be handled separately by the debian Ada maintainers. binutils should be pretty stable until the bullseye release, not planning an update to 2.36. GCC 10 should be updated to 10.3, or close to 10.3 (the release date is not yet known, could be Feb 2021). I'd like to get rid off GCC 8 and GCC 9 for the bullseye release. Matthias
Re: Bug#949187: transition: python3.8
On 2/3/20 8:22 PM, Simon McVittie wrote: > On Sun, 02 Feb 2020 at 09:35:04 +0100, Matthias Klose wrote: >> I think this is now in shape to be started. > > Please can this wait until the remaining bits of the libffi7 transition > and the restructuring of the libgcc_s packaging have settled down? > > I'm still trying to sort out the missing Breaks around > gobject-introspection, as highlighted by autopkgtest failures: this has > been delayed by needing coordinated action between multiple packages, > some of them quite big (glib2.0), and by Paul and I being at FOSDEM. > This is entangled with python3.8 via pygobject (which will fail tests > with python3.8 as default - an upload is pending to fix that). fine. I'm happy to see that fixed. > Meanwhile, multiple packages seem to FTBFS on s390x with the new libgcc_s > (I've just opened the bug for that, so no bug number known yet), which is > going to limit the ability to get things into testing. please retry your package builds. it's fixed in unstable.
Re: Bug#949187: transition: python3.8
Control: tags -1 - moreinfo On 1/18/20 9:30 PM, Paul Gevers wrote: > Control: tags -1 moreinfo > > Hi Matthias, > > On 17-01-2020 23:28, Matthias Klose wrote: >> Please add a transition tracker to switch the python3 default to 3.8. It's >> not >> yet ready, however it would be good to see affected packages. Please copy it >> from the 3.7 defaults change if possible. > > Tracker is in place. Please remove the moreinfo tag when you're ready. I think this is now in shape to be started. bug reports have been filed for issues with 3.8 in [1], and there may be some which are missing the proper tagging. I also filed bug reports for a dozen missing dh-python build dependencies. I would prefer a transition slot with less activity with transitions in the science area, like opencv, openmpi, gnuradio, ros*, and others which only build extensions for the default python3 version. Matthias [1] https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.8;users=debian-pyt...@lists.debian.org
Re: Adding Python 3.8 as a supported Python3 version
On 18.11.19 07:52, Alastair McKinstry wrote: > So I've uploaded a python-xarray 0.14.0-2 which passes on python3.7. > > Its failing on python 3.8 but apparently due to pandas not ready on 3.8, so it > should be ok to pass as the transition completes. > > xarray really needs dask >= 2.0 but version 1 is in unstable; I've disabled > some > functionality (dask='parallelize') and marked some tests xfail until this is > done. > > Alastair yes, people are aware of a needed dask update, see #942235. There's now also a dask 2.8 release available, I didn't look at that yet. > On 07/11/2019 14:08, Matthias Klose wrote: >> This weekend, I am planning to upload python3-defaults, adding python3.8 as a >> supported Python3 version. This may introduce some churn in unstable until >> the basic binNMUs are available as well. >> >> >> - python-xarray #944044. 1.4 is already broken with Python 3.7. For >> Ubuntu I fell back to 1.2.1. 1.2.3 might work as well, still seeing >> one or two test failures. >> >> Packages building on top of numpy/scipy/pandas, like the PyAstro stack, >> continue to work with these updates, despite some new test failures. >> >> Matthias >> >> [1] https://wiki.debian.org/Python/Python3.8. >> [2] >> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.8;users=debian-pyt...@lists.debian.org >> >>
Re: Bug#942106: Adding Python 3.8 as a supported Python3 version
On 12.11.19 23:39, Matthias Klose wrote: > On 07.11.19 15:08, Matthias Klose wrote: >> This weekend, I am planning to upload python3-defaults, adding python3.8 as a >> supported Python3 version. This may introduce some churn in unstable until >> the >> basic binNMUs are available as well. >> >> Details for the addition can be found at [1], known issues and patches are >> filed >> [2]. There was no test rebuild in Debian itself, but the addition was made >> in >> the current Ubuntu development series. Things look good, and from my point >> of >> view don't block any unrelated transition work. python3-defaults will get a >> RC >> issue to stay in unstable until the packages build-depending on >> python3-all-dev >> are built, and after doing a test rebuild with the two supported Python3 >> versions. Earlier test rebuilds don't make sense. >> >> There are a few packages, where the upstream versions used in Ubuntu diverge >> from the ones currently in unstable (not naming those already updated in >> unstable by the DPMT): >> >> - hypothesis #942693, not sure if this is really needed, >> the testsuite stack might be fixed by the new pluggy >> version as well. >> >> - python-xarray #944044. 1.4 is already broken with Python 3.7. For >> Ubuntu I fell back to 1.2.1. 1.2.3 might work as well, still seeing >> one or two test failures. >> >> - Using numpy from experimental, and only building the Python2 numpy >> packages from the python-numpy source. >> >> - Using python-scipy from experimental, building the Python3 packages, >> and renaming the python-scipy package to python2-scipy, only building >> the Python2 packages. >> >> - matplotlib and pandas don't have Python2 packages in Ubuntu >> anymore, so I can't tell much what is needed here. Pandas needs >> a new upstream for 3.8. >> >> Packages building on top of numpy/scipy/pandas, like the PyAstro stack, >> continue >> to work with these updates, despite some new test failures. > > So this upload didn't happen, but thanks to Rebecca we now have an almost > Python2 free pandas in unstable. And apologies to the science team for the > 1-day > delayed NMUs for patsy and scikit-learn. > > Now planning the python3-defaults upload for Thursday or Friday. python3-defaults with 3.8 as a supported Python3 version is now built in unstable. Release team, please schedule binNMUs fpr stage1 and then stage2. I'll raise severity of the 3.8 issues, and follow-up with 2-days delayed NMUs. Matthias
Re: Adding Python 3.8 as a supported Python3 version
On 07.11.19 15:08, Matthias Klose wrote: > This weekend, I am planning to upload python3-defaults, adding python3.8 as a > supported Python3 version. This may introduce some churn in unstable until > the > basic binNMUs are available as well. > > Details for the addition can be found at [1], known issues and patches are > filed > [2]. There was no test rebuild in Debian itself, but the addition was made in > the current Ubuntu development series. Things look good, and from my point of > view don't block any unrelated transition work. python3-defaults will get a > RC > issue to stay in unstable until the packages build-depending on > python3-all-dev > are built, and after doing a test rebuild with the two supported Python3 > versions. Earlier test rebuilds don't make sense. > > There are a few packages, where the upstream versions used in Ubuntu diverge > from the ones currently in unstable (not naming those already updated in > unstable by the DPMT): > > - hypothesis #942693, not sure if this is really needed, > the testsuite stack might be fixed by the new pluggy > version as well. > > - python-xarray #944044. 1.4 is already broken with Python 3.7. For > Ubuntu I fell back to 1.2.1. 1.2.3 might work as well, still seeing > one or two test failures. > > - Using numpy from experimental, and only building the Python2 numpy > packages from the python-numpy source. > > - Using python-scipy from experimental, building the Python3 packages, > and renaming the python-scipy package to python2-scipy, only building > the Python2 packages. > > - matplotlib and pandas don't have Python2 packages in Ubuntu > anymore, so I can't tell much what is needed here. Pandas needs > a new upstream for 3.8. > > Packages building on top of numpy/scipy/pandas, like the PyAstro stack, > continue > to work with these updates, despite some new test failures. So this upload didn't happen, but thanks to Rebecca we now have an almost Python2 free pandas in unstable. And apologies to the science team for the 1-day delayed NMUs for patsy and scikit-learn. Now planning the python3-defaults upload for Thursday or Friday. Matthias
Re: Bug#942106: python3.8 / pandas py2removal
On 10.11.19 14:46, Rebecca N. Palmer wrote: mdp isn't in testing either, but if you're using a policy of "no py2removals that break packages in testing", tnseq-transit (Depends: statsmodels) and possibly stimfit (Recommends: pandas) need to be done as well. Those are both thought to need new upstream versions. tnseq-transit is a leaf package, raising the severity now, could be removed and re-enter later. IMO, we should care about the recommends later ... (patsy isn't a leaf package for py2removal, it's in a cycle with pandas and statsmodels, i.e. for a non-breaking removal they all have to go together.) ok I'm undoing the NMU from the delayed queue, you'll find it at https://people.debian.org/~doko/tmp/
Re: Bug#942106: python3.8 / pandas py2removal
On 10.11.19 14:22, Matthias Klose wrote: patsy is a leaf package, no problem. scikit-learn, needs mdp, pymvpa2, I think that's manageable, so I'm volunteering to do the NMUs, preferably with a 0 or 1 day delay. PyMVPA has other RC issues, is removed from testing, so ignore it for now. and I'm raising the severity of the py2removal bug.
Re: Bug#942106: python3.8 / pandas py2removal
[CCing debian-science] On 10.11.19 13:18, Rebecca N. Palmer wrote: Matthias Klose wrote: yes, please do [raise pandas 0.25 blocking bugs to "important"] Done, but only 2 of them have been fixed since. This leaves 13: has patch or Ubuntu fix: matplotlib2 patsy python-apptools scikit-learn may need more extensive work: cnvkit python-feather-format python-skbio stimfit tnseq-transit already not in testing: mdp psychopy pymvpa q2-types If you can get that done with [pandas] 0.25, fine, It took longer than I was expecting, but pandas 0.25 / statsmodels 0.10 now appear to be working, including with python3.8. (Though we won't actually know if #943732 is gone until mipsel tries to build it.) \o/ and I wouldn't worry too much about the other four breaking packages at this point. Was this intended as... ... a request to upload pandas 0.25 / statsmodels 0.10 to unstable (and apply the Ubuntu py2removal patches to patsy and scikit-learn?), overriding normal py2removal rules? patsy is a leaf package, no problem. scikit-learn, needs mdp, pymvpa2, I think that's manageable, so I'm volunteering to do the NMUs, preferably with a 0 or 1 day delay. So yes, please go ahead. ... a request to split pandas into a pandas2 0.23 and a pandas 0.25? (since 4 is only the number of non-py2removal breakages, and the wiki page https://wiki.debian.org/Python/Python3.8 says to do this if 2.7 and 3.8 need different upstream versions) Should be technically easy, but means going through NEW. ... a statement that once pandas 0.25 works, this is no longer my problem, i.e. that I don't have to fix 0.23? I don't think that's necessary at this point. matplotlib and pandas don't have Python2 packages in Ubuntu anymore, so I can't tell much what is needed here matplotlib has already been split into Python 2 and Python 3 source packages. (matplotlib2 is in Ubuntu, and unbuildable there due to #943924.) Python2 matplotlib are already removed in Ubuntu, I shouldn't have synced matplotlib2 from experimental. According to its Ubuntu build log: https://launchpad.net/ubuntu/+source/matplotlib/3.0.2-2ubuntu1/+build/17942804 matplotlib has one python3.8-specific test failure, test_axes.py::test_pathological_hexbin. This is currently being ignored (#942766) along with (many) errors that also happen on 3.7.
Adding Python 3.8 as a supported Python3 version
This weekend, I am planning to upload python3-defaults, adding python3.8 as a supported Python3 version. This may introduce some churn in unstable until the basic binNMUs are available as well. Details for the addition can be found at [1], known issues and patches are filed [2]. There was no test rebuild in Debian itself, but the addition was made in the current Ubuntu development series. Things look good, and from my point of view don't block any unrelated transition work. python3-defaults will get a RC issue to stay in unstable until the packages build-depending on python3-all-dev are built, and after doing a test rebuild with the two supported Python3 versions. Earlier test rebuilds don't make sense. There are a few packages, where the upstream versions used in Ubuntu diverge from the ones currently in unstable (not naming those already updated in unstable by the DPMT): - hypothesis #942693, not sure if this is really needed, the testsuite stack might be fixed by the new pluggy version as well. - python-xarray #944044. 1.4 is already broken with Python 3.7. For Ubuntu I fell back to 1.2.1. 1.2.3 might work as well, still seeing one or two test failures. - Using numpy from experimental, and only building the Python2 numpy packages from the python-numpy source. - Using python-scipy from experimental, building the Python3 packages, and renaming the python-scipy package to python2-scipy, only building the Python2 packages. - matplotlib and pandas don't have Python2 packages in Ubuntu anymore, so I can't tell much what is needed here. Pandas needs a new upstream for 3.8. Packages building on top of numpy/scipy/pandas, like the PyAstro stack, continue to work with these updates, despite some new test failures. Matthias [1] https://wiki.debian.org/Python/Python3.8. [2] https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.8;users=debian-pyt...@lists.debian.org
GCC 9 / gfortran changes?
Alastair, please could you have a look if anything special needs to be done with the GC defaults change to 9 on the Fortran side? Thanks, Matthias
Re: dropping python2 [was Re: scientific python stack transitions]
On 07.07.19 16:55, Drew Parsons wrote: > On 2019-07-07 22:46, Mo Zhou wrote: >> Hi science team, >> >> By the way, when do we start dropping python2 support? >> The upstreams of the whole python scientific computing >> stack had already started dropping it. > > Good question. I think it is on the agenda this cycle, but debian-python will > have the call on it. you can start dropping it now, however please don't drop anything yet with reverse dependencies. So leaf packages first. Matthias
Re: Python 3 Statsmodels & Pandas
On 16.09.2017 22:59, Yuri D'Elia wrote: > On Sat, Sep 16 2017, Diane Trout wrote: >> python3-pandas: Pandas is not installable >> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=875723 > > I would have expected the rebuild of python packages affected by the > fpectl extension with a transition, but it doesn't seem to be the case? no, we don't want to rename the python2.7 package. > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=874253 > > More Breaks where added to {python,python3}-stdlib itself, but there are > still packages which didn't rebuild. fix them, they fail for unrelated reasons ;)
Re: Bug#729956: Forwarded upstream
On 06.09.2017 23:45, Diane Trout wrote: > I was trying to build it right now but I'm getting a dependency error. > > libpython2.7-stdlib : Breaks: python-pandas-lib (<= 0.20.3-1) but > 0.20.3-1 is to be installed this is unrelated, and waiting for #874413.
Re: gcc7 transition and dh_fortran
On 05.07.2017 11:28, Alastair McKinstry wrote: > Hi, > > We will be transitioning to gcc7 soon for Buster, and with it there is > another Fortran transition. This is because the format for gfortrans mod > files has changed again. did it? At least the fortran module version is the same upstream (14). However the libgfortran soname was bumped from 3 to 4, so we will have a transition / rebuild cycle anyway. > We do not have a clean mechanism for tracking gfortran-dependent > packages in Debian. A module dh_fortran_mod for doing so was written by > Sebastian Villemont (see #714730) but this issue is no longer being > pushed by him due to a change in personal direction. > > I propose to resurrect this module and add dependencies to all current > gfortran-using packages so we can track a transition. > > Can anyone comment on current issues with this code, or this approach ? > > > A second issue worth tracking is the 'flang' compiler. This is a new > Apache-licensed Fortran compiler, based on a front-end written by PGI > and Nvidia, using the LLVM backend. While currently not in Debian, it > may be so soon .. > > https://github.com/flang-compiler/flang > > Should compilers be set to check a subdirectory of /usr/include to look > for .mod files suitable for their use? I haven't looked into that, however if compiler patches would be required then please discuss these with upstream first. Matthias
Re: quick Q to Matthias Re: Sympy(+sagemath+nipy+~30 more) removal ahead
sorry, somehow I dropped the b-d. Now re-uploaded. Matthias On 10.01.2017 22:44, Yaroslav Halchenko wrote: > > On Tue, 10 Jan 2017, Rebecca N. Palmer wrote: > >>> missing python{,3}-olefile in Build-Depends. > >> There is no such package in unstable - was this a typo? > > may be confusion is due to my shell rotten fingers typing? ;) > > $> apt-cache policy python{,3}-olefile > python-olefile: > Installed: (none) > Candidate: 0.44-1 > Version table: > 0.44-1 600 > 600 http://http.debian.net/debian sid/main amd64 Packages > 600 http://http.debian.net/debian sid/main i386 Packages > python3-olefile: > Installed: (none) > Candidate: 0.44-1 > Version table: > 0.44-1 600 > 600 http://http.debian.net/debian sid/main amd64 Packages > 600 http://http.debian.net/debian sid/main i386 Packages > > > my guess is that divergence was due to severe novelty of that package: > > olefile (0.44-1) unstable; urgency=medium > > * New upstream release. > > -- Matthias Klose <d...@debian.org> Mon, 09 Jan 2017 12:52:45 +0100 > > olefile (0.43-1) unstable; urgency=medium > > * Initial release (closes: #850404). > > -- Matthias Klose <d...@debian.org> Fri, 06 Jan 2017 07:36:25 +0100 > > >> (For me the only failed tests were the two in the bug, and just the patch >> was enough to fix that.) > > and I guess recent python-pil pruned olefile from the depends (I believe it > was there in prev version) > > although I don't see any corresponding changelog entry for that in > http://metadata.ftp-master.debian.org/changelogs/main/p/pillow/pillow_4.0.0-2_changelog > > Matthias -- could you clarify if olefile depends was consciously removed or > should we complain with a bug report? ;-) unfortunately there is no VCS > defined for python-pil to go through the laundry: > > # apt-cache policy python-pil > python-pil: > Installed: 4.0.0-2 > Candidate: 4.0.0-2 > Version table: > *** 4.0.0-2 500 > 500 http://127.0.0.1:3142/debian sid/main amd64 Packages > 100 /var/lib/dpkg/status > # apt-cache show python-pil | grep olefile > > > $> debcheckout python-pil > No repository found for package python-pil. > >
Bug#790757: transition: gfortran module version 14
Package: release.debian.org Severity: normal User: release.debian@packages.debian.org Usertags: transition The gfortran module version 14 is triggered by making gfortran-5 the default gfortran (which should happen at the same time as the C/C++ default switch). It affects all packages depending on gfortran modules, the previous transition tracker bug https://bugs.debian.org/746805 has a list of packages, which probably need some updates. adios cmor elmerfem etsf-io libgetdata grib-api hdf5 mpich netcdf oasis3 openmpi petsc plplot qd slepc libxc -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5593fb8f.5060...@debian.org
gfortran-5 (fortran90 module version) transition
Hi, at least for GCC 4.8 and GCC 4.9, the debian science team took care about the fortran90 module version transitions. Would it be possible to do the same for GCC 5? The amount of packages to rebuild is small, so please check that out and the extend/correct https://wiki.debian.org/GCC5 Thanks, Matthias -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/557765de.3080...@debian.org
Re: Bug#746805: transition: gfortran module version changing from 10 to 12
Am 24.06.2014 16:55, schrieb Emilio Pozuelo Monfort: On 21/06/14 21:00, Cyril Brulebois wrote: Sébastien Villemot sebast...@debian.org (2014-06-21): Le mardi 17 juin 2014 à 01:15 +0200, Emilio Pozuelo Monfort a écrit : On 12/06/14 12:55, Matthias Klose wrote: Am 12.06.2014 11:07, schrieb Emilio Pozuelo Monfort: Will this need to migrate together with gfortran-4.9 or some other package, or can they migrate one at a time, whenever they are ready? these probably should wait for the gcc-defaults migration. gcc-defaults has migrated now, and so have the binnmus. Shall we close this now, or is there anything else to do here? Could you please give back slepc? It failed to recompile because it build depends on petsc, which was not yet recompiled for the new Fortran. Done; first builds seem successful so far. It's built now, though the package is not in testing anyway. Anything else here? Sorry for asking repeatedly, but I don't know gfortran and the lack of a tracker or proper dependencies doesn't help. from my point of view, this looks complete now. -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53b14e45.7000...@debian.org
Re: Bug#746805: transition: gfortran module version changing from 10 to 12
Am 12.06.2014 11:07, schrieb Emilio Pozuelo Monfort: Will this need to migrate together with gfortran-4.9 or some other package, or can they migrate one at a time, whenever they are ready? these probably should wait for the gcc-defaults migration. -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53998711.9020...@debian.org
Re: Bug#746805: transition: gfortran module version changing from 10 to 12
Alastair, Sebastien, gfortran-4.9 is now the default for all release architectures. Can you proceed with the binNMUs? thanks, Matthias -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5398ae6e.2040...@debian.org
Re: transition: gfortran module version changing from 10 to 12
Am 07.05.2014 15:43, schrieb Sébastien Villemot: Le samedi 03 mai 2014 à 21:18 +0200, Matthias Klose a écrit : Package: release.debian.org GCC 4.9 changes the module version for fortran 90 code from 10 to 12. Shouldn't this transition also include a binNMU of all packages that ship a .mod file? The long term solution is a fix to #714730, but in the short term I guess we should manually generate the list of package to be rebuilt. I am willing to do that if agreed by the Release Team. If you can prepare such a list, please go ahead. -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/536a3a7f.4000...@debian.org
Re: Bug#714730: gfortran: handling binNMU for .mod file format change
Am 24.03.2014 18:46, schrieb Sébastien Villemot: Le mercredi 12 mars 2014 à 18:44 +0100, Sébastien Villemot a écrit : More specifically, I think that dh_fortran_mod should do the following: - it would read a file debian/[package.]fortran-mod, which would contain a list of mod files created by gfortran and to be installed into binary packages - for each of these mod files, it would determine the mod version by inspecting the contents, and then install the file into /usr/include/fortran/gcc-version/multiarch-triplet/. This is the proposed standard location, see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49138 - it would also add the relevant virtual dependencies on gfortran-mod-version in ${misc:Depends} I have implemented this. The code is on alioth: http://anonscm.debian.org/gitweb/?p=debian-science/packages/dh-fortran-mod.git;a=tree git://anonscm.debian.org/debian-science/packages/dh-fortran-mod.git doko: does that sound like a reasonable plan to you? In particular, are you willing to incorporate a patch to gfortran to implement the standard include location? so are the mod files always architecture dependent? If not then maybe we should have two directories. into /usr/include/fortran/gcc-version/multiarch-triplet/. This is we have to change this one to use a location in /usr/include/multiarch-triplet/ like we do for c++, to find the correct files for foreign architectures. do we really need the gcc- prefix? this doesn't seem to be orthogonal to the c++ case. if possible, please test such a patch with a cross compiler too. So we should have /usr/include/multiarch-triplet/fortran/version and if needed /usr/include/fortran/version Matthias -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53653e8a.5030...@debian.org
Bug#746805: transition: gfortran module version changing from 10 to 12
Package: release.debian.org GCC 4.9 changes the module version for fortran 90 code from 10 to 12. A test rebuild of GCC 4.9 only shows good results when a package doesn't build-depend on another package having the same gfortran module version. Asking first the maintainers of gfortran based software to test-rebuild these packages, then giving feedback about the viability of gfortran-4.9 as the default gfortran version for jessie. So please build the build-dependencies of these packages, and then these packages themself with gfortran-4.9. cdftools elkcode flexpart slepc Please follow-up with results to this bug report. -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53654120.2050...@debian.org
Re: Location for Fortran90 module files
Am 25.07.2013 14:25, schrieb Michael Banck: What to do about the compatibilty is a different matter, did this come up before? looks like one hand in the debian-science team doesn't know what the other hands are doing ... See #714730, even on the debian-science ML. -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51f26d7b.7060...@debian.org
Re: Location for Fortran90 module files
Am 26.07.2013 15:35, schrieb Andreas Tille: BTW, it would have been perceived even without the unfriendly tenor. In case you might not know the team is maintaining a wide range of packages and so there is not always a need to know about specific Fortran issues. well, apparently you did perceive it as unfriendly. However in the past I did point out many issues with (unaddressed) build failures for packages maintained by the -science (and the -med) team, which didn't get any feedback. Which I do perceive as unfriendly. So apparently this team does have a size where it cannot manage the maintained packages any more, or where not everybody in the team is interested in every package. Other teams do have this problem too (which form my point of view is a problem). Sorry, but it is not the first time that I see this with the -science team. It even might happen that not everybody does read each mail on the list. well, why not, and who else should be addressed? Matthias -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51f27fc6.40...@debian.org
future of blas/lapack/atlas packages?
Hi all, blas and lapack are basic numeric libraries much used within Debian; the same holds for atlas which seems to be be needed by more and more packages. We had some difficulties getting these packages converted to build with gfortran; initial packages were made for experimental, then some packages were provided by the package maintainer, which were further updated to the current state in unstable/testing. However it looks like these packages are basically unmaintained, and the packaging is an interesting one, in that many people did fail to upgrade the atlas package from 3.6 to 3.8. How could we go on with the packaging of these packages? Camm currently doesn't seem to have enough time, so is there anybody interested in maintainance who would be able to update the atlas package and maybe set up a packaging group which could be open for the current package maintainer? thanks, Matthias -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]