Bug#1043240: transition: pandas 1.5 -> 2.1
Hi Rebecca, Am Tue, Feb 20, 2024 at 10:10:46PM + schrieb Rebecca N. Palmer: > Remaining blockers for testing migration: > - python-ulmo #1044057: has a patch, please upload I've uploaded this yesterday. > - pydevd #1063274: unclear whether my patch breaks something else, please > leave alone for now I've read the discussion in this bug report and leave the decision to Julian who is obviously way deeper involved into this. > Status unclear: > - python-xarray: autopkgtest has failed 3 times, but all 3 are (different) > failures that have occurred without pandas 2, apparently at random I admit I see no flaws here. Tracker page looks fine and debci[1] has only issues for ppc64el and riscv64 - if it would be Pandas it would probably fail on all architectures, thought. > - q2-*: autopkgtest fails in "fixed" versions, but possibly because they're > not all being tested together I uploaded the latest versions of q2* packages by enabling tests of all supported Python3 versions. The tests for Python3.12 are failing in most of the q2-* packages due to some code in qiime which is not related to Pandas (IMHO). I've reported this upstream[2] and hope for a quick solution. Thanks a lot for all your work and support Andreas. [1] https://ci.debian.net/packages/p/python-xarray/ [2] https://github.com/qiime2/qiime2/issues/751 -- http://fam-tille.de
Bug#1043240: transition: pandas 1.5 -> 2.1
Remaining blockers for testing migration: - python-ulmo #1044057: has a patch, please upload - pydevd #1063274: unclear whether my patch breaks something else, please leave alone for now Status unclear: - python-xarray: autopkgtest has failed 3 times, but all 3 are (different) failures that have occurred without pandas 2, apparently at random - q2-*: autopkgtest fails in "fixed" versions, but possibly because they're not all being tested together
Bug#1043240: transition: pandas 1.5 -> 2.1
Control: block -1 by 1063274 Thank you for uploading those fixes. Note to self: pandas will need another upload, to remove the numba B-D and skip those tests (because numba is not in testing), and do something about 'ignoredtests' being slow enough to time out in i386 and arm64.
Bug#1044071: Bug#1043240: transition: pandas 1.5 -> 2.1 - please upload fixes
Hi again, Am Fri, Feb 02, 2024 at 09:56:14PM +0100 schrieb Andreas Tille: > Hi Rebecca, > > Am Tue, Jan 30, 2024 at 08:05:35AM + schrieb Rebecca N. Palmer: > > I intend to upload pandas 2.x to unstable soon. These packages have a patch > > in their bug - please upload them (I'm a DM, I can't do that), or if you > > think this patch won't work or isn't a good idea, tell me why: > > dials > > Was uploaded, all bugs closed. > > > python-altair > > I tried hard to get the latest version which implements what you suggested > independently in the bug report. Unfortunately it needs a new dependency > as I wrote in my comment in the bug report[2] and I was not able to easily > exclude the test that fails due to the missing module. Maybe I'd rather revert to the version currently in Debian. I might check later if nobody will beat me. > > python-feather-format I've followed the hint given by Rebecca. Unfortunately there are new Cython issues as you can see in Salsa CI[1]. Any hint would be welcome. > > seaborn Discussed in other mails > > tqdm > > I try to check later. Kind regards Andreas. [1] https://salsa.debian.org/python-team/packages/python-feather-format/-/jobs/5246082 -- http://fam-tille.de
Bug#1043240: transition: pandas 1.5 -> 2.1 - please upload fixes
seaborn has now been fixed. I intend to look at python-altair later.
Bug#1043240: transition: pandas 1.5 -> 2.1 - please upload fixes
Hi Rebecca, Am Tue, Jan 30, 2024 at 08:05:35AM + schrieb Rebecca N. Palmer: > I intend to upload pandas 2.x to unstable soon. These packages have a patch > in their bug - please upload them (I'm a DM, I can't do that), or if you > think this patch won't work or isn't a good idea, tell me why: > dials Was uploaded, all bugs closed. > influxdb-python I've answered to the bug[1] that the hints you gave seem to be incomplete. Unfortunately I have no idea how to fix this. > python-altair I tried hard to get the latest version which implements what you suggested independently in the bug report. Unfortunately it needs a new dependency as I wrote in my comment in the bug report[2] and I was not able to easily exclude the test that fails due to the missing module. > python-feather-format seaborn tqdm I try to check later. > In particular, I'd like the seaborn fix uploaded before pandas, so I can set > Breaks for it. (The pandas documentation build-depends on seaborn.) I will check soon as long as noone will beat me in doing so. Kind regards and thanks for all your work on pandas Andreas. [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1044076#43 [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1044073#23 -- http://fam-tille.de
Bug#1043240: transition: pandas 1.5 -> 2.1 - please upload fixes
On Tue, Jan 30, 2024 at 08:05:35AM +, Rebecca N. Palmer wrote: > In particular, I'd like the seaborn fix uploaded before pandas, so I can set > Breaks for it. (The pandas documentation build-depends on seaborn.) https://tracker.debian.org/news/1498923/accepted-seaborn-0132-1-source-into-unstable/ Best, Nilesh signature.asc Description: PGP signature
Bug#1043240: transition: pandas 1.5 -> 2.1 - please upload fixes
for dials it seems that the CI works with pandas 2.1 from experimental. https://ci.debian.net/packages/d/dials/unstable/amd64/41962612/#S4 - Le 30 Jan 24, à 9:05, Rebecca N. Palmer rebecca_pal...@zoho.com a écrit : > I intend to upload pandas 2.x to unstable soon. These packages have a > patch in their bug - please upload them (I'm a DM, I can't do that), or > if you think this patch won't work or isn't a good idea, tell me why: > dials influxdb-python python-altair python-feather-format seaborn tqdm > > In particular, I'd like the seaborn fix uploaded before pandas, so I can > set Breaks for it. (The pandas documentation build-depends on seaborn.)
Bug#1043240: transition: pandas 1.5 -> 2.1 - please upload fixes
I intend to upload pandas 2.x to unstable soon. These packages have a patch in their bug - please upload them (I'm a DM, I can't do that), or if you think this patch won't work or isn't a good idea, tell me why: dials influxdb-python python-altair python-feather-format seaborn tqdm In particular, I'd like the seaborn fix uploaded before pandas, so I can set Breaks for it. (The pandas documentation build-depends on seaborn.)
Bug#1043240: transition: pandas 1.5 -> 2.1
On Fri, Jan 26, 2024 at 08:43:03AM +0200, Graham Inggs wrote: > Hi > > On Tue, 23 Jan 2024 at 14:38, Julian Gilbey wrote: > > We're nearly there (the transition page says it's 99% done), and when > > this transition is complete, then python3-defaults 3.11.6+ will be > > able to migrate to testing. > > python3-defaults/3.11.6-1 with Python 3.12 as a supported version is > now in testing [1]. Wonderful news! Congratulations to everyone who helped to make this happen! Best wishes, Julian
Bug#1043240: transition: pandas 1.5 -> 2.1
Hi On Tue, 23 Jan 2024 at 14:38, Julian Gilbey wrote: > We're nearly there (the transition page says it's 99% done), and when > this transition is complete, then python3-defaults 3.11.6+ will be > able to migrate to testing. python3-defaults/3.11.6-1 with Python 3.12 as a supported version is now in testing [1]. > Yes - please don't upload it to unstable yet. Uploading to > experimental is fine. Uploading to unstable now should be fine, but maybe wait for pandas/1.5.3+dfsg-12 to migrate first (in about four hours). Regards Graham [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1055085#29
Bug#1043240: transition: pandas 1.5 -> 2.1
On Mon, Jan 22, 2024 at 08:50:55PM +, Rebecca N. Palmer wrote: > On 22/01/2024 11:51, Julian Gilbey wrote: > > Please could we wait until the "Python 3.12 is a supported version" > > transition is completed? > > How are you defining that? python3-defaults 3.11.6+ in testing? (I was > previously told 3.12-supporting pandas and numpy in testing, which has > happened. I don't think any of these 25 packages are on > https://qa.debian.org/excuses.php?package=python3-defaults , but I haven't > checked carefully, and at least influxdb-python and tqdm do have what I > suspect are Python 3.12 related issues.) https://release.debian.org/transitions/html/python3.12-add.html We're nearly there (the transition page says it's 99% done), and when this transition is complete, then python3-defaults 3.11.6+ will be able to migrate to testing. I don't fully understand the problem with transitions, but there was a request to hold back with significant upgrades until a python3.12-supporting python3-defaults has reached testing. > > Adding another 25 or so RC bugs at this > > point will just slow down that transition. > > What exactly do you want not done until then? Just not uploading pandas > 2.x to unstable, or is it also a problem to have these bugs marked as RC in > the BTS? (In all 22 cases that are in testing at all, the bug is also > present in the version in testing, so it being RC shouldn't block > migration.) Yes - please don't upload it to unstable yet. Uploading to experimental is fine. > > (Unless pandas 1.5 is > > preventing the transition, that is.) > > It isn't. Good! Julian
Bug#1043240: transition: pandas 1.5 -> 2.1
On 22/01/2024 11:51, Julian Gilbey wrote: Please could we wait until the "Python 3.12 is a supported version" transition is completed? How are you defining that? python3-defaults 3.11.6+ in testing? (I was previously told 3.12-supporting pandas and numpy in testing, which has happened. I don't think any of these 25 packages are on https://qa.debian.org/excuses.php?package=python3-defaults , but I haven't checked carefully, and at least influxdb-python and tqdm do have what I suspect are Python 3.12 related issues.) Adding another 25 or so RC bugs at this point will just slow down that transition. What exactly do you want not done until then? Just not uploading pandas 2.x to unstable, or is it also a problem to have these bugs marked as RC in the BTS? (In all 22 cases that are in testing at all, the bug is also present in the version in testing, so it being RC shouldn't block migration.) (Unless pandas 1.5 is preventing the transition, that is.) It isn't.
Bug#1043240: transition: pandas 1.5 -> 2.1
On Sun, Jan 21, 2024 at 03:29:21PM +, Rebecca N. Palmer wrote: > Control: severity 1053943 1053939 1053942 1044053 1044056 serious > Control: severity 1044074 1053946 1044078 1044079 1044077 serious > Control: severity 1044071 1044067 1044068 1044055 1044060 serious > Control: severity 1044072 1044073 1044064 1053945 1044054 serious > Control: severity 1044076 1053940 1044057 1053944 1050144 serious > > As previously discussed in this bug, I'd like to move pandas 2.x into > unstable reasonably soon. I'm aiming to get it in before the Ubuntu 24.04 > freeze (in about a month), but I am open to disagreement on whether this is > a good idea. Please could we wait until the "Python 3.12 is a supported version" transition is completed? Adding another 25 or so RC bugs at this point will just slow down that transition. (Unless pandas 1.5 is preventing the transition, that is.) Best wishes, Julian
Bug#1043240: transition: pandas 1.5 -> 2.1
Hi, how to remove myself from these lists? Thank you On Sun, 21 Jan 2024 at 18:30, Andreas Tille wrote: > Hi Rebecca, > > Am Sun, Jan 21, 2024 at 03:29:21PM + schrieb Rebecca N. Palmer: > > > > Hence, doing this transition now would involve breaking some reverse > > dependencies with no known fix, but given the number of packages > involved, > > trying to wait until they're all fixed is rather likely to instead end in > > pandas 1.5 being broken by a new Python/numpy/etc. > > Just go for it and lets try to fix issues as soon as possible. > > Thanks a lot for all your work on pandas > > Andreas. > > -- > http://fam-tille.de > >
Bug#1043240: transition: pandas 1.5 -> 2.1
Hi Rebecca, Am Sun, Jan 21, 2024 at 03:29:21PM + schrieb Rebecca N. Palmer: > > Hence, doing this transition now would involve breaking some reverse > dependencies with no known fix, but given the number of packages involved, > trying to wait until they're all fixed is rather likely to instead end in > pandas 1.5 being broken by a new Python/numpy/etc. Just go for it and lets try to fix issues as soon as possible. Thanks a lot for all your work on pandas Andreas. -- http://fam-tille.de
Bug#1043240: transition: pandas 1.5 -> 2.1
Control: severity 1053943 1053939 1053942 1044053 1044056 serious Control: severity 1044074 1053946 1044078 1044079 1044077 serious Control: severity 1044071 1044067 1044068 1044055 1044060 serious Control: severity 1044072 1044073 1044064 1053945 1044054 serious Control: severity 1044076 1053940 1044057 1053944 1050144 serious As previously discussed in this bug, I'd like to move pandas 2.x into unstable reasonably soon. I'm aiming to get it in before the Ubuntu 24.04 freeze (in about a month), but I am open to disagreement on whether this is a good idea. dask, python-skbio and python-upsetplot have been fixed since the previous discussion, but that still leaves the above 25. 6 of these have a known-to-me fix (dials influxdb-python python-altair python-feather-format seaborn tqdm - see their bugs for details). Hence, doing this transition now would involve breaking some reverse dependencies with no known fix, but given the number of packages involved, trying to wait until they're all fixed is rather likely to instead end in pandas 1.5 being broken by a new Python/numpy/etc.
Bug#1043240: transition: pandas 1.5 -> 2.1
On 10/12/2023 20:16, Julian Gilbey wrote: > [...]I'd be in favour of doing the pandas > transition now, which will allow Cython 3.0 to move into unstable. Cython 3 is already in unstable; pandas is currently using cython-legacy. And yes, my list of packages broken by pandas 2.x is those identified by builds and autopkgtests. On 11/12/2023 17:31, Matthias Klose wrote: On 11.12.23 08:12, Matthias Klose wrote: up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. I just nmued pyrle and sorted-nearest, having dependencies on cython3-legacy, letting the pyranges autopkg tests fail. Once this succeeds, pandas should be able to migrate. pandas' testing migration is also blocked by its build-dependency on python-xarray, and xarray's on python-sparse and cfgrib. If necessary, pandas can be built without xarray, but this skips some tests and probably adds some error messages to the documentation (the examples are run at build time).
Bug#1043240: transition: pandas 1.5 -> 2.1
On 11.12.23 08:12, Matthias Klose wrote: On 10.12.23 14:06, Rebecca N. Palmer wrote: Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. I just nmued pyrle and sorted-nearest, having dependencies on cython3-legacy, letting the pyranges autopkg tests fail. Once this succeeds, pandas should be able to migrate.
Bug#1043240: transition: pandas 1.5 -> 2.1
On 12/11/23 08:12, Matthias Klose wrote: On 10.12.23 14:06, Rebecca N. Palmer wrote: I'd like to move forward with the pandas 1.5 -> 2.1 transition reasonably soon. Given that pandas 2.x is *not* required for Python 3.12 (but is required for Cython 3.0), should we wait for the Python 3.12 transition to be done first? These are broken by pandas 2.x and have a possible (but untested) fix in their bug - please test and apply it: dask(?) dials influxdb-python* python-altair python-feather-format python-upsetplot seaborn tqdm* (* = this package is currently also broken for a non-pandas reason, probably Python 3.12, that I don't have a fix for) These are broken by pandas 2.x and have no known-to-me fix: augur cnvkit dyda emperor esda mirtop pymatgen pyranges python-anndata python-biom-format python-cooler python-nanoget python-skbio python-ulmo q2-quality-control q2-demux q2-taxa q2-types q2templates sklearn-pandas Some generic things to try are pandas.util.testing -> pandas.testing, .iteritems() -> .items(), and if one exists, a more recent upstream version. Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. Is there a way to see the binNMUs which are still stuck in unstable, and don't migrate? Matthias As a reminder: it's best practice to first upload the new release to Experimental, so we can see what happens with autopkgtest before destroying everything at once... Cheers, Thomas Goirand (zigo)
Bug#1043240: transition: pandas 1.5 -> 2.1
On 10.12.23 14:06, Rebecca N. Palmer wrote: I'd like to move forward with the pandas 1.5 -> 2.1 transition reasonably soon. Given that pandas 2.x is *not* required for Python 3.12 (but is required for Cython 3.0), should we wait for the Python 3.12 transition to be done first? These are broken by pandas 2.x and have a possible (but untested) fix in their bug - please test and apply it: dask(?) dials influxdb-python* python-altair python-feather-format python-upsetplot seaborn tqdm* (* = this package is currently also broken for a non-pandas reason, probably Python 3.12, that I don't have a fix for) These are broken by pandas 2.x and have no known-to-me fix: augur cnvkit dyda emperor esda mirtop pymatgen pyranges python-anndata python-biom-format python-cooler python-nanoget python-skbio python-ulmo q2-quality-control q2-demux q2-taxa q2-types q2templates sklearn-pandas Some generic things to try are pandas.util.testing -> pandas.testing, .iteritems() -> .items(), and if one exists, a more recent upstream version. Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. Is there a way to see the binNMUs which are still stuck in unstable, and don't migrate? Matthias
Bug#1043240: transition: pandas 1.5 -> 2.1
On Sun, Dec 10, 2023 at 01:06:01PM +, Rebecca N. Palmer wrote: > I'd like to move forward with the pandas 1.5 -> 2.1 transition reasonably > soon. > > Given that pandas 2.x is *not* required for Python 3.12 (but is required for > Cython 3.0), should we wait for the Python 3.12 transition to be done first? Well, I have seen at least one package that has an RC bug for the Python 3.12 transition that might be because it's still using an old version of cython3 :( So it's a bit of chicken-and-egg - having Cython 3.0 might be very helpful. But then there is this list of 28 packages broken by pandas 2.x. On the other hand, these will need fixing at some point soon anyway, so I'd be in favour of doing the pandas transition now, which will allow Cython 3.0 to move into unstable. Just my 2 cents' worth... Best wishes, Julian
Bug#1043240: transition: pandas 1.5 -> 2.1
I'd like to move forward with the pandas 1.5 -> 2.1 transition reasonably soon. Given that pandas 2.x is *not* required for Python 3.12 (but is required for Cython 3.0), should we wait for the Python 3.12 transition to be done first? These are broken by pandas 2.x and have a possible (but untested) fix in their bug - please test and apply it: dask(?) dials influxdb-python* python-altair python-feather-format python-upsetplot seaborn tqdm* (* = this package is currently also broken for a non-pandas reason, probably Python 3.12, that I don't have a fix for) These are broken by pandas 2.x and have no known-to-me fix: augur cnvkit dyda emperor esda mirtop pymatgen pyranges python-anndata python-biom-format python-cooler python-nanoget python-skbio python-ulmo q2-quality-control q2-demux q2-taxa q2-types q2templates sklearn-pandas Some generic things to try are pandas.util.testing -> pandas.testing, .iteritems() -> .items(), and if one exists, a more recent upstream version. Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. Build logs: https://launchpad.net/~rebecca-palmer/+archive/ubuntu/pandas2p1/+builds?build_text=_state=failed https://launchpad.net/~rebecca-palmer/+archive/ubuntu/pandas2p1n/+builds?build_text=_state=failed (The second is more recent, but includes fewer packages.) Autopkgtest logs: https://qa.debian.org/excuses.php?experimental=1=pandas (Because of the Python 3.12 transition, this may currently be wrong about what is a regression and what is not.)
Bug#1043240: transition: pandas 1.5 -> 2.1
astropy isn't actually a regression (i.e. it's probably _a_ bug, but unrelated to pandas 2.x), and python-hypothesis appears to be fixed (by upstream, in 6.83.1). I have filed individual bugs for the others.
Bug#1043240: transition: pandas 1.5 -> 2.1
Control: retitle -1 transition: pandas 1.5 -> 2.1 pandas 2.1 is now in experimental. In addition to the above, it breaks these packages: astropy dask patsy pymatgen python-cooler python-geopandas q2-demux q2-taxa q2-types seaborn tqdm and maybe python-hypothesis. (python-pauvre and sunpy are probably already fixed; drms and skyfield are probably unrelated to pandas.) Build logs: https://launchpad.net/~rebecca-palmer/+archive/ubuntu/pandas2p1/+builds?build_text=_state=failed Autopkgtest logs: https://qa.debian.org/excuses.php?experimental=1=pandas
