Bug#1068422: can't import dask.dataframe - TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object
Control: forwarded -1 https://github.com/dask/dask/pull/11035 Control: tags -1 fixed-upstream patch Thanks and probably yes (but I haven't tested that fix myself), as while it didn't happen in 3.11.8, it *does* happen in 3.11.9: https://ci.debian.net/packages/d/dask/unstable/amd64/46185892/
Bug#1069608: topplot: missing test-depends on python3-all
Source: topplot Version: 0.2.2+repack-1 Tags: patch Severity: serious Justification: blocks testing migration of other packages topplot tries to run its autopkgtest in all versions of Python (which is good), but does not test-depend on all those versions of Python. This previously worked because numpy depended on them. However, this has now been removed (see #945824), causing topplot's autopkgtests to fail. Adding python3-all to the Depends in debian/*tests*/control should fix this bug, but I have not actually tested this.
Bug#1068422: possibly caused by python 3.12.3 Re: Bug#1068422: can't import dask.dataframe - TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object
This bug is not *obviously* known to dask upstream, but their CI is failing and I haven't checked why. It happens only in Python 3.12, not 3.11: https://ci.debian.net/packages/d/dask/unstable/amd64/45013666/ and still doesn't happen in testing, but does happen in mostly-testing with src:python3-defaults,src:db5.3,src:keras,src:nodejs,src:openssl,src:python3-stdlib-extensions,src:python3.11,src:python3.12,src:readline,src:udisks2,src:viagee from unstable: https://ci.debian.net/packages/d/dask/testing/amd64/45564690/ This suggests that the trigger may be the upgrade of Python itself (3.12.2-1 in testing -> 3.12.3-1 in unstable). *Possibly* related items from the upstream Python changelog: https://github.com/python/cpython/issues/101293 https://github.com/python/cpython/issues/117110
Bug#1068349: nbsphinx/nbconvert broken by lxml 5.2: lxml.html.clean
Control: unblock 1068104 by -1 Control: unblock 1068104 by 1068422 To avoid being blocked by this bug, the pandas version I just uploaded temporarily disables the documentation. This is also an option for any other affected packages that urgently need to be uploaded. (I don't know whether the other two listed Blocks/Affects are that urgent.)
Bug#1068422: can't import dask.dataframe - TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object
Package: python3-dask Version: 2023.12.1+dfsg-2 Severity: serious Control: affects -1 src:pandas Control: block 1068104 by -1 Importing dask.dataframe currently fails with the error TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object amd64 https://salsa.debian.org/science-team/pandas/-/jobs/5543041 https://ci.debian.net/packages/d/dask/unstable/arm64/44706021/ i386 https://salsa.debian.org/science-team/pandas/-/jobs/5543042 This appears to have started recently, as this log from 2024-03-31 doesn't have it, and to occur in unstable but not testing: https://tests.reproducible-builds.org/debian/rb-pkg/unstable/armhf/pandas.html As the version of dask has not changed in this time, the cause is probably a change in some other package; I don't know which one.
Bug#1068104: pandas: FTBFS on 32-bit architectures with -D_TIME_BITS=64
Thanks - I plan to look at this tomorrow.
Bug#1066801: pandas: FTBFS: /usr/lib/python3/dist-packages/dateutil/zoneinfo/__init__.py:26: UserWarning: I/O error(2): No such file or directory
Control: retitle -1 pandas: test-failing warning with new xarray This is a warning being treated as an error, but the one in test_to_xarray (probably due to the new version of xarray), not the zoneinfo one. This is a FutureWarning, so it should be OK to *use* the current pandas with the new xarray. The same warning happens in autopkgtest but isn't treated as an error there, so didn't block the new xarray. I intend to try fixing this later.
Bug#1063274: pydevd: autopkgtest-failing warning with pandas 2.1
Is that a yes to>> Does just the patch (not the new upstream) also break debugpy?or have you not tried specifically that? (I'm looking for a quick fix for the autopkgtest fail to unblock the pandas 2.x transition. I agree that upgrading to a new upstream is a good idea in the long run.) the bytecode version in pydevd is behind that in Debian, This is currently true (Debian has python3-bytecode 0.15.1, upstream pydevd 2.10.0 vendors 0.13.0.dev), but if it were a problem I'd expect it to mean that pydevd/debugpy were *already* buggy, not that my patch makes them worse.
Bug#1063274: pydevd: autopkgtest-failing warning with pandas 2.1
Thank you for caring about not breaking other packages, and yes, that's a good reason to not upload that new upstream for now. Does just the patch (not the new upstream) also break debugpy? (It shouldn't be able to, since it only touches test code.)
Bug#1063274: pydevd: autopkgtest-failing warning with pandas 2.1
This has been merged but not uploaded - is there a reason it shouldn't be, or have you just not had time?
Bug#1044057: python-ulmo and pandas 2
Control: tags -1 patch
This should fix this (and as the filename suggests, also re-adds spme
older fixes that seem to have been dropped by mistake between -1.1 and
-2), but has not been tested: the relevant tests aren't run by default
and there vaguely might be legal issues around ulmo's tests.Description: Don't fail on malformed or changed test data
CDEC has malformed lines that pandas 1.4+ errors out on
(I'm not sure why earlier pandas didn't do the same);
GHCN has simply changed at the source.
Author: Rebecca N. Palmer (but upstream independently came up with the on_bad_lines part)
Bug-Debian: https://bugs.debian.org/1017573 https://bugs.debian.org/1044057
Forwarded: partly no, partly not-needed, partly https://github.com/ulmo-dev/ulmo/pull/214
--- a/test/cdec_historical_test.py
+++ b/test/cdec_historical_test.py
@@ -9,7 +9,7 @@ def test_get_stations():
stations_file = 'cdec/historical/all_stations.csv'
with test_util.mocked_urls(stations_file):
stations = ulmo.cdec.historical.get_stations()
-assert 2000 < len(stations)
+assert 1900 < len(stations)
assert u'PRA' in stations.index
--- a/test/ghcn_daily_test.py
+++ b/test/ghcn_daily_test.py
@@ -10,12 +10,12 @@ import test_util
test_stations = [
{
'country': 'US',
-'elevation': 286.5,
+'elevation': 325.8,
'gsn_flag': 'GSN',
'hcn_flag': 'HCN',
'id': 'USW3870',
-'latitude': 34.8831,
-'longitude': -82.2203,
+'latitude': 34.8833,
+'longitude': -82.2197,
'name': 'GREER',
'network': 'W',
'network_id': '3870',
--- a/ulmo/cdec/historical/core.py
+++ b/ulmo/cdec/historical/core.py
@@ -74,9 +74,9 @@ def get_stations():
# I haven't found a better list of stations, seems pretty janky
# to just have them in a file, and not sure if/when it is updated.
url = 'http://cdec.water.ca.gov/misc/all_stations.csv'
-# the csv is malformed, so some rows think there are 7 fields
-col_names = ['id','meta_url','name','num','lat','lon','junk']
-df = pd.read_csv(url, names=col_names, header=None, quotechar="'",index_col=0)
+# the csv is malformed, so some rows think there are 7-8 fields
+col_names = ['id','meta_url','name','num','lat','lon']
+df = pd.read_csv(url, names=col_names, header=None, quotechar="'",index_col=0,on_bad_lines='skip')
return df
@@ -170,7 +170,7 @@ def get_station_sensors(station_ids=None, sensor_ids=None, resolutions=None):
sensor_list.columns = ['sensor_id', 'variable', 'resolution','timerange']
except:
sensor_list.columns = ['variable', 'sensor_id', 'resolution', 'varcode', 'method', 'timerange']
-sensor_list[['variable', 'units']] = sensor_list.variable.str.split(',', 1, expand=True)
+sensor_list[['variable', 'units']] = sensor_list.variable.str.split(',', n=1, expand=True)
sensor_list.resolution = sensor_list.resolution.str.strip('()')
station_sensors[station_id] = _limit_sensor_list(sensor_list, sensor_ids, resolutions)
Bug#1044071: feather and pandas 2.0
Control: tags -1 patch It does sometimes happen that fixing "can't import the tests" reveals "the tests fail". Fixed in the Salsa merge request.
Bug#1063274: pandas 2.x fixes
Control: tags -1 patch See the Salsa merge request.
Bug#1061761: marked as pending in snakemake
Control: tag -1 pending Hello, Bug #1061761 in snakemake reported by you has been fixed in the Git repository and is awaiting an upload. You can see the commit message below and you can check the diff of the fix at: https://salsa.debian.org/med-team/snakemake/-/commit/607bd710a51b77502c641443cf68a2dae025c695 Adapt to f-strings being tokenized in Python 3.12. (Closes: #1061761) (this message was generated automatically) -- Greetings https://bugs.debian.org/1061761
Bug#1063273: seaborn: autopkgtest fail on i386 - probable rounding error
Control: tags -1 pending This appears to be fixed in Salsa (before I reported it) - is there any reason not to upload this now?
Bug#1063435: geopandas: test fail with pandas 2.x on 32 bit - int32 vs int64 mismatch
Package: python3-geopandas Version: 0.14.3-1 Severity: serious Control: block 1043240 by -1 In pandas 2.x (now in unstable), there are a few places where geopandas uses native-size int but the plain pandas objects used as test references are always int64, failing the test: https://ci.debian.net/packages/p/python-geopandas/testing/armhf/42777226/ Probably a reasonable response is to ignore this (check_index_type=False/check_dtype=False) but I haven't looked carefully.
Bug#1063274: pydevd: autopkgtest-failing warning with pandas 2.1
Source: pydevd Severity: serious Tags: patch A pydevd test uses DataFrame.applymap(), and fails because this now raises a FutureWarning. Replacing it with DataFrame.map() as this message suggests would probably fix it.
Bug#1063273: seaborn: autopkgtest fail on i386 - probable rounding error
Package: python3-seaborn Version: 0.13.2-1 Severity: serious seaborn's autopkgtest failed on i386, with differences small enough that they are plausibly rounding error (i.e. should be ignored, by using *almost_equal instead of exact comparisons), but I haven't looked carefully.
Bug#1044073: Sorry Re: python-altair and pandas 2.0
On 03/02/2024 22:01, Andreas Tille wrote: The point I was making in my mail was that I had trouble running the tests in **latest** upstream version (5.2.0). Sorry - I'd misread your previous message as you having tried 5.x, found that it didn't work due to the missing dependency, and decided to try 4.x with tests skipped instead. I think we should strive for latest upstream in general Agreed, assuming that doing so doesn't break things.
Bug#1061761: snakemake issues with Python 3.12
It looks like this is at least two issues: - Tests mix tabs and spaces, which is no longer allowed = upstream 2459 - Assumes f-strings are not tokenized, which they now are = upstream 2485/2588/2649 Fix in progress on the debian-v7 branch. (The main branch has 8.x, which doesn't work due to missing dependencies.)
Bug#1044073: python-altair and pandas 2.0
Please don't skip/xfail tests - my suggestion above is an actual fix: https://salsa.debian.org/rnpalmer-guest/python-altair/-/tree/fix1044073?ref_type=heads (In a fork because, despite its description, this is not actually a debian-science package. The Salsa CI "fail" is because the *old* version is uninstallable due to Breaks: in pandas and the piuparts upgrade test counts this as a fail.)
Bug#1044076: influxdb-python and pandas 2.1
My fixes are pushed to Salsa, but they're in a fork because this isn't a debian-science package: https://salsa.debian.org/rnpalmer-guest/influxdb-python
Bug#1044076: influxdb-python and pandas 2.1
Note that this uncertainty is only around whether this is a complete fix - even in the case where it's not, it *wouldn't* be actively worse than doing nothing, though it would be hiding the problem.
Bug#1044076: influxdb-python and pandas 2.1
Some looking through the code suggests that the precision is user-set and hence constant within a single query, and hence that this fix is OK, but I'm not entirely certain of that. There are ways to make pandas 2.x accept mixed time format, but I think they're 2.x _only_ and/or slow.
Bug#1053946: tqdm and pandas 2.1 / python 3.12
That turned out to be easier than it looked - fixing the easy one also made the others go away. Please upload this: https://salsa.debian.org/rnpalmer-guest/tqdm
Bug#1058160: tqdm and pandas 2.1 / python 3.12
Control: retitle 1058160 tqdm: tests failing/hanging in Python 3.12 That works for #1053946 (pandas 2.x), but #1058160 (python 3.12) is more than a single hanging test, and it's not immediately obvious what should be done. Attempted fix (caution, currently just skips the hanging test) and failure logs: https://salsa.debian.org/rnpalmer-guest/tqdm
Bug#1056828: Severity / marking of "uses cython3-legacy" bugs
pandas and statsmodels are currently using cython3-legacy. I left their cython 3.0 bugs open when I made this change, as I do want to actually switch them to cython 3.x at some point. These bugs have now been raised to RC. Is this an intentional attempt to remove cython3-legacy (which seems unlikely as there is no such bug on cython3-legacy itself), or have they been mistaken for packages that still depend on plain cython3 but don't build with cython3 3.x (which would be an active FTBFS and hence RC)? If the second, how should such bugs be marked to make them less confusing? Is removing the usertag enough?
Bug#1055848: cfgrib failing to find eccodes
Control: retitle -1 cfgrib: test failure - can't find eccodes Because we now enforce that build-depends of testing must be in testing, this bug is blocking python-xarray and hence pandas from testing, so is more important than this package's popcon would suggest. I consider it likely that the real bug is in ecmwflibs, specifically this function, but haven't actually tested that: https://sources.debian.org/src/ecmwflibs/2%3A0.5.7-1/ecmwflibs/__init__.py/#L118
Bug#1055847: python-sparse crash on arm64 (and possibly other non-x86)
At least on arm64, this appears to be a crash in a @numba.jit function. Given numba's known issues on non-x86 (e.g. #1033907), this suggests that actually fixing it _may_ not be reasonable. In pandas I don't recommend/build/test-depend on numba on the problem architectures, but python-sparse has a hard Depends on numba so that doesn't appear to be an option here. A potential option is to disable the tests on these architectures and display a warning on import (for an example, see https://salsa.debian.org/science-team/statsmodels/-/commit/a64430166b1e48135c58ec41330e3ce9b2206b46 ).
Bug#1056504: python-sparse in Python 3.12
Control: tags -1 fixed-upstream This was fixed upstream by https://github.com/pydata/sparse/commit/266af7307b5232a37daa1d7659f02faf6a42e46e . It still exists here because debian/patches/fix-test-fails-on-i386.patch effectively reverses that fix, possibly by accident.
Bug#1031414: Proposed RM: libgpuarray - abandoned, RC-buggy
libgpuarray has been de facto abandoned upstream since 2018, and is currently unused in Debian. (Its original user was theano, which was removed in #1042574.) It currently has what may or may not be a wrong-answer bug (#1031414), and two broken-by-transition bugs. (The latter two _might_ be easy to fix - I haven't tried because I've already mostly given up on it.) Hence, I intend to request removal.
Bug#1055801: pandas: test failures with Python 3.12
Control: severity -1 normal Control: retitle -1 pandas/pytables: ignored test fails with Python 3.12 Control: found -1 2.1.3+dfsg-1 Control: tags 1057805 pending In 1.5.3+dfsg-8, these are ignored, so are no longer an FTBFS but are still a bug. I don't know whether they're a pandas bug or a pytables bug; they don't appear to be known to either upstream. (The build failure on armhf, #1057805, was because this xfail took precedence over the existing run=False xfail and hence exposed a long-known crash, #790925. This should be fixed in -9.) If the 'affects' is because you were trying to test those reverse dependencies in Python 3.12, this should now be possible.
Bug#1055801: pandas: test failures with Python 3.12
Control: unblock -1 by 1043240 The remaining test failures (all pytables-related) exist in both pandas 1.5 and 2.1, and do not appear to be known upstream. If they are all crashes rather than wrong answers (which I think they are), I intend to ignore them for now to unblock this, but still consider them bugs.
Bug#1053700: xarray test failure on s390x
I retried the arm64 test and it passed, so it's probably just a random hang (I've seen those before in pytest). Hence, I now _do_ suggest uploading current Salsa.
Bug#1053700: xarray test failure on s390x
This fix worked but the new upstream version added another similar issue, so s390x failed again. I have pushed a fix for this, and enabled allow-stderr to fix armel (error output from an already-xfailed test). I don't know what's causing the hang on arm64, so please do _not_ upload this just yet.
Bug#1050854: xarray test failure on amd64
I have added what should be fixes for #1053700 and #1044869, but they are currently untested as salsa-ci only runs the autopkgtest on amd64. #1050854 appears to have been fixed (or at least xfailed - it passed on salsa-ci but they say it's flaky) by upstream.
Bug#1053700: xarray test failure on s390x
Control: tags -1 patch
These all come from xarray/tests/test_coding_times.py, and are probably
fixable by replacing both instances of ("M8[ns]" if
sys.byteorder=='big' else "haven't actually tried that.
Bug#1050854: xarray test failure on amd64
This bug is now a crash (not the original error message): https://ci.debian.net/data/autopkgtest/testing/amd64/p/python-xarray/39962554/log.gz I don't know whether the same patch still works. The Salsa repository now appears to be up to date. You probably also want to include the fixes for the other two RC bugs (see there).
Bug#1053672: Apple copyright notices in test .xlsx files
Source: openpyxl Version: 3.0.9-2 Severity: serious licensecheck -r --copyright . on openpyxl finds these: ./openpyxl/formatting/tests/data/conditional-formatting.xlsx: UNKNOWN [Copyright: 2007 Apple Inc.] ./openpyxl/reader/tests/data/complex-styles.xlsx: UNKNOWN [Copyright: 2007 Apple Inc.] If opened in 'less' these files do contain the text "Copyright 2007 Apple Inc., all rights reserved.", marked as 'etext'. However, if opened in LibreOffice, no _obvious_ command displays this text. File > Properties instead displays creation and modification dates/authors that match the openpyxl upstream commit dates/authors (2012-4), which suggests that these are *not* straightforward copies of Apple-owned example files. Their actual contents are mostly formatting, i.e. they look like something created specifically to be a test case for reading formatting, or possibly a documentation example for formatting but the above makes that less likely, not something copied from real-world use. Given this, why is that copyright notice there, and what does it imply that we should do? (E.g. does Apple Numbers automatically copy Apple-owned items (e.g. fonts) into files it creates? If so, is there a way to remove these items to get a Free file?)
Bug#1052454: numexpr: unnecessarily disables security check
Package: python3-numexpr Version: 2.8.6-2 Severity: serious Justification: block testing migration of a known security hole Tags: patch numexpr 2.8.5 introduced a security check, which was initially buggy enough to break pyfai and pandas (#1049326). Fixes for this were sent upstream, but only some of them made it into numexpr 2.8.6. Hence, Debian 2.8.6-2 disabled this security check. However, this is not actually necessary to fix these bugs, and reopens a code execution security hole if numexpr is used to parse untrusted input. This is fixed by the fix1049326v2 branch in Salsa. This fix has also been sent upstream as https://github.com/pydata/numexpr/pull/452. (Sorry that this didn't get to you earlier - I tried to post to #1049326, and didn't notice the error message that posting to archived bugs is not allowed.)
Bug#1033907: numba crashes on non-x86
numba also crashes on mips64el, mipsel and armhf (and s390x but that was already known; not amd64, i386 or ppc64el). However, I agree that arm64 is a good place to start trying to fix this, given how common it is. For examples, see the pandas 2.0.3+dfsg-3 / 1.5.3+dfsg-5 build logs. Given the different error messages, there may be more than one bug here.
Bug#1049326: pandas FTBFS with numexpr 2.8.5
On further thought, both of those probably should be fixed in numexpr - I have pushed them to fix1049326. With those changes to numexpr, the new pandas (1.5.3+dfsg-5) should build; pytables will still also need its upstream fix.
Bug#1049326: pandas FTBFS with numexpr 2.8.5
stringToExpression() in this numexpr patch needs a default of True for sanitize (like the others already have), because pytables calls that directly: https://salsa.debian.org/science-team/pandas/-/jobs/4571451/raw (The other failure in there is one I plan to fix.) The two pytables tests with [] are operations that are supposed to fail but that now check for the wrong exception type - upstream fix: https://github.com/PyTables/PyTables/pull/1046
Bug#1049326: pandas FTBFS with numexpr 2.8.5
Given that this is a security feature and we hence want it in quickly, I'd consider that patch enough to xfail/disable the remaining issues in pandas, but discussions with upstream are ongoing. (It also won't do anything about the pytables breakage, known upstream as #1044.)
Bug#1049326: pandas FTBFS with numexpr 2.8.5
I pushed this fix to Salsa as the 'fix1049326' branch - it isn't enough to fix pandas by itself, but is plausibly an improvement.
Bug#1049326: pandas FTBFS with numexpr 2.8.5
Numexpr upstream have what they think is a fix for the exception issue (though not the overflow change), but it hasn't been tested yet: https://github.com/pydata/numexpr/issues/442
Bug#1049326: pandas FTBFS with numexpr 2.8.5
Control: forwarded -1 https://github.com/pandas-dev/pandas/issues/54546 Control: affects -1 python3-pandas There's actually *two* bugs here: an exception in eval/query, https://github.com/pandas-dev/pandas/issues/54449, and a changed result with integer overflow, https://github.com/pandas-dev/pandas/issues/54546. I now consider the exception non-RC, but I don't currently know whether the changed result counts as a wrong result (RC) or as something that was always undefined (not RC, disable the test). I have also checked that disabling numexpr in pandas is straightforward (though for performance reasons, I'd prefer not to), and that pandas 1.5.x is also affected (by both bugs).
Bug#1049326: pandas FTBFS with numexpr 2.8.5
Package: python3-numexpr Version: 2.8.5-1 Severity: serious Control: forwarded -1 https://github.com/pandas-dev/pandas/issues/54449 (actually they found it first) I only have actual logs for 2.0.x, but I suspect 1.5.x is also affected. As a short term fix, it may be possible to disable pandas' numexpr support, but I haven't yet had time to try that. It's also possible that as this is a security feature, it should be forced through ignoring the breakage, but I don't know enough to have a meaningful opinion.
Bug#1043048: Please give back statsmodels on mips64el Re: (Bug#1043048: fixed in openblas 0.3.23+ds-3)
This does seem to have worked: the openblas build log no longer contains FATAL ERROR. Hence, please give back statsmodels on mips64el. (DMs aren't allowed to use the self-service link.)
Bug#1043048: openblas: gives wrong results on mips64el, ignores test failures
Control: tags -1 upstream Upstream CI's mips64 qemu test has what look like the same FATAL ERRORs, in MIPS64_GENERIC but not SICORTEX, but is still listed as passing. Which suggests that they also have both parts of this bug (wrong answers on mips64el, and errors not actually failing the build), and that switching back to SICORTEX will probably work.
Bug#1043048: openblas: gives wrong results on mips64el, ignores test failures
The most obviously relevant change between those is that the installed BLAS changed from atlas to openblas. It looks like this wasn't a change of default, but that experimental uses libatlas and unstable uses libopenblas. (I don't know why, as the obvious dependency chain is src:statsmodels -> python3-scipy -> python3-pythran -> libopenblas-dev | libatlas-base-dev on both, and libopenblas is installable in experimental.) It looks like version 0.3.21+ds-4 is not affected by this issue and passes its testsuite. Can you possibly check whether statsmodels builds against that version? It did (successfully) in statsmodels version 0.13.5+dfsg-7. The openblas build logs also show the issue starting between .21 and .22.
Bug#1043048: openblas: gives wrong results on mips64el, ignores test failures
Package: libopenblas0-pthread Version: 0.3.23+ds-2 Control: affects -1 src:statsmodels Severity: serious Justification: the default BLAS should NOT silently give wrong answers (i.e. if there's no easy way to actually fix this, please switch the default on mips64el back to atlas, and consider removing this package from mips64el) statsmodels recently (between 0.14.0+dfsg-1 and -2) started to FTBFS on mips64el with multiple wrong test results. The most obviously relevant change between those is that the installed BLAS changed from atlas to openblas. openblas' own tests on mips64el ( https://buildd.debian.org/status/fetch.php?pkg=openblas=mips64el=0.3.23%2Bds-2=1686760279=0 ) have 64 instances of "FATAL ERROR - COMPUTED RESULT IS LESS THAN HALF ACCURATE". (I don't know why this isn't failing the build, which is possibly a bug in itself.) openblas upstream are not _obviously_ aware of this. Given the existence of .github/workflows/mips64.yml, this suggests it _may_ be nontrivial to reproduce in qemu.
Bug#1042043: pandas: FTBFS: failed tests
Control: tags -1 fixed-upstream patch This is probably caused by the new fsspec version, and is also seen in autopkgtest: https://ci.debian.net/data/autopkgtest/testing/amd64/p/pandas/36146393/log.gz Upstream fix: https://github.com/pandas-dev/pandas/commit/ce9e3d658d6efa3600a4563ca7a00e7a15b80272
Bug#1041803: hyperspy: FTBFS test_image fails
Package: hyperspy Version: 1.7.3-1 Severity: serious hyperspy currently FTBFS with several failing tests: https://launchpadlibrarian.net/678551154/buildlog_ubuntu-mantic-amd64.hyperspy_1.7.3-1_BUILDING.txt.gz
Bug#1027215: Proposed RM: theano, keras, deepnano -- broken by numpy 1.24, theano mostly abandoned upstream
Control: reopen -1 1026539 theano has been mostly abandoned upstream since 2018. (The Aesara fork is not abandoned, but includes interface changes including the import name, so would break reverse dependencies not specifically altered for it.) It is currently broken (FTBFS due to multiple test failures), probably by numpy 1.24 (#1027215). Its reverse dependencies (keras and deepnano) are both also broken by this. (keras _also_ has apparently unrelated problems, #1026738, and is orphaned, #1027938.) This was previously discussed in #1027215, where it was noted that removing keras would also block the addition of qmean (ITP #976981), but attempts to fix it failed. (Note that what is in Salsa isn't actually a fix: theano skips most of its tests in Salsa CI because they take several hours.)
Bug#974792: Proposed RM: beignet -- RoM; FTBFS with LLVM > 9, replaced by intel-opencl-icd
Additional information: As a rough estimate of how much speed this would cost on older hardware, I previously measured beignet as 8-16x faster than pocl (but this will obviously depend on hardware and application). However, beignet (on hardware that can't use intel-opencl-icd) is limited to single precision and OpenCL 1.2, so some OpenCL code won't run at all in beignet (e.g. #877316). beignet skips its tests if it can't find suitable GPU hardware (because buildds/debci probably don't have it), so if you think you have a fix, check whether the tests actually ran. There are some hints on enabling the GPU in a chroot in README.source, but I'm not sure if they actually work.
Bug#974792: Proposed RM: beignet -- RoM; FTBFS with LLVM > 9, replaced by intel-opencl-icd
beignet FTBFS with LLVM > 9 (#974792). beignet uses libllvm/libclang heavily enough that it usually has this kind of bug on a new LLVM release, but this time my attempts to fix it failed (see the Salsa CI logs). It is abandoned upstream and mostly replaced by intel-opencl-icd. (This is not a perfect replacement because it has a higher minimum hardware requirement, but pocl-opencl-icd also exists and should (slowly) work there.) Given this, I don't consider it worth spending more time on, and will make this into a real RM request if nobody objects. (I may have already kept it around for longer than actually made sense because it was my first package as an official maintainer.)
Bug#1033589: python3-theano: module 'numpy' has no attribute 'bool'.
While that particular problem is easy to fix, there are other incompatibilities that aren't (see #1026539, #1027215) and I intend to let autoremoval happen.
Bug#1027215: Bug#1026539: How much do we lose if we remove theano (+keras, deepnano, invesalius)?
On 03/03/2023 06:00, Andreas Tille wrote: Ahh, so aesara is not really a "fork" but a "rename"? The original is abandoned (no new development since 2017, and now mostly unmaintained, which is probably why it has this kind of bug). Aesara is a continuation by a new upstream (possibly one that mostly wanted it for their own use), that chose to break compatibility.
Bug#1026539: Bug#1027215: Bug#1026539: How much do we lose if we remove theano (+keras, deepnano, invesalius)?
On 02/03/2023 10:38, Andreas Tille wrote: I admit I do not see any good reason to stick to the old version if we decided before that keras/deepnano are no real blockers to even drop theano. Thus I was considering it more promising to spent my time on the latest version. 1.1.2 isn't the latest version, just the latest that calls the module theano rather than aesara. (I chose it to package because I didn't want to fully break compatibility, then abandoned it because I didn't like how much it did break.) Do you want to ask release team for permission to do this? If it would have build smoothly on all architectures, yes. If you wouldn't want to do this work if it can't get into bookworm (possibly because you'd prefer to package 2.0 as aesara if you have to wait until trixie anyway), you might want to ask them first. But we have the first trouble on arm64[1] Sorry - I forgot that theano skips most of its tests on Salsa-CI (because there's enough of them to take several hours), which is probably why it "passed" that but failed the full build. Given how many failures there are (everywhere), I don't consider this worth fixing, but you are of course free to disagree. (If you're planning to xfail tests, note that I consider *wrong-answer* bugs (though not crash bugs) in this kind of package to be RC. I haven't checked whether we currently have any.)
Bug#1026539: How much do we lose if we remove theano (+keras, deepnano, invesalius)?
I agree that switching to Aesara is probably the only reasonable option other than removal. (I'd given up on trying to fix 1.0, and was intending to let removal happen.) However, it's a much bigger change than is normally allowed in bookworm at this point. (1.1 includes multiple breaking changes, which is why it's in experimental, but a quick codesearch suggests these parts *may* not be used in keras/deepnano. https://github.com/aesara-devs/aesara/releases?page=8 ) Do you want to ask release team for permission to do this? Or do you want to try the same patches on 1.0? (I suspect that that won't work, but I haven't actually tried it.) (Also, you might not want numpy1p24_compat.patch - the v1p0 branch is currently in whatever state it was in when I gave up on it, and my vague memory is that this was a failed experiment, though I don't know if that meant "actively bad" or just "not a (full) solution".)
Bug#1031706: optuna: test_create_new_trial randomly fails on s390x
Package: python3-optuna Severity: serious Control: affects -1 python3-pandas test_create_new_trial (both parts) sometimes fails on s390x, failing the autopkgtest. It might make sense to remove this package on s390x, if it isn't reasonable to actually fix this.
Bug#1031705: pymatgen: 7th item of test_babel randomly hangs autopkgtest
Package: python3-pymatgen Severity: serious Control: affects -1 python3-pandas The autopkgtest sometimes times out, and when it does, the last line is io/tests/test_babel.py and 6 'pass' dots, implying that the 7th item hung. All known instances are on amd64.
Bug#1031414: clinfo breaks libgpuarray autopkgtest on i386: numerical deltas
Possibly useful here: gdb disassemble works inside pocl kernels, see #920497 for an example.
Bug#1031414: clinfo breaks libgpuarray autopkgtest on i386: numerical deltas
Control: reassign -1 src:pocl,src:libgpuarray Control: found -1 pocl/3.1-3 Control: found -1 libgpuarray/0.7.6-13 Control: retitle -1 libgpuarray: i386 test fail with new pocl The trigger is probably pocl, not clinfo. I don't yet know whether the bug is in pocl or libgpuarray.
Bug#1030096: dask.distributed intermittent autopkgtest fail
On 10/02/2023 06:41, Andreas Tille wrote: Am Fri, Feb 10, 2023 at 08:44:01AM +0530 schrieb Nilesh Patra: But I am not sure if this counter would be set to 2 days (from 5 days) or not -- will likely need to ask release team. As far as I observed the migration time is now 5 days (no matter whether autopkgtest or not). I think the "5 days" on the tracker page is because the reverse dependencies' autopkgtests haven't all been run yet, and will change to 2 days once they have: https://release.debian.org/testing/freeze_policy.html On 10/02/2023 03:43, Diane Trout wrote: > Though I made two changes that should dramatically increase the odds of > the tests passing. > > One I told it to skip all the "isinstalled" marked tests, which are all > skipped during build time, and the build seems to run far more > reliably. > > I got the idea because it seemed like the vast majority of test > failures are related to the daemons running or failing to shut down. That might be true on amd64, but I don't think it's true of arm*/s390x: the tests that are failing there do *not* appear to be isinstalled tests. I suspect the tests wouldn't have worked on those architectures in 2022.02 either, and we didn't notice because the previously mentioned bug was causing the autopkgtest to not actually run the tests. (dask.distributed is arch:all, so it's only built once, presumably on amd64.) > While talking on IRC about dask.distributed problems I learned of the > flaky autopkgtest restriction which says the test is expected to fail > intermittently and is not suitable for gating continuous integratin. > > So I marked the current whole autopkgtest run as flaky. > > So CI should also ignore the results of failed test runs now. Having only flaky tests that fail counts as having no tests: https://sources.debian.org/src/autopkgtest/5.28/doc/README.package-tests.rst/#L230 That presumably means 5 days, which we don't have, i.e. *don't* unless release team tell you otherwise. > When under less time pressure I think a good plan might be two split > the tests internally marked as isinstalled or flaky into a separate > autopkgtest run that's marked flaky and let the CI gate on the more > reliable tests. As stated above, that's probably the wrong split for this.
Bug#1030096: dask.distributed intermittent autopkgtest fail
Upstream's reason for treating warnings as errors is just generic 'find potential problems' (https://github.com/dask/distributed/issues/6048). On 09/02/2023 21:33, Diane Trout wrote: That worked, but armel (test_steal_twice), armhf (something outright crashing) and s390x (lots) all failed. Aren't those all still on -1? I only see amd64 and arm64 having run 2022.12.1+ds1-2 At https://ci.debian.net/packages/d/dask.distributed/ That summary listing is the wrong place to look for that information - either use tracker.debian.org or click the 'testing' (*not* 'unstable') links. All 3 of them have failed repeatedly. (armel's failure is sometimes test_single_executable_deprecated instead.) The armhf crash is a bus error (possibly unaligned memory access?) in protocol/tests/test_highlevelgraph.py, and the traceback suggests it *may* be in something other than dask.distributed, though it's also possible that dask.distributed is copying objects around with the wrong alignment: File "/usr/lib/python3/dist-packages/pandas/core/array_algos/take.py", line 163 in _take_nd_ndarray File "/usr/lib/python3/dist-packages/pandas/core/array_algos/take.py", line 117 in take_nd File "/usr/lib/python3/dist-packages/pandas/core/internals/blocks.py", line 880 in take_nd File "/usr/lib/python3/dist-packages/pandas/core/internals/managers.py", line 752 in File "/usr/lib/python3/dist-packages/pandas/core/internals/managers.py", line 751 in reindex_indexer File "/usr/lib/python3/dist-packages/pandas/core/internals/managers.py", line 978 in take File "/usr/lib/python3/dist-packages/pandas/core/generic.py", line 3886 in _take File "/usr/lib/python3/dist-packages/pandas/core/generic.py", line 3871 in take File "/usr/lib/python3/dist-packages/dask/dataframe/backends.py", line 517 in group_split_pandas File "/usr/lib/python3/dist-packages/dask/utils.py", line 640 in __call__ File "/usr/lib/python3/dist-packages/dask/dataframe/shuffle.py", line 941 in shuffle_group File "/usr/lib/python3/dist-packages/dask/layers.py", line 47 in __call__ File "/usr/lib/python3/dist-packages/distributed/worker.py", line 3047 in apply_function_simple File "/usr/lib/python3/dist-packages/distributed/worker.py", line 3025 in apply_function File "/usr/lib/python3/dist-packages/distributed/_concurrent_futures_thread.py", line 65 in run File "/usr/lib/python3/dist-packages/distributed/threadpoolexecutor.py", line 57 in _worker File "/usr/lib/python3.11/threading.py", line 975 in run File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap
Bug#1030096: dask.distributed intermittent autopkgtest fail
On 09/02/2023 21:33, Diane Trout wrote: The place to ask is debian-release; no comment on the likely result. I'll try to ask. Looking at the old logs, I think the old "passing" test *didn't actually run* the tests (just collected them, which was enough to fail on a dask/dask.distributed mismatch), because it did -k "not ( $SKIP_TEST )" when the variable was actually called SKIP_TESTS, causing ERROR: Wrong expression passed to '-k': not ( ): at column 8: expected not OR left parenthesis OR identifier; got right parenthesis and apparently-no tests run. (This was fixed in https://salsa.debian.org/python-team/packages/dask.distributed/-/commit/24cb367f4608a72d9f770cc619af3520bfdb1990 , apparently without noticing that it had ever existed.) Which makes this not-a-regression...
Bug#1030096: dask.distributed intermittent autopkgtest fail
On 09/02/2023 17:07, Diane Trout wrote: Would it make sense to drop those errors back to warnings, and do you know enough about the setup.cfg language to do it quickly? Plausibly yes but I don't actually know, and remove the 'error' line at setup.cfg:60. I went ahead and requested another run for the failed amd64 run and left the passing arm64 run alone. That worked, but armel (test_steal_twice), armhf (something outright crashing) and s390x (lots) all failed. Also how did numba 0.56.4-1 get overridden to be back in testing? Can we get dask.distributed forced back in? It looks like it mostly works, it feels like we're mostly fighting over it not being robust to environment specific issues. The place to ask is debian-release; no comment on the likely result.
Bug#1030096: dask.distributed intermittent autopkgtest fail
On 09/02/2023 06:36, Diane Trout wrote: Also there's still some flaky tests as the rebuild triggered by my just committing the changelog release had a failure in "test_release_retry" I don't think I've seen that particular one before, though like several others it's a warning being treated as an error because dask.distributed now does that (in setup.cfg). debci doesn't appear to have run yet. (If it does and fails, note that there's a retry button next to failure reports. Given how tight we are on time (we need to be in testing by the 12th), I'd rather not re-upload (restarting the migration clock) if we don't have to.) Also, we need to close this bug (by email _not_ by uploading).
Bug#1030096: dask.distributed intermittent autopkgtest fail
On 08/02/2023 22:09, Diane Trout wrote: I just got back a passed from salsa. So does anyone want to make any more changes? Or should we release this version? The *maybe* remaining issue is that test_balance_expensive_tasks[enough work to steal] seems to fail repeatedly when it fails, so if we want to ignore that (I don't claim to know whether that's a good idea), it needs an xfail/skip not just a flaky. I haven't seen that failure in your runs, but I don't know whether that means you've fixed it or just that you were lucky. Mostly, please upload *something* today, because we won't know for sure whether it passes on a real buildd/debci until we try that, and if it doesn't then the sooner we find out the better.
Bug#1030096: dask.distributed intermittent autopkgtest fail
I think I got the code to detect if IPv6 is available to work correctly so I could set the DISABLE_IPV6 environment variable that dask.distrubted supports. This probably refers to https://salsa.debian.org/python-team/packages/dask.distributed/-/blob/debian/master/debian/tests/run-tests#L11 This doesn't work: because run-tests is set -e, failing this check immediately ends the autopkgtest with failure (without even running the main tests): https://salsa.debian.org/python-team/packages/dask.distributed/-/jobs/3918057 I think you instead need the command inside the if-test, something like if ip -6 addr | grep global ; then if ping6 -c 4 2001:4860:4860:: ; then echo "Working ipv6 connectivity" but I haven't tested that. I went with skipping the 32bit tests instead of xfailed because I don't think they can work as written since I really think they're making really large memory requests that can't ever succeed on 32bit. Agreed - I'm possibly in the habit of using xfail rather than skip because I'm often xfailing the kind of things that might get fixed later. You did a lot of work on trying to get the flaky tests to work more reliably, and all that's included. https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/commit/b82894aa5247cd11607b177d60975387c2fd796a (marking a few more tests as flaky) isn't included. However as previously mentioned, I don't claim to know whether we actually should be marking those particular tests as flaky, so it's fine to omit that one if you think we shouldn't. Also as previously mentioned, test_balance_expensive_tasks[enough work to steal] seems to fail repeatedly when it fails, so if we want to ignore that (again, I don't claim to know whether that's a good idea), it needs an xfail/skip not just a flaky.
Bug#1030096: Any ideas Re: #1030096 dask.distributed intermittent autopkgtest fail ?
On 07/02/2023 03:20, Diane Trout wrote: What's your test environment like? Salsa CI. I don't think head is hugely different from what was released in -1. The diff looks like Andreas adjusted the dask dependency version, configured a salsa CI run, and added some upstream metadata files That sounds like you're not looking at my branch at all. As previously stated, that's https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/tree/fix1030096 (It's in a fork because I can't push to debian-python.) See earlier in this bug for which test failures still remain.
Bug#1030096: Any ideas Re: #1030096 dask.distributed intermittent autopkgtest fail ?
I agree that xfailing the tests *may* be a reasonable solution. I'm only saying that it should be done by someone with more idea than me of whether these particular tests are important, because blindly xfailing everything that fails is effectively not having tests. If we do choose that approach, at least test_balance_expensive_tasks needs to be an outright xfail/skip not just a flaky, because when it fails it fails repeatedly. On 06/02/2023 19:38, Diane Trout wrote: The most important thing about dask / dask.distributed is they really should be at about the same upstream version. I knew that, and was planning on 2022.12.1 of both when I decided to go ahead with pandas. What went wrong was that I only tested a build, not an autopkgtest, and thought the failing tests were dask.distributed's (known) inability to run all its tests in a buildd environment. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1027254#21 I'm not 100% sure how to mark that in the d/control file. Possibly Depends: python3-dask (>= 2022.12.1~), python3-dask (<< 2022.12.2~) but I haven't tested that.
Bug#1030096: Any ideas Re: #1030096 dask.distributed intermittent autopkgtest fail ?
(Background: the pandas + dask transition broke dask.distributed and it was hence removed from testing; I didn't notice at the time that if we don't get it back in we lose Spyder.) On 05/02/2023 21:44, Rebecca N. Palmer wrote: I currently have this in a state where it sometimes succeeds and sometimes doesn't: https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/tree/fix1030096 Tests I've seen to fail multiple times (and don't have a fix for): test_balance_expensive_tasks[enough work to steal] https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3902376 (Seems to be the most common problem. Using @pytest.mark.flaky to try 3 times doesn't seem to have helped, suggesting that if it fails once it keeps failing in that run. Applying part of upstream pull 7253 seemed to make things worse, but I haven't tried applying the whole thing.) test_popen_timeout https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3902745 Tests I've seen to fail once: test_stress_scatter_death https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3902040 test_tcp_many_listeners[asyncio] https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3896327 And now test_failing_task_increments_suspicious (once): https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3903956 (We don't have to pass build-i386 (as this is an arch:all package) or reprotest, but if these are effectively-random failures, they might also be able to occur in build or autopkgtest.) I'm probably the wrong person to be working on this - I don't know enough about this package to say whether ignoring this kind of intermittent failure (as my 'flaky' marks do) is appropriate, or to have much idea how to actually fix it. We could also try upgrading dask + dask.distributed to 2023.1, but that's a risky move at this point.
Bug#1030096: #1030096 dask.distributed autopkgtest fail
I currently have this in a state where it sometimes succeeds and sometimes doesn't: https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/tree/fix1030096 Tests I've seen to fail multiple times (and don't have a fix for): test_balance_expensive_tasks[enough work to steal] https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3902376 (Seems to be the most common problem. Using @pytest.mark.flaky to try 3 times doesn't seem to have helped, suggesting that if it fails once it keeps failing in that run. Applying part of upstream pull 7253 seemed to make things worse, but I haven't tried applying the whole thing.) test_popen_timeout https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3902745 Tests I've seen to fail once: test_stress_scatter_death https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3902040 test_tcp_many_listeners[asyncio] https://salsa.debian.org/rnpalmer-guest/dask.distributed/-/jobs/3896327
Bug#1030096: #1030096 dask.distributed autopkgtest fail
That removed most of the test failures, but there seem to be a few apparently random ones left (2 runs both failed, but with different errors). On 04/02/2023 21:35, Andreas Tille wrote: Any reason to not push to master directly? I don't do that with packages that aren't mine. Also see above.
Bug#1030096: #1030096 dask.distributed autopkgtest fail
I have an attempt to fix this in Salsa now (in my fork), but it hasn't had time to run the tests yet, so I don't know whether it works. Note the above mention that if we lose dask.distributed then we lose Spyder, which makes this a bigger issue than I initially thought.
Bug#1030208: Acknowledgement (statsmodels: FTBFS with scipy 1.10 on 32-bit: int64 -> int32 cast)
Control: merge -1 1030210 1030221 Not obviously known upstream. It's due to the test passing an int64 array to np.bincount (which casts to native-size int). Hence, we can probably get it to pass by explicitly casting to int32 in the test code, but I haven't tried that yet. I don't know exactly what caused this array to be int64 instead of int32.
Bug#1030208: statsmodels: FTBFS with scipy 1.10 on 32-bit: int64 -> int32 cast
Package: python3-statsmodels Version: 0.13.5+dfsg-4 Severity: serious Since scipy 1.10, some statsmodels tests fail on 32-bit with a message that int64 -> int32 casting is unsafe. I haven't yet had time to investigate further.
Bug#1017573: patch Re: ulmo broken by pandas 1.4+
Control: tags -1 patch Control: unblock -1 by 1011225 (Block possibly added by mistake - these don't look related.) This patch is now available as a Salsa merge request, and passed build+autopkgtest there.
Bug#1004869: python-xarray autopkgtest fail on i386
Control: reopen -1 Control: found -1 2023.01.0-1 This patch was dropped in the next upload (probably by accident), and hence this bug has reappeared. It is currently listed as blocking pandas from testing, though I'm not sure why (it isn't obvious why pandas and xarray would have to go together).
Bug#1029251: pandas: FTBFS: tests failed
Control: tags -1 pending Looks like rounding error in a test; ignored in Salsa.
Bug#1029232: dask.distributed: broken by version mismatch with dask
Would you like some help? If so, please push what you currently have to Salsa so we can see it. (No promises - #1029251 doesn't look like a big job but I might be wrong about that.)
Bug#1029232: dask.distributed: broken by version mismatch with dask
Package: python3-distributed Version: 2022.02.0+ds1-3 Severity: serious dask.distributed has been failing its autopkgtests since dask was upgraded to 2022.12.1. This is the expected result of upgrading one but not the other: I'd seen the same in my test PPA, and the upstream setup.py/requirements.txt require exactly matching upstream versions of dask and dask.distributed. Hence, it can probably be fixed by upgrading dask.distributed to 2022.12.1. (See also discussion in #1028667.) (Upgrading *both* dask.distributed and dask to 2023.1.0 probably also works, but I haven't tested that.) My PPA package of dask.distributed 2022.12 might be useful: https://launchpad.net/~rebecca-palmer/+archive/ubuntu/dask2022p12v2/+sourcepub/14427094/+listing-archive-extra (Don't upload that as-is: it tries to run more of the tests during the build because PPAs don't have autopkgtest, which doesn't work.) This and ulmo #1017573 (which has a patch) are the remaining blockers for pandas 1.5.
Bug#1028667: dask BD-Uninstallable
On 16/01/2023 07:22, Diane Trout wrote: On Sun, 2023-01-15 at 14:42 -0800, Diane Trout wrote: On Sun, 2023-01-15 at 18:16 +, Rebecca N. Palmer wrote: Probably fixed - see my merge request on Salsa. I'd tried to build versions of dask from 2022.03 - 2022.06 and they all failed with what appeared to be a strong dependency on pyarrow, which debian doesn't have. Did you get around that for 2022.12? Apparently 2022.12.1 builds without issue. I merged your salsa PR, have a couple useful things from my branch and there's about 2 tests triggering 5 failures in the autopkgtests to see if I can tell whats wrong with in 2022.12.1 Autopkgtest passes on my version: https://salsa.debian.org/python-team/packages/dask/-/jobs/3799175 Are you using pytest-xdist? I vaguely remember that breaking things in pandas, though I might be wrong about that. Though before releasing it I would like to try to build the matching dask.distributed package. https://launchpad.net/~rebecca-palmer/+archive/ubuntu/dask2022p12v2/+sourcepub/14427094/+listing-archive-extra (That fails some tests, but it's trying to run them during the build, which is known to be an issue for this package.) Though I also need to clean up my git history to get the 2022.12.1 changes I made onto salsa. But I'm going to do that tomorrow morning. Diane
Bug#1028667: dask BD-Uninstallable
Probably fixed - see my merge request on Salsa.
Bug#1028667: dask BD-Uninstallable
Control: retitle -1 dask: B-D on python-numpy-doc which no longer exists Control: tags -1 pending The python-numpy-doc error is because that package no longer exists. This build-dependency has already been removed in Salsa. However, that version doesn't build the dask documentation due to other issues. There's a version in my test PPA that I think does build the documentation, but I have *not* checked whether that documentation works (that PPA existed for checking whether the new dask + dask.distributed broke any of their reverse dependencies): https://launchpad.net/~rebecca-palmer/+archive/ubuntu/dask2022p12v2/+packages The python3-pandas error is because pandas now declares a Breaks: on dask (<= current) because of #1025393, and dask build-depends on python3-distributed and hence indirectly on itself. (dask 2022.12 fixed #1025393, but there's a bootstrapping problem there...) Does temporarily removing the build-dependency on python3-distributed work, or do we need a (temporary?) pandas without that Breaks?
Bug#1027215: How much do we lose if we remove theano (+keras, deepnano, invesalius)?
theano has been mostly abandoned upstream since 2018. (The Aesara fork is not abandoned, but includes interface changes including the import name, so would break reverse dependencies not specifically altered for it.) Its reverse dependencies are keras, deepnano and invesalius. It is currently broken, probably by numpy 1.24 (#1027215), and the immediately obvious fixes weren't enough (https://salsa.debian.org/science-team/theano/-/pipelines). Is this worth spending more effort on fixing, or should we just remove it?
Bug#1028373: pandas: test-failing warnings with numpy 1.24
Package: python3-pandas Version: 1.5.2+dfsg-4 Severity: serious Looks like the numpy 1.24 issues I knew about weren't the only ones. Fix in progress in Salsa.
Bug#1028359: pygpu: fails to import in numpy 1.24
Package: python3-pygpu
Version: 0.7.6-12
Severity: grave
numpy 1.24 no longer has np.bool, which makes pygpu fail to import:
pygpu/dtypes.py:74: in _fill_dtype_registry
register_dtype(np.bool, ["ga_bool", "bool"])
numpy/__init__.py:284: in __getattr__
raise AttributeError("module {!r} has no attribute "
E AttributeError: module 'numpy' has no attribute 'bool'
Full log: https://salsa.debian.org/science-team/theano/-/jobs/3771271
Bug#1027254: dask and pandas 1.5
(I meant to send that to the dask/pandas bug, which is #1025393, but both of them are potential reasons to update dask.) The intake issue is now https://github.com/dask/dask/issues/9813 (They're planning to fix it in intake rather than dask, but we might not be allowed to do that after the 12th.)
Bug#1027254: dask and pandas 1.5
To get the new upstream version to work, dask_sphinx-theme, dask and dask.distributed all need to be updated, in that order. Tests here: https://launchpad.net/~rebecca-palmer/+archive/ubuntu/dask2022p12v2/+packages dask didn't run its tests (PPAs only build, not autopkgtest), and dask.distributed failed its tests but possibly because I was trying to run more of the tests than it normally would. There were 2 issues in rdeps: intake forwards error messages in a way that doesn't work if they contain newlines, which one of the new dask's does. My fix for that was to patch dask to not have this newline. python-streamz can't find loop_in_thread - I don't yet know why, given that that still exists in the new dask.distributed. - If you instead want to patch 2022.02 (warning, I haven't tested this approach), there seem to be 2 sets of failing tests: dropna: fixed by https://github.com/dask/dask/commit/edb8a10f8c9706ec8868d174633b95355ef6cc91 groupby: these are tests that compare dask to pandas, in a case where pandas' behaviour changed. Upstream dask decided to make the same change (https://github.com/dask/dask/pull/8961). Warning, I haven't tested this approach, and since numpy 1.24 has gone ahead that bug would also need to be fixed.
Bug#1026539: theano FTBFS
Please do *not* upload theano 1.12 to unstable - it's the Aesara version, which may seriously break backward compatibility. I haven't had time to properly look at this bug yet.
Bug#1024906: apparently fixed Re: dask autopkgtest fail with python 3.11
This combination stopped failing on 2022-11-22/23 (without a dask upload, suggesting it was fixed somewhere other than dask itself). Hence, I suggest closing this bug.
Bug#1000922: llvmlite: uninstallable due to llvm-11 removal
Control: severity -1 grave Control: merge -1 1024795 Control: retitle -1 llvmlite: uninstallable due to llvm-11 removal It looks like closing and reopening this removed its "blocks" relationship to 1000941 (the llvm-11 removal request). Hence, llvm-11 has now been removed, making llvmlite (and hence numba) uninstallable. Do we have an llvm-13/14 llvmlite + numba that works on amd64? The one that was uploaded as 0.38.1-2 failed even there (#1014690), but the above implies there's been more work since then. As noted above, numba already has bugs on non-amd64 (https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=numba). I suggest either removing llvmlite/numba on non-amd64, or having them warn on import on non-amd64 (like https://salsa.debian.org/science-team/pandas/-/blob/main/debian/patches/numba_fail_32bit.patch#L120 ). (I generally consider it a serious bug - in the Debian BTS sense of "I'd rather remove the package than release it like this" - for a mathematical library to silently give wrong answers. However, I don't know how common this opinion is, or whether any of numba's non-amd64 bugs actually are wrong answers rather than exceptions/crashes.)
Bug#1020060: statsmodels sometimes FTBFS
It also doesn't fail on the buildds. (The failed binNMU of statsmodels for Python 3.11 is instead the expected result of trying to build statsmodels on a Python version without pandas - pandas failed due to #1023965.) Is there anything special about the mass-rebuild environment that might explain why it fails there and not elsewhere? (Possibly the standard email should link to a description of that environment?) Has it failed there more than once?
Bug#1023965: pandas FTBFS with Python 3.11 as supported version
Control: block 1021984 by -1 Control: tags -1 fixed-upstream fixed-in-experimental There's more than just those 2[0]. Some are tests that expect an error and check the message (not just the type), and fail because the message has been reworded - these should be trivial to fix. The 2 in test_null_quote_char/test_null_byte_char don't look like this, but also don't look like major problems. The ones that may need someone else's attention (because DMs can't upload arbitrary packages) are that Debian's parso is too old to support 3.11 - I've filed a new bug about this. I'm in the process of moving to pandas 1.5 (#1017809), which probably fixes these[1], but also breaks a few reverse dependencies[2]. Do I take the 'serious' tag to mean this is urgent enough to fix in 1.3? [0] https://launchpadlibrarian.net/631797785/buildlog_ubuntu-lunar-amd64.pandas_1.3.5+dfsg-5build1_BUILDING.txt.gz [1] https://buildd.debian.org/status/fetch.php?pkg=pandas=ppc64el=1.5.1%2Bdfsg-3=1668226153=0 (the lxml-not-found errors are probably because it hadn't been built in 3.11 yet) [2] https://qa.debian.org/excuses.php?experimental=1=pandas and maybe more because I haven't done a build test yet
Bug#1020060: statsmodels: FTBFS: make[1]: *** [debian/rules:105: override_dh_auto_test] Error 1
This is the same test as #1014278 / upstream 8341, but now on amd64. From the upstream discussion, it may be a numerically unstable test. It doesn't fail everywhere: a test build on Salsa CI passed, as did the last debci runs. (reproducible-builds FTBFS on the second build, on amd64/sid only, but on a _different_ test.)
Bug#992673: libgpuarray *gemv, *ger test fails
Control: tags -1 pending (fixed in Salsa, as far as I consider it my bug) It looks like both of these originate further down the stack, not in libgpuarray: The *ger error code is an OpenCL compile failure from inside clblast, so likely either pocl or clblast. The *gemv problem appears to be that the definition of gemv is y=alpha*A.dot(x)+beta*y, the spec allows and possibly requires the second term to be skipped entirely if beta is 0, clblas does this but clblast doesn't, and libgpuarray pygpu has a short form for beta=0 that uses an uninitialized (i.e. potentially NaN) y. As a workaround I am changing this to a zero y, but it maybe should be fixed in clblast.
