Re: Removing ATLAS?
Le mercredi 12 juillet 2023 à 15:34 +0200, Rafael Laboissière a écrit : > * Sébastien Villemot [2023-07-08 10:01]: > > > As the maintainer of the atlas package over the last decade, I now > > wonder whether we should remove it from the archive. […] > Thanks for starting this discussion and proposing a plan, Sébastien. > > I looked at xmds2, which currently build-depends and depends on > libatlas-base-dev. For those who do not know, xmds2 is a tool written in > Python for generating C++ code that solvesq differential equations, using > a XML file with the description of the problem. > > When generating the C++ code, the xmds tool tries to link against > libcblas.so, which, currently, only exists in the libatlas-base-dev > package. > > I “fixed” the problem with a hacky patch to the file > xpdeint/support/wscript and by changing the dependencies to > libopenblas-dev | libblis-dev. If someone can review my changes [*], I > will be grateful. Your fix looks good. Note that an even better fix is to simply Build- Depend on libblas-dev. Linking against an optimized BLAS does not really help at build time, because since all variants are ABI compatible and use the same SONAME, it’s the runtime dependency that really matters. -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
Re: Removing ATLAS?
Le mercredi 12 juillet 2023 à 16:05 +0200, Andreas Tille a écrit : > Am Wed, Jul 12, 2023 at 03:34:09PM +0200 schrieb Rafael Laboissière: > > When generating the C++ code, the xmds tool tries to link against > > libcblas.so, which, currently, only exists in the libatlas-base-dev package. > > I admit this is actually the reason why any of the packages I'm maintaining > depends from libatlas-base-dev: These are linking against libcblas which is > only provided in this package. If we might find a way to provide some kind > of "wrapper" so other implementations could provide libcblas as well, this > could be some generic solution. Unfortunately I'm probably pretty naive > and I'm asking for the impossible. Short version: any package that links against libcblas can normally be fixed by linking against libblas. Long version: for historical reasons, libblas.so in Debian contains both the BLAS Fortran routines, and a C interface to these routines (called CBLAS). Hence all the symbols that are in libcblas.so (which have the common “cblas_” prefix) are present in libblas.so. (Note that this is different from the LAPACK case: liblapack.so only includes the Fortran routines, while the C interface is provided by the separate liblapacke.so). You can check this for yourself by running the following commands: $ objdump -T /usr/lib/x86_64-linux-gnu/libblas.so $ objdump -T /usr/lib/x86_64-linux-gnu/libcblas.so -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
Re: Removing ATLAS?
Le lundi 10 juillet 2023 à 22:01 +0200, Andreas Tille a écrit : > Hi Sébastian, > > Am Sat, Jul 08, 2023 at 10:01:15AM +0200 schrieb Sébastien Villemot: > > > > So, given all that, I’m inclined to (try to) remove atlas during the > > trixie development cycle. > > Sounds reasonable. > > > Any thought on this? > > I've checked my responsibility for the dependencies and stumbled about > emmax > > > emmax.c:10:10: fatal error: clapack.h: No such file or directory >10 | #include > | ^~~ > compilation terminated. > > > and noticed > > $ apt-file search clapack.h > libatlas-base-dev: /usr/include/x86_64-linux-gnu/clapack.h > ... (other instances are not blas related) > > which was probably the reason why I've picked libatlas-base-dev > originally. I would not say that emmax is important and its > also a not maintained upstream any more. However, I vaguely > remember that this >#include > in some code pieces of Debian Med was the reason to actually > pick this blas implementation. Any hint how to deal with such > cases? clapack.h is apparently an early attempt at providing a C interface to some LAPACK functions (which are in Fortran). It indeed looks ATLAS- specific. The modern solution for that problem is to use LAPACKE (note the final “E”), which is provided by liblapacke-dev. It is a standardized and maintained C interface to all LAPACK routines. I guess it should be feasible to adapt emmax to make it work with LAPACKE. -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
Re: How to build compatible packages that use Eigen?
Hi Anton. Thanks for replying. Anton Gladky writes: > I have been maintaining the Eigen library in Debian for over 12 years, > and I cannot recall the specific bug ticket related to this topic. I believe it. I suspect the reason is that you get mystery crashes that require more-than-the-usual-amount-of knowledge to conclusively debug and then report the bug to the right place. In any case, once I figured out what was causing my problem, I see references to it all over the place. For instance: - https://eigen.tuxfamily.org/dox/TopicPreprocessorDirectives.html (search for "ABI") - https://stackoverflow.com/questions/50723121/eigen-crash-due-to-unaligned-access-with-msvc14-and-avx - https://eigen.tuxfamily.org/bz/show_bug.cgi?id=554 - https://stackoverflow.com/questions/71647793/eigen-static-lib-aligned-free-double-free-or-corruption Upstream should do better here, but I guess they decided they don't care. > However, it seems that the question you have raised is indeed valid. > If patching Eigen in those two places would help resolve the issue, > please prepare a patch, and I believe we can proceed with pushing it. OK. Let me construct the minimal program for the other crash of this kind I've seen (something about alignment), and then I'll propose some patches. > Does it make sense to discuss this with the Eigen developers? Or is > the question very specific to Debian (or packaging) and might not be > of interest to them? If I was them I wouldn't want my thing to crash. But there're references in their own docs and bug tracker that suggest that it's a known and accepted issue that mixed build flags make it die. Probably we can suggest the safe behavior behind and #ifdef at least. Let me make the patches so that we can have something to talk about. Thanks much.
Re: How to build compatible packages that use Eigen?
Hi Dima,
I have been maintaining the Eigen library in Debian for over 12 years,
and I cannot recall the specific bug ticket related to this topic. However,
it seems that the question you have raised is indeed valid. If patching
Eigen in those two places would help resolve the issue, please prepare
a patch, and I believe we can proceed with pushing it.
Does it make sense to discuss this with the Eigen developers? Or is the
question very specific to Debian (or packaging) and might not be of interest
to them?
Best regards,
Anton
Am Mi., 12. Juli 2023 um 01:24 Uhr schrieb Dima Kogan :
> Hi.
>
> Apologies for taking so long to look at this again; I've been busy.
>
> I just looked into it, and my conclusion is that there's no way to
> ensure that Eigen won't crash without us patching our package. So we
> should patch our package.
>
> I'm attaching a tiny demo program. Extract it and
>
> make && ./main
>
> You'll see that it crashes. We have lib.cc:
>
> #include
>
> __attribute__((visibility("default")))
> Eigen::MatrixXd* make_array()
> {
> return new Eigen::MatrixXd(10,20);
> }
>
> This is an analogue of some library we would be packaging for Debian.
> This would be built with no cpu-specific options, which is what the
> Makefile in this demo does.
>
> We also have a main.cc:
>
> #include
>
> Eigen::MatrixXd* make_array();
> int main(void)
> {
> Eigen::MatrixXd* matrix = make_array();
> delete matrix;
> return 0;
> }
>
> This is an analogue of some user program that uses this library. This
> isn't going into Debian, and the user wants to use their CPU fully, so
> they build with -mavx. This causes Eigen to crash. Because the
> allocation and deallocation paths don't work the same.
>
> In this demo no Eigen symbols are exported from lib.so, so it's not a
> case of the linker picking the wrong weak symbol, and this cannot be
> fixed by symbol versioning or visibility settings or anything.
>
> This isn't a contrived example. I hit this in the real-world with a
> package I'm going to upload soon (g2o).
>
> Can anybody see ways to make Eigen work reliably here without patching
> away the different paths in aligned_malloc() and aligned_free() ?
>
>
> https://sources.debian.org/src/eigen3/3.4.0-4/Eigen/src/Core/util/Memory.h/#L179
>
> https://sources.debian.org/src/eigen3/3.4.0-4/Eigen/src/Core/util/Memory.h/#L200
>
> I don't see any way to do it currently, and probably we should patch
> these out.
>
> There was also a second similar problem I saw earlier, that resulted in
> crashes due to inconsistent alignment. I'm going to revisit that
> shortly.
>
> Thanks.
>
>
