Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
Le mardi 23 octobre 2018 à 14:12 +, Mo Zhou a écrit : > On Mon, Oct 22, 2018 at 07:58:38PM +0200, Bastian Blank wrote: > > On Mon, Oct 22, 2018 at 07:55:10PM +0200, Sébastien Villemot wrote: > > > For BLAS/LAPACK implementations implemented in C, like OpenBLAS, they > > > will be compiled using LP64, and not ILP64. Only integers exposed > > > through the interface will be affected, through the use of appropriate > > > types. > > > > So you could also to a proper library transition and drop the support > > for 32-bit indicies completely? > > Completely dropping 32-bit-index version of BLAS/LAPACK for 64-bit > architectures is a long way to go. We can keep 32-bit-index version and > 64-bit-index version at the same time for a while and see if the > 32-bit-version is really droppable. > > This reminds me two points about wheter the 32-bit-index version is > droppable. As far as I know, Debian (will) have these BLAS[1] providers: > > (1) bin:libblas3 from src:lapack > (2) bin:libatlas3-base from src:atlas > (3) bin:libopenblas-base from src:openblas > (4) bin:libblis1 from src:blis [WIP] > (5) bin:libmkl-rt from src:intel-mkl [non-free] > (6) bin:libnvblas9.1 from src:nvidia-cuda-toolkit [non-free] [2] > > * I confirm these providers support 64-bit index in the API. > (2) (3) (4) (5) > > @Sebastien could you please confirm the status of 64-bit-index support > in lapack, i.e. (1) ? Since the BLAS and LAPACK implementations provided by src:lapack are pure Fortran code, my understanding is that it's just a matter of compiling them with -fdefault-integer-8 to get 64-bit indexing. This package also provides C interfaces (resp. CBLAS and LAPACKE). LAPACKE provides the necessary type aliases to get 64-bit indexing. Curiously, CBLAS does not seem to have that flexibility, and will probably need some (reasonably simple) patching. -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ http://sebastien.villemot.name ⠈⠳⣄ http://www.debian.org signature.asc Description: This is a digitally signed message part
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
Le mardi 23 octobre 2018 à 14:17 +, Mo Zhou a écrit : > Two in the audience are object to the "-ilp64" naming convention. > Then how about this? > > src:openblas > bin:libblas-base (...) > bin:libblas-dev(...) > bin:libblas64-base (filename=libblas64.so.3, SONAME=libblas64.so.3, > provides=libblas64.so.3-x86_64-linux-gnu) > bin:libblas64-dev (...) Adding a "64" suffix to the package name and to the SONAME (compared to the 32-bit indexing version) sounds good to me. -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ http://sebastien.villemot.name ⠈⠳⣄ http://www.debian.org signature.asc Description: This is a digitally signed message part
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On 2018-10-23 22:17, Mo Zhou wrote: Hi Sebastien, Two in the audience are object to the "-ilp64" naming convention. Then how about this? src:openblas bin:libblas-base (...) bin:libblas-dev(...) bin:libblas64-base (filename=libblas64.so.3, SONAME=libblas64.so.3, provides=libblas64.so.3-x86_64-linux-gnu) bin:libblas64-dev (...) This is consistent (modulo hyphen) with Cray, which has cray-petsc cray-petsc-64 cray-tpslcray-tpsl-64 Curiously Cray's BLAS implementation (cray-libsci) doesn't have a separate 64 version. Drew
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On Tue, Oct 23, 2018 at 02:12:16PM +, Mo Zhou wrote: > (1) bin:libblas3 from src:lapack > (2) bin:libatlas3-base from src:atlas > (3) bin:libopenblas-base from src:openblas > (4) bin:libblis1 from src:blis [WIP] > (5) bin:libmkl-rt from src:intel-mkl [non-free] > (6) bin:libnvblas9.1 from src:nvidia-cuda-toolkit [non-free] [2] > > * I confirm these providers support 64-bit index in the API. > (2) (3) (4) (5) > > @Sebastien could you please confirm the status of 64-bit-index support > in lapack, i.e. (1) ? Sorry, I mistyped. (2) shouldn't be there. (2) seems not supporting 64-bit index. It's header uses "const int" as the type of index. header available in bin:libatlas-base-dev And I took a look into the netlib blas, aka the standard BLAS implementation and it's index type is "const int" too ... Not every BLAS provider can provide 64-bit-index API. It's fine if only OpenBLAS (3), MKL (5), BLIS (4) can provide 64-bit-index API, because they are basically the fastest cpu-based BLAS implementations publically available.
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
Hi Sebastien, Two in the audience are object to the "-ilp64" naming convention. Then how about this? src:openblas bin:libblas-base (...) bin:libblas-dev(...) bin:libblas64-base (filename=libblas64.so.3, SONAME=libblas64.so.3, provides=libblas64.so.3-x86_64-linux-gnu) bin:libblas64-dev (...) * Unlike Fedora or Julia upstream, we don't mangle symbol names. On Mon, Oct 22, 2018 at 07:55:10PM +0200, Sébastien Villemot wrote: > Le lundi 22 octobre 2018 à 18:38 +0100, Simon McVittie a écrit : > > On Mon, 22 Oct 2018 at 18:17:32 +0100, Ben Hutchings wrote: > > > On Mon, 2018-10-22 at 15:07 +, Mo Zhou wrote: > > > > Here are some references: > > > > > > > > 1. > > > > https://software.intel.com/en-us/mkl-linux-developer-guide-using-the-ilp64-interface-vs-lp64-interface > > > > > > > > The Intel MKL ILP64 libraries use the 64-bit integer type (necessary > > > > for indexing large arrays, with more than 231-1 elements), whereas > > > > the LP64 libraries index arrays with the 32-bit integer type. > > > > > > [...] > > > > > > The correct C types for indexing arrays are ptrdiff_t and size_t. > > > These are already 64-bit in LP64 ABIs. So this seems like a workaround > > > for code using the wrong types. > > > > Do BLAS/LAPACK really mean ILP64, or do they really mean "ABI with large > > array indexes"? > > The latter. This is why I think that "ILP64" is a misnomer, and should > not be used for labeling the newly introduced libraries. > > The ambiguity arises from the fact that some BLAS/LAPACK > implementations are written in Fortran, and use the default integer > type for array indexes. Hence the solution is to compile them with > -fdefault-integer-8. But this does not mean that this code is really > ILP64, because it's not C and hence it does not uses the C ABI. Only > integers exposed through the BLAS/LAPACK ABI are affected (most of them > are array indices, the remaining others are return codes). > > For BLAS/LAPACK implementations implemented in C, like OpenBLAS, they > will be compiled using LP64, and not ILP64. Only integers exposed > through the interface will be affected, through the use of appropriate > types. > > > Best, > > -- > ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot > ⣾⠁⢠⠒⠀⣿⡁ Debian Developer > ⢿⡄⠘⠷⠚⠋⠀ http://sebastien.villemot.name > ⠈⠳⣄ http://www.debian.org
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On Mon, Oct 22, 2018 at 07:58:38PM +0200, Bastian Blank wrote: > On Mon, Oct 22, 2018 at 07:55:10PM +0200, Sébastien Villemot wrote: > > For BLAS/LAPACK implementations implemented in C, like OpenBLAS, they > > will be compiled using LP64, and not ILP64. Only integers exposed > > through the interface will be affected, through the use of appropriate > > types. > > So you could also to a proper library transition and drop the support > for 32-bit indicies completely? Completely dropping 32-bit-index version of BLAS/LAPACK for 64-bit architectures is a long way to go. We can keep 32-bit-index version and 64-bit-index version at the same time for a while and see if the 32-bit-version is really droppable. This reminds me two points about wheter the 32-bit-index version is droppable. As far as I know, Debian (will) have these BLAS[1] providers: (1) bin:libblas3 from src:lapack (2) bin:libatlas3-base from src:atlas (3) bin:libopenblas-base from src:openblas (4) bin:libblis1 from src:blis [WIP] (5) bin:libmkl-rt from src:intel-mkl [non-free] (6) bin:libnvblas9.1 from src:nvidia-cuda-toolkit [non-free] [2] * I confirm these providers support 64-bit index in the API. (2) (3) (4) (5) @Sebastien could you please confirm the status of 64-bit-index support in lapack, i.e. (1) ? * Provider (6) will become a tragedy when Debian no longer provide 32-bit-index version of BLAS... nvBLAS is a quite special BLAS implementation. NVIDIA calls it a "drop in" BLAS library but in fact it only implemented several level-3 BLAS routines. When a program calls a BLAS function that doesn't exist in nvBLAS, nvBLAS will pass the call into another BLAS implementation which is defined in a configuration file given by the user. That means, nvBLAS should call a library which uses index in the same width. And nvblas.h suggests that this library only supports 32-bit index. 60 /* GEMM */ 61 void sgemm_ (const char *transa, const char *transb, const int *m, const int *n, const int *k, 62 const float *alpha, const float *a, const int *lda, const float *b, const int *ldb, 63 const float *beta, float *c, const int *ldc); nvBLAS blocks the removal of 32-bit-index version of BLAS... [1] Not mentioning LAPACK here. [2] It has not been added to the update-alternatives system yet.
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On Mon, Oct 22, 2018 at 09:57:56PM +0200, Florian Weimer wrote: > > Proposal: > > > > * The "-ilp64" postfix should be appended to the SONAME of all the new > > shared objects that provide ILP64 interface. For example: > > > > libblas.so.3 (LP64) -> libblas-ilp64.so.3 (ILP64) > > > > As a result, the same postfix should be added to the binary package > > name. For example: > > > > libblas3 (LP64) -> libblas-ilp64-3 (ILP64) > > > > * No change will be made to all the present BLAS/LAPACK libraires that > > provide LP64 interface. > > > > * No change will be made to either API or ABI of BLAS/LAPACK. > > > > * This proposal only applies to 64-bit-capable architectures. > > Why do you want to retain the libraries with 32-bit indices? Is it > for ABI compatibility with Fortran code that uses them directly? BLAS/LAPACK providers with 32-bit index have to stay in the archive for a while for compatibility. And BLAS/LAPACK maintainers will have enough time to align all the affected pacakges and diagnoze with possible problems. > What's the time frame for these changes? Is it likely that a Fortran > ABI bump occurs before that anyway? I have no ETA. 32-bit and 64-bit index version of BLAS/LAPACK packages are expected to co-exist for a while. Only when all the reverse dependencies don't break with 64-bit index version should we consider dropping the 32-bit index version.
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
* Mo Zhou: > Proposal: > > * The "-ilp64" postfix should be appended to the SONAME of all the new > shared objects that provide ILP64 interface. For example: > > libblas.so.3 (LP64) -> libblas-ilp64.so.3 (ILP64) > > As a result, the same postfix should be added to the binary package > name. For example: > > libblas3 (LP64) -> libblas-ilp64-3 (ILP64) > > * No change will be made to all the present BLAS/LAPACK libraires that > provide LP64 interface. > > * No change will be made to either API or ABI of BLAS/LAPACK. > > * This proposal only applies to 64-bit-capable architectures. Why do you want to retain the libraries with 32-bit indices? Is it for ABI compatibility with Fortran code that uses them directly? What's the time frame for these changes? Is it likely that a Fortran ABI bump occurs before that anyway?
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On 2018-10-22 23:07, Mo Zhou wrote: Are any other packages likely to start wanting to use ILP64 ABIs? I guess it's very much an 'HPC' sort of thing at the moment. So yeah, some clarification in order I think, and an explanation of use-cases. HPC is indeed a related use case. I don't know any other package that would need such an ILP64 BLAS/LAPACK interface except for Julia. Actually by default Julia uses ILP64 version of openblas instead of LP64, see [julia-ilp64-default]. I don't have a strong opinion on what 64 bit policy Debian should push at this point in time. But I do have some experience with it on Cray. I found that FEniCS failed to run on multiple nodes using the standard (non-64) libraries (1 or 2 nodes was ok). But I got it running and scaling well up to 60 nodes (1440 processors) when I built against the 64 bit versions of the libraries (PETSc and others). It didn't really make sense since my mesh didn't have so many degrees of freedom that you would have thought 64 bit pointers were needed. But in practice there was a clear operational advantage to having the 64 bit libraries available. Drew
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On Mon, Oct 22, 2018 at 07:55:10PM +0200, Sébastien Villemot wrote: > For BLAS/LAPACK implementations implemented in C, like OpenBLAS, they > will be compiled using LP64, and not ILP64. Only integers exposed > through the interface will be affected, through the use of appropriate > types. So you could also to a proper library transition and drop the support for 32-bit indicies completely? Bastian -- A Vulcan can no sooner be disloyal than he can exist without breathing. -- Kirk, "The Menagerie", stardate 3012.4
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
Le lundi 22 octobre 2018 à 18:38 +0100, Simon McVittie a écrit : > On Mon, 22 Oct 2018 at 18:17:32 +0100, Ben Hutchings wrote: > > On Mon, 2018-10-22 at 15:07 +, Mo Zhou wrote: > > > Here are some references: > > > > > > 1. > > > https://software.intel.com/en-us/mkl-linux-developer-guide-using-the-ilp64-interface-vs-lp64-interface > > > > > > The Intel MKL ILP64 libraries use the 64-bit integer type (necessary > > > for indexing large arrays, with more than 231-1 elements), whereas > > > the LP64 libraries index arrays with the 32-bit integer type. > > > > [...] > > > > The correct C types for indexing arrays are ptrdiff_t and size_t. > > These are already 64-bit in LP64 ABIs. So this seems like a workaround > > for code using the wrong types. > > Do BLAS/LAPACK really mean ILP64, or do they really mean "ABI with large > array indexes"? The latter. This is why I think that "ILP64" is a misnomer, and should not be used for labeling the newly introduced libraries. The ambiguity arises from the fact that some BLAS/LAPACK implementations are written in Fortran, and use the default integer type for array indexes. Hence the solution is to compile them with -fdefault-integer-8. But this does not mean that this code is really ILP64, because it's not C and hence it does not uses the C ABI. Only integers exposed through the BLAS/LAPACK ABI are affected (most of them are array indices, the remaining others are return codes). For BLAS/LAPACK implementations implemented in C, like OpenBLAS, they will be compiled using LP64, and not ILP64. Only integers exposed through the interface will be affected, through the use of appropriate types. Best, -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ http://sebastien.villemot.name ⠈⠳⣄ http://www.debian.org signature.asc Description: This is a digitally signed message part
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On 22 Oct 2018, at 18:17, Ben Hutchings wrote: > On Mon, 2018-10-22 at 15:07 +, Mo Zhou wrote: >> Hi Wookey and Bastian, >> >> On Sun, Oct 21, 2018 at 06:51:16PM +0100, Wookey wrote: >>> On 2018-10-21 17:16 +0200, Bastian Blank wrote: Hi On Sun, Oct 21, 2018 at 09:51:15AM +, Mo Zhou wrote: > about naming convention of SONAME and package name. > > As discussed in [1][2][3], Debian will need a set of ILP64[4] interface > to BLAS/LAPACK in the future. Could you please describe what you mean? All 64-bit Debian architectures are LP64. So building a single binary using ILP64 will even break the ABI for glibc and it will most likely not run very far. (A file descriptor is defined as "int", so even the most basic file calls will be incompatible.) >> >> I missed some points in the original post. The proposal meant to add >> a new set of BLAS/LAPACK packages that are compiled in ILP64, and the >> existing BLAS/LAPACK API/ABI won't be changed and nothing will break. >> >> Here is a detailed example for demonstration >> >> src:openblas >>bin:libopenblas-base (LP64, won't be changed at all) >> provides libblas.so.3 >> bin:libopenblas-dev (LP64, won't be changed at all) >>provides libblas.so >> bin:libopenblas-ilp64 (ILP64, newly-added) >>provides libblas-ilp64.so.3 >> bin:libopenblas-ilp64-dev (ILP64, newly-added) >>provides libblas-ilp64.so >> >> So there is no "ABI bump" but just "adding a new set of ABI". At the >> same time, it won't result in any transition or breakage. > > The way we manage different basic ABIs in Debian is to define separate > Debian architectures. E.g. on x86-64 we have the amd64 (LP64) and x32 > (ILP32) architectures. > > How you can use an ILP64 library without also building applications for > ILP64? And then, don't you also need libc6 built for ILP64? So there seems to be a lot of confusion about what this is, and as an observer I went to read [1] as referenced by lumin. It's rather badly written but if you scroll down to the table under "Coding for ILP64" it becomes clear that they are not defining a new system ABI; C's int remains 32-bit. Of course, that makes total sense, as otherwise how can you have 8-bit, 16-bit and 32-bit integral types in POSIX C with only char and short, not to mention all the incompatibilities everywhere? So, like many others here I'm sure, I don't think this should be called "ILP64" at all; it's still LP64, just everything that was an int in the BLAS API is now a 64-bit type. I would suggest referring to it as something like BLAS64/LAPACK64, or, if people really insist on this awfully-confusing ILP64 terminology, at least try and disambiguate by calling it BLAS-ILP64. TL;DR: This is not ILP64. This is an all-64-bit variant of the BLAS/LAPACK API. James [1] https://software.intel.com/en-us/mkl-linux-developer-guide-using-the-ilp64-interface-vs-lp64-interface
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On Mon, 22 Oct 2018 at 18:17:32 +0100, Ben Hutchings wrote: > On Mon, 2018-10-22 at 15:07 +, Mo Zhou wrote: > > Here are some references: > > > > 1. > > https://software.intel.com/en-us/mkl-linux-developer-guide-using-the-ilp64-interface-vs-lp64-interface > > > >The Intel MKL ILP64 libraries use the 64-bit integer type (necessary > >for indexing large arrays, with more than 231-1 elements), whereas > >the LP64 libraries index arrays with the 32-bit integer type. > [...] > > The correct C types for indexing arrays are ptrdiff_t and size_t. > These are already 64-bit in LP64 ABIs. So this seems like a workaround > for code using the wrong types. Do BLAS/LAPACK really mean ILP64, or do they really mean "ABI with large array indexes"? A true ILP64 ABI would be one where open() takes a 64-bit flags parameter and returns a 64-bit result. As Ben says, that's a question of operating system ABI, not an individual library's ABI. If BLAS/LAPACK want a version that uses 64-bit quantities in places where they previously used ints, they should change the type used, not the meaning of "int". Prior art 1: in the CPython 2.5 ABI, PyList_GetItem took an int argument, but in CPython 2.6+ it takes a Py_ssize_t argument, where Py_ssize_t is a signed integer the same size as a size_t - that's a ssize_t on platforms that support that type, like Debian. This required an ABI break for Python extensions, but not for the whole operating system. Prior art 2: libpcre has multiple ABI versions that use different sizes for an abstract character in a Unicode string (libpcre3 for 8-bit units encoding Unicode in UTF-8, libpcre16-3 for 16-bit units encoding Unicode in UTF-16, and libpcre32-3 for 32-bit units encoding Unicode in UCS-4) but nobody claims that the different libpcre ABIs have different sizes for 'char'. smcv
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On Mon, 2018-10-22 at 15:07 +, Mo Zhou wrote: > Hi Wookey and Bastian, > > On Sun, Oct 21, 2018 at 06:51:16PM +0100, Wookey wrote: > > On 2018-10-21 17:16 +0200, Bastian Blank wrote: > > > Hi > > > > > > On Sun, Oct 21, 2018 at 09:51:15AM +, Mo Zhou wrote: > > > > about naming convention of SONAME and package name. > > > > > > > > As discussed in [1][2][3], Debian will need a set of ILP64[4] interface > > > > to BLAS/LAPACK in the future. > > > > > > Could you please describe what you mean? All 64-bit Debian > > > architectures are LP64. So building a single binary using ILP64 will > > > even break the ABI for glibc and it will most likely not run very far. > > > (A file descriptor is defined as "int", so even the most basic file > > > calls will be incompatible.) > > I missed some points in the original post. The proposal meant to add > a new set of BLAS/LAPACK packages that are compiled in ILP64, and the > existing BLAS/LAPACK API/ABI won't be changed and nothing will break. > > Here is a detailed example for demonstration > > src:openblas > bin:libopenblas-base (LP64, won't be changed at all) > provides libblas.so.3 > bin:libopenblas-dev (LP64, won't be changed at all) > provides libblas.so > bin:libopenblas-ilp64 (ILP64, newly-added) > provides libblas-ilp64.so.3 > bin:libopenblas-ilp64-dev (ILP64, newly-added) > provides libblas-ilp64.so > > So there is no "ABI bump" but just "adding a new set of ABI". At the > same time, it won't result in any transition or breakage. The way we manage different basic ABIs in Debian is to define separate Debian architectures. E.g. on x86-64 we have the amd64 (LP64) and x32 (ILP32) architectures. How you can use an ILP64 library without also building applications for ILP64? And then, don't you also need libc6 built for ILP64? > > I wondered about this. The mail said that the BLAS/LAPACK ABIs do not > > change, so I presumed that this was about internal data layouts for > > the data being passed which. But reading the bugreps it does sound > > like just a new ABI using ILP64. That would be properly done using > > multiarch or multilib paths, and needs some thought about how best to > > lay things out and what else would be needed to make it work. > > The simplest solution is just to create a separate ABI with different > name and different package name. And it introduces the least overhead. It might seem simple, but can the added ABI really be limited to just a few library packages? > > Are any > > other packages likely to start wanting to use ILP64 ABIs? I guess it's > > very much an 'HPC' sort of thing at the moment. > > > > So yeah, some clarification in order I think, and an explanation of > > use-cases. > > HPC is indeed a related use case. I don't know any other package that > would need such an ILP64 BLAS/LAPACK interface except for Julia. > Actually by default Julia uses ILP64 version of openblas instead of > LP64, see [julia-ilp64-default]. > > Here are some references: > > 1. > https://software.intel.com/en-us/mkl-linux-developer-guide-using-the-ilp64-interface-vs-lp64-interface > >The Intel MKL ILP64 libraries use the 64-bit integer type (necessary >for indexing large arrays, with more than 231-1 elements), whereas >the LP64 libraries index arrays with the 32-bit integer type. [...] The correct C types for indexing arrays are ptrdiff_t and size_t. These are already 64-bit in LP64 ABIs. So this seems like a workaround for code using the wrong types. Ben. -- Ben Hutchings Knowledge is power. France is bacon.
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
Hi Wookey and Bastian, On Sun, Oct 21, 2018 at 06:51:16PM +0100, Wookey wrote: > On 2018-10-21 17:16 +0200, Bastian Blank wrote: > > Hi > > > > On Sun, Oct 21, 2018 at 09:51:15AM +, Mo Zhou wrote: > > > about naming convention of SONAME and package name. > > > > > > As discussed in [1][2][3], Debian will need a set of ILP64[4] interface > > > to BLAS/LAPACK in the future. > > > > Could you please describe what you mean? All 64-bit Debian > > architectures are LP64. So building a single binary using ILP64 will > > even break the ABI for glibc and it will most likely not run very far. > > (A file descriptor is defined as "int", so even the most basic file > > calls will be incompatible.) I missed some points in the original post. The proposal meant to add a new set of BLAS/LAPACK packages that are compiled in ILP64, and the existing BLAS/LAPACK API/ABI won't be changed and nothing will break. Here is a detailed example for demonstration src:openblas bin:libopenblas-base (LP64, won't be changed at all) provides libblas.so.3 bin:libopenblas-dev (LP64, won't be changed at all) provides libblas.so bin:libopenblas-ilp64 (ILP64, newly-added) provides libblas-ilp64.so.3 bin:libopenblas-ilp64-dev (ILP64, newly-added) provides libblas-ilp64.so So there is no "ABI bump" but just "adding a new set of ABI". At the same time, it won't result in any transition or breakage. > I wondered about this. The mail said that the BLAS/LAPACK ABIs do not > change, so I presumed that this was about internal data layouts for > the data being passed which. But reading the bugreps it does sound > like just a new ABI using ILP64. That would be properly done using > multiarch or multilib paths, and needs some thought about how best to > lay things out and what else would be needed to make it work. The simplest solution is just to create a separate ABI with different name and different package name. And it introduces the least overhead. > Are any > other packages likely to start wanting to use ILP64 ABIs? I guess it's > very much an 'HPC' sort of thing at the moment. > > So yeah, some clarification in order I think, and an explanation of use-cases. HPC is indeed a related use case. I don't know any other package that would need such an ILP64 BLAS/LAPACK interface except for Julia. Actually by default Julia uses ILP64 version of openblas instead of LP64, see [julia-ilp64-default]. Here are some references: 1. https://software.intel.com/en-us/mkl-linux-developer-guide-using-the-ilp64-interface-vs-lp64-interface The Intel MKL ILP64 libraries use the 64-bit integer type (necessary for indexing large arrays, with more than 231-1 elements), whereas the LP64 libraries index arrays with the 32-bit integer type. 2. https://bugzilla.redhat.com/show_bug.cgi?id=1287541 Julia upstream and fedora chose a different solution... they mangled the BLAS/LAPACK symbol names by adding an "64_" suffix. However, Julia is the only upstream that use this mangling rule in practice, and maybe some other ILP64-capable BLAS/LAPACK providers doesn't allow easy name mangling. That means if we still want to take advantage from the update-alternatives mechanism, we should not mangle the symbol names. src:intel-mkl is currently the only ILP64-ready BLAS/LAPACK provider in the archive, and it doesn't mangle symbol names. [julia-ilp64-default] https://salsa.debian.org/julia-team/julia/blob/master/DISTRIBUTING.md#L126-130
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
On 2018-10-21 17:16 +0200, Bastian Blank wrote: > Hi > > On Sun, Oct 21, 2018 at 09:51:15AM +, Mo Zhou wrote: > > about naming convention of SONAME and package name. > > > > As discussed in [1][2][3], Debian will need a set of ILP64[4] interface > > to BLAS/LAPACK in the future. > > Could you please describe what you mean? All 64-bit Debian > architectures are LP64. So building a single binary using ILP64 will > even break the ABI for glibc and it will most likely not run very far. > (A file descriptor is defined as "int", so even the most basic file > calls will be incompatible.) I wondered about this. The mail said that the BLAS/LAPACK ABIs do not change, so I presumed that this was about internal data layouts for the data being passed which. But reading the bugreps it does sound like just a new ABI using ILP64. That would be properly done using multiarch or multilib paths, and needs some thought about how best to lay things out and what else would be needed to make it work. Are any other packages likely to start wanting to use ILP64 ABIs? I guess it's very much an 'HPC' sort of thing at the moment. So yeah, some clarification in order I think, and an explanation of use-cases. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Re: RFC: Naming convention for ILP64 variant of BLAS/LAPACK
Hi On Sun, Oct 21, 2018 at 09:51:15AM +, Mo Zhou wrote: > about naming convention of SONAME and package name. > > As discussed in [1][2][3], Debian will need a set of ILP64[4] interface > to BLAS/LAPACK in the future. Could you please describe what you mean? All 64-bit Debian architectures are LP64. So building a single binary using ILP64 will even break the ABI for glibc and it will most likely not run very far. (A file descriptor is defined as "int", so even the most basic file calls will be incompatible.) So I assume you mean something different than ILP64. I also found no option in gcc to change this part of the ABI, but I have not looked very far. Regards, Bastian -- Captain's Log, star date 21:34.5...
