Re: [gentoo-science] Usage of blas and lapack with 64 bit integers
On 10/23/20 8:14 AM, Honza Macháček wrote: On 10/22/20 1:06 PM, Aisha Tammy wrote: > ...Currently I use in my work mainly sci-chemistry/nwchem, now at the > 7.0.0 version. It uses blas and lapack, and prefers compilation with > 64 bit integers (ILP64) data model, although an utility to convert it > for LP64 model with 32 bit integers is provided. I'm attaching my > ebuild that I've finally used to install and start using the new > NWChem version. I'm also adding all the patches the ebuild references > in case you wanted to test the ebuild. I'm affraid the way I made it > use the ILP64 version of sci-libs/openblas-0.3.10 is rather ugly, and > some more elegant, more general, more systematic way should be > available... Yes, you can try to install the OpenBLAS package from ::gentoo with the 'index-64bit' use flag enabled, which will install a libopenblas64.so which should have what you want. I am working on cleanin up these packages, but this should work for your particular problem. I'm sorry I wrote too much of mostly irrelevant talking, and failed to describe my problem properly. Yes, I've found out that to get sci-libs/openblas compiled in the ILP64 model I have to set the “index-64bit” USE flag instead of the former “int64”. I don't understand why the flag was renamed, and I don't like it (I'll write more on the topic further below), but I can live with it as long as I know what I have to set to get what I need. The worst problem I see in the effect of the flag: sci-libs/openblas-0.3.10 compiles in the LP64 (32 bit integer) model, and installs the library with all the bells and whistles. In addition, if the “index-64bit” is set, the ILP64 (64 bit integer) model version is compiled, and the resulting library file installed — completely bare, with no pkg-config file, no eselect configuration. As a result eselect knows nothing of the ILP64 version being installed, and packages that want to use it cannot get the necessary configuration information from pkg-config, at least not directly; you can see in my sci-chemistry/nwchem ebuild that I let it query pkg-config for the library directories of the LP64 openblas version, and add the name of the ILP64 library, manually hardcoding in the ebuild the knowledge got from reading the sci-libs/openblas ebuild of the name used for the ILP64 library and of the fact it is installed into the same directory as the LP64 version. This is non-elegant and error-prone in case changes are made to the sci-libs/openblas ebuild in future version changing the name used for the ILP64 library or the directory it's installed into. Maybe I should ask questions and protest the current state of afaires in the Gentoo bugzilla when the sci-libs/openblas ebuild resides in the main tree but I think I've seen phasing out the eselect alternatives concept and moving the ebuild to the main repository mentioned a few times in this list, so I think people responsible for the changes, and knowing their rationale are somewhere around. The problem with eselect may span more packages, at least sci-libs/mkl, the only linear algebra library besides sci-libs/openblas that supports the ILP64 model, as far as I know. In the past, in the eselect alternatives ecosystem, ILP64 model libraries were installed as a separate category of linear algebra libraries. Now sci-libs/openblas with the “index-64bit” flag set installs both LP64 and ILP64 libraries, only does not inform pkg-config nor eselect of the ILP64 version existence. On the other hand sci-libs/mkl isntalls, as far as I understand its ebuild, always only one version, and the “int64” flag selects whether it should use the LP64 }32 bit integer) or ILP64 (64 bit integer) model. I don't use sci-libs/mkl and I have not tested its behaviour but I suspect that after installing it with “int64”, selecting it with eselect will break all the programs using linear algebra libraries and expecting 32 bit integers. By the way, there are newer versions of sci-libs/mkl in the science overlay than in the main tree but they still depend on the old removed eselect alternatives ecosystem. That may add to the relevance of my questions and comments on mostly main tree packages to the science overlay project, as it suggests that phasing out the old eselect alternatives ecosystem, partial to the science overlay, is still a work in progress, and while it includes moving packages from the overlay to the main tree, recreating the functionality based in the past on that ecosystem is a job of the overlay developers. I have not experimented with setting the default integer size to 64 bits for my whole system, so I don't know if such an approach is viable. I believe that having LP64 and ILP64 mode packages coexisting side by side is desirable, and that eselect should recognize LP64 and ILP64 libraries as separate categories, and offer switching among different available implementations inside these categories
Re: [gentoo-science] Usage of blas and lapack with 64 bit integers
On 10/22/20 1:06 PM, Aisha Tammy wrote: > ...Currently I use in my work mainly sci-chemistry/nwchem, now at the > 7.0.0 version. It uses blas and lapack, and prefers compilation with > 64 bit integers (ILP64) data model, although an utility to convert it > for LP64 model with 32 bit integers is provided. I'm attaching my > ebuild that I've finally used to install and start using the new > NWChem version. I'm also adding all the patches the ebuild references > in case you wanted to test the ebuild. I'm affraid the way I made it > use the ILP64 version of sci-libs/openblas-0.3.10 is rather ugly, and > some more elegant, more general, more systematic way should be > available... Yes, you can try to install the OpenBLAS package from ::gentoo with the 'index-64bit' use flag enabled, which will install a libopenblas64.so which should have what you want. I am working on cleanin up these packages, but this should work for your particular problem. I'm sorry I wrote too much of mostly irrelevant talking, and failed to describe my problem properly. Yes, I've found out that to get sci-libs/openblas compiled in the ILP64 model I have to set the “index-64bit” USE flag instead of the former “int64”. I don't understand why the flag was renamed, and I don't like it (I'll write more on the topic further below), but I can live with it as long as I know what I have to set to get what I need. The worst problem I see in the effect of the flag: sci-libs/openblas-0.3.10 compiles in the LP64 (32 bit integer) model, and installs the library with all the bells and whistles. In addition, if the “index-64bit” is set, the ILP64 (64 bit integer) model version is compiled, and the resulting library file installed — completely bare, with no pkg-config file, no eselect configuration. As a result eselect knows nothing of the ILP64 version being installed, and packages that want to use it cannot get the necessary configuration information from pkg-config, at least not directly; you can see in my sci-chemistry/nwchem ebuild that I let it query pkg-config for the library directories of the LP64 openblas version, and add the name of the ILP64 library, manually hardcoding in the ebuild the knowledge got from reading the sci-libs/openblas ebuild of the name used for the ILP64 library and of the fact it is installed into the same directory as the LP64 version. This is non-elegant and error-prone in case changes are made to the sci-libs/openblas ebuild in future version changing the name used for the ILP64 library or the directory it's installed into. Maybe I should ask questions and protest the current state of afaires in the Gentoo bugzilla when the sci-libs/openblas ebuild resides in the main tree but I think I've seen phasing out the eselect alternatives concept and moving the ebuild to the main repository mentioned a few times in this list, so I think people responsible for the changes, and knowing their rationale are somewhere around. The problem with eselect may span more packages, at least sci-libs/mkl, the only linear algebra library besides sci-libs/openblas that supports the ILP64 model, as far as I know. In the past, in the eselect alternatives ecosystem, ILP64 model libraries were installed as a separate category of linear algebra libraries. Now sci-libs/openblas with the “index-64bit” flag set installs both LP64 and ILP64 libraries, only does not inform pkg-config nor eselect of the ILP64 version existence. On the other hand sci-libs/mkl isntalls, as far as I understand its ebuild, always only one version, and the “int64” flag selects whether it should use the LP64 }32 bit integer) or ILP64 (64 bit integer) model. I don't use sci-libs/mkl and I have not tested its behaviour but I suspect that after installing it with “int64”, selecting it with eselect will break all the programs using linear algebra libraries and expecting 32 bit integers. By the way, there are newer versions of sci-libs/mkl in the science overlay than in the main tree but they still depend on the old removed eselect alternatives ecosystem. That may add to the relevance of my questions and comments on mostly main tree packages to the science overlay project, as it suggests that phasing out the old eselect alternatives ecosystem, partial to the science overlay, is still a work in progress, and while it includes moving packages from the overlay to the main tree, recreating the functionality based in the past on that ecosystem is a job of the overlay developers. I have not experimented with setting the default integer size to 64 bits for my whole system, so I don't know if such an approach is viable. I believe that having LP64 and ILP64 mode packages coexisting side by side is desirable, and that eselect should recognize LP64 and ILP64 libraries as separate categories, and offer switching among different available implementations inside these categories separately, just like it used to
Re: [gentoo-science] Usage of blas and lapack with 64 bit integers
On 10/22/20 5:43 AM, Honza Macháček wrote: Dear all, after the removal of eselect-alternatives, and all the corresponding reorganisation of linear algebra libraries in the science overlay I have serious problems compiling programs that I use. I'm afraid I may not be entitled to your help and advice since I dropped from contributing to the overlay. I used to maintain the sci-physics/abinit ebuild, and few others I had added over time, but after one particularly long pause in my updating those ebuilds I found that I had lost access to the overlay. I have a vague impression that there were some changes in the ways the overlay is managed and contributions submitted but I have not yet even actually tried to find the relevant documentation, and study it. Besides I was reluctant to ask for the access to contributing until I had solved my problems with the abinit ebuild that I haven't solved yet, so I've kept updating my handful of ebuilds that I more or less use in my personal overlay only. And now, after the recent changes in the overlay, they're broken completely — even my updated versions, let alone those I left in the overlay, unless somebody took over maintaining them —, and I'm at a loss how to repair them. Currently I use in my work mainly sci-chemistry/nwchem, now at the 7.0.0 version. It uses blas and lapack, and prefers compilation with 64 bit integers (ILP64) data model, although an utility to convert it for LP64 model with 32 bit integers is provided. I'm attaching my ebuild that I've finally used to install and start using the new NWChem version. I'm also adding all the patches the ebuild references in case you wanted to test the ebuild. I'm affraid the way I made it use the ILP64 version of sci-libs/openblas-0.3.10 is rather ugly, and some more elegant, more general, more systematic way should be available. Is there anything like that? Does anybody at least plan to introduce anything like that? Is there any documentation I should read that would guide me in updating my ebuilds in line with the recent overlay development, and possibly to try and contribute them to the overlay? All the best Honza Macháček Yes, you can try to install the OpenBLAS package from ::gentoo with the 'index-64bit' use flag enabled, which will install a libopenblas64.so which should have what you want. I am working on cleanin up these packages, but this should work for your particular problem. Aisha
