Re: [gentoo-dev] Re: [PATCH v6 2/2] profiles/desc: add amdgpu_targets.desc for USE_EXPAND
On Wed, Aug 31, 2022 at 10:59:41PM -, Duncan wrote: > Yiyang Wu posted on Thu, 1 Sep 2022 00:40:34 +0800 as excerpted: > > > +# Referene： > > Non-ASCII unicode mangled "Reference:"?? > > Looks like "Referene here. kcharselect says U+ff1a is unicode > category "punctuation, other", block "half-width and full-width forms", > approximate equivalent U+003a, colon (presumably what you intended). > > Looks like the "c" is missing as well, but that could be my client's > display indigestion on the unicode. Sorry for the mess. It should be ":" but my input method gives a full-width unicode character. I'll fix that. The Github PR is now updated. -- Yiyang Wu
[gentoo-dev] Re: [PATCH v6 2/2] profiles/desc: add amdgpu_targets.desc for USE_EXPAND
Yiyang Wu posted on Thu, 1 Sep 2022 00:40:34 +0800 as excerpted: > +++ b/profiles/desc/amdgpu_targets.desc > @@ -0,0 +1,17 @@ > +# Copyright 1999-2022 Gentoo Authors. > +# Distributed under the terms of the GNU General Public License v2 > + > +# Referene： Non-ASCII unicode mangled "Reference:"?? Looks like "Referene here. kcharselect says U+ff1a is unicode category "punctuation, other", block "half-width and full-width forms", approximate equivalent U+003a, colon (presumably what you intended). Looks like the "c" is missing as well, but that could be my client's display indigestion on the unicode. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Add systemd/merged-usr profiles
> On 31 Aug 2022, at 21:36, Jaco Kroon wrote:
>
> Hi,
>
> On 2022/08/31 19:38, Mike Gilbert wrote:
>> On Wed, Aug 31, 2022 at 12:29 PM Jaco Kroon wrote:
>>> Hi,
>>>
>>> That really depends.
>>>
>>> If the expectation is that everything in /usr/{bin,sbin,lib*} needs to now
>>> fit on / rather than /usr we're queued to re-install a very, very large
>>> number of hosts.
>> You have that reversed: the expectation is that everything in
>> /{bin,sbin,lib} will fit in /usr. In other words, we move files from /
>> into /usr.
>
> That's a relieve, but as per Sam this is only relevant to systemd
> profiles, which for some reason I also completely overlooked as per the
> subject. However, these things do have a tendency to filter through to
> non-systemd systems eventually.
FWIW, our support (as you've sort of noticed before) for split-usr
(Which is related, but distinct, from non-merged -usr) is sort of tenuous
and while it's your right to do such installs, I'd consider at least
not installing new machines with such a configuration
as a way of very gradually phasing it out.
It's often a pain to handle properly. For example,
It's seemingly not possible to (easily, at least, and
portably) handle split-usr (separate /usr, as in
separate partition) in app-arch/zstd with pkg-config,
as pkg-config shouldn't point to the loader script,
but the real library [0].
As you can imagine, these complications together
with other, more pressing bugs, means it is at least
not very high on my list to look into such issues,
even though I do try when such bugs are within
my purview.
[0]
https://github.com/trofi/nix-guix-gentoo/commit/8f194519982fbfabb6b3ca84c0806b1a379b5d06
best,
sam
signature.asc
Description: Message signed with OpenPGP
Re: [gentoo-dev] Add systemd/merged-usr profiles
Hi,
On 2022/08/31 19:38, Mike Gilbert wrote:
> On Wed, Aug 31, 2022 at 12:29 PM Jaco Kroon wrote:
>> Hi,
>>
>> That really depends.
>>
>> If the expectation is that everything in /usr/{bin,sbin,lib*} needs to now
>> fit on / rather than /usr we're queued to re-install a very, very large
>> number of hosts.
> You have that reversed: the expectation is that everything in
> /{bin,sbin,lib} will fit in /usr. In other words, we move files from /
> into /usr.
That's a relieve, but as per Sam this is only relevant to systemd
profiles, which for some reason I also completely overlooked as per the
subject. However, these things do have a tendency to filter through to
non-systemd systems eventually.
Sorry for the noise.
Kind Regards,
Jaco
Re: [gentoo-dev] Add systemd/merged-usr profiles
> On 31 Aug 2022, at 17:29, Jaco Kroon wrote:
>
> Hi,
>
> That really depends.
>
> If the expectation is that everything in /usr/{bin,sbin,lib*} needs to now
> fit on / rather than /usr we're queued to re-install a very, very large
> number of hosts.
Of course, this is only for systemd users anyway. But see floppy's correction.
>
> Kind Regards,
> Jaco
>
> On 2022/08/31 18:01, Jeff Gazso wrote:
>
>> Just out of curiosity, how much pain is this likely to cause existing
>> installations that will need to migrate from a split-usr setup to a
>> merged-usr setup?
>>
>> On Tue, Aug 30, 2022 at 2:28 PM Mike Gilbert wrote:
>> This patch series adds a "merged-usr" feature profile, and subprofiles
>> for each systemd profile.
>>
>> As background: systemd upstream is preparing to drop support for
>> split-usr systems soon. All systemd users on Gentoo will eventually
>> need to migrate to a merged-usr system.
>>
>>
signature.asc
Description: Message signed with OpenPGP
Re: [gentoo-dev] Add systemd/merged-usr profiles
On Wed, Aug 31, 2022 at 12:29 PM Jaco Kroon wrote:
>
> Hi,
>
> That really depends.
>
> If the expectation is that everything in /usr/{bin,sbin,lib*} needs to now
> fit on / rather than /usr we're queued to re-install a very, very large
> number of hosts.
You have that reversed: the expectation is that everything in
/{bin,sbin,lib} will fit in /usr. In other words, we move files from /
into /usr.
[gentoo-dev] [PATCH v6 2/2] profiles/desc: add amdgpu_targets.desc for USE_EXPAND
Signed-off-by: Yiyang Wu --- profiles/desc/amdgpu_targets.desc | 17 + 1 file changed, 17 insertions(+) create mode 100644 profiles/desc/amdgpu_targets.desc diff --git a/profiles/desc/amdgpu_targets.desc b/profiles/desc/amdgpu_targets.desc new file mode 100644 index ..df013d4f2c08 --- /dev/null +++ b/profiles/desc/amdgpu_targets.desc @@ -0,0 +1,17 @@ +# Copyright 1999-2022 Gentoo Authors. +# Distributed under the terms of the GNU General Public License v2 + +# Referene： +# GPU name and Architecture codename: https://github.com/GPUOpen-Tools/device_info/blob/master/DeviceInfo.cpp +# See also: https://www.coelacanth-dream.com/posts/2019/12/30/did-rid-product-matome-p2/#fn:67 + +gfx803 - Fiji GPU, codename fiji, including Radeon R9 Nano/Fury/FuryX, Radeon Pro Duo, FirePro S9300x2, Radeon Instinct MI8 +gfx900 - Vega GPU, codename vega10, including Radeon Vega Frontier Edition, Radeon RX Vega 56/64, Radeon RX Vega 64 Liquid, Radeon Pro Vega 48/56/64/64X, Radeon Pro WX 8200/9100, Radeon Pro V320/V340/SSG, Radeon Instinct MI25 +gfx906 - Vega GPU, codename vega20, including Radeon (Pro) VII, Radeon Instinct MI50/MI60 +gfx908 - CDNA Accelerator, codename arcturus, including AMD Instinct MI100 Accelerator +gfx90a - CDNA2 Accelerator, codename aldebaran, including AMD Instinct MI200 series Accelerators +gfx1010 - RDNA GPU, codename navi10, including Radeon RX 5700XT/5700/5700M/5700B/5700XTB/5600XT/5600/5600M, Radeon Pro 5700XT/5700, Radeon Pro W5700X/W5700 +gfx1011 - RDNA GPU, codename navi12, including Radeon Pro 5600M/V520 +gfx1012 - RDNA GPU, codename navi14, including Radeon RX 5500XT/5500/5500M/5500XTB/5300/5300M, Radeon Pro 5500XT/5500M/5300/5300M, Radeon Pro W5500X/W5500/W5500M/W5300M +gfx1030 - RDNA2 GPU, codename navi21/sienna cichlid, including Radeon RX 6950XT/6900XT/6800XT/6800, Radeon Pro W6800 +gfx1031 - RDNA2 GPU, codename navi22/navy flounder, including Radeon RX 6750XT/6700XT/6800M/6700M -- 2.34.1
[gentoo-dev] [PATCH v6 1/2] rocm.eclass: new eclass
This eclass provides utilities for ROCm libraries in
https://github.com/ROCmSoftwarePlatform, e.g. rocBLAS, rocFFT.
It contains a USE_EXPAND, amdgpu_targets_*, which handles the GPU
architecture to compile, and keep targets coherent among dependencies.
Packages that depend on ROCm libraries, like cupy, can also make use of
this eclass, mainly specify GPU architecture and it's corresponding
dependencies via USE_EXPAND.
Closes: https://bugs.gentoo.org/810619
Bugs: https://bugs.gentoo.org/817440
Signed-off-by: Yiyang Wu
---
eclass/rocm.eclass | 284
profiles/base/make.defaults | 2 +-
2 files changed, 285 insertions(+), 1 deletion(-)
create mode 100644 eclass/rocm.eclass
diff --git a/eclass/rocm.eclass b/eclass/rocm.eclass
new file mode 100644
index ..1866d6b7cc94
--- /dev/null
+++ b/eclass/rocm.eclass
@@ -0,0 +1,284 @@
+# Copyright 2022 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+# @ECLASS: rocm.eclass
+# @MAINTAINER:
+# Gentoo Science Project
+# @AUTHOR:
+# Yiyang Wu
+# @SUPPORTED_EAPIS: 7 8
+# @BLURB: Common functions and variables for ROCm packages written in HIP
+# @DESCRIPTION:
+# ROCm packages such as sci-libs/* can utilize functions in this
+# eclass. Currently, it handles the AMDGPU_TARGETS variable via USE_EXPAND, so
+# user can use USE flag to control which GPU architecture to compile, and
+# ensure coherence among dependencies. It also specify CXX=hipcc, to let hipcc
+# compile. Another important function is src_test, which checks whether a valid
+# KFD device exists for testing, and then execute the test program.
+#
+# Most ROCm packages use cmake as build system, so this eclass does not export
+# phase functions which overwrites the phase functions in cmake.eclass. Ebuild
+# should explicitly call rocm-{configure,test} in src_configure and src_test.
+#
+# @EXAMPLE:
+# @CODE
+# # Example ebuild for ROCm library in https://github.com/ROCmSoftwarePlatform
+# # whcih depends on rocBLAS
+# inherit cmake rocm
+# # ROCm libraries SRC_URI is usually in form of:
+#
SRC_URI="https://github.com/ROCmSoftwarePlatform/${PN}/archive/rocm-${PV}.tar.gz
-> ${P}.tar.gz"
+# S=${WORKDIR}/${PN}-rocm-${PV}
+# SLOT="0/$(ver_cut 1-2)"
+# IUSE="test"
+# REQUIRED_USE="${ROCM_REQUIRED_USE}"
+# RESTRICT="!test? ( test )"
+#
+# RDEPEND="
+# dev-util/hip
+# sci-libs/rocBLAS:${SLOT}[${ROCM_USEDEP}]
+# "
+#
+# src_configure() {
+# local mycmakeargs=(
+# -DBUILD_CLIENTS_TESTS=$(usex test ON OFF)
+# )
+# rocm-configure
+# }
+#
+# src_test() {
+# rocm-test
+# }
+# @CODE
+#
+# # Example for packages depend on ROCm libraries -- a package depend on
+# # rocBLAS, and use comma seperated ${HCC_AMDGPU_TARGET} to determine GPU
+# # architecture to compile. Requires ROCm version >5.
+# @CODE
+# ROCM_VERSION=5.1
+# inherit rocm
+# IUSE="rocm"
+# REQUIRED_USE="rocm? ( ${ROCM_REQUIRED_USE} )"
+# DEPEND="rocm? ( >=dev-util/hip-${ROCM_VERSION}
+# >=sci-libs/rocBLAS-${ROCM_VERSION}[${ROCM_USEDEP}] )"
+#
+# src_configure() {
+# if use rocm; then
+# local AMDGPU_FLAGS=$(get_amdgpu_flags)
+# export HCC_AMDGPU_TARGET=${AMDGPU_FLAGS//;/,}
+# fi
+# default
+# }
+# @CODE
+
+if [[ ! ${_ROCM_ECLASS} ]]; then
+
+case ${EAPI} in
+ 7|8) ;;
+ *) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;;
+esac
+
+inherit edo
+
+# @ECLASS_VARIABLE: ROCM_VERSION
+# @DEFAULT_UNSET
+# @PRE_INHERIT
+# @DESCRIPTION:
+# The ROCm version of current package. Default is ${PV}, but for other packages
+# that depend on ROCm libraries, this can be set to match the version of
+# required ROCm libraries.
+
+# @ECLASS_VARIABLE: ALL_AMDGPU_TARGETS
+# @INTERNAL
+# @DESCRIPTION:
+# The list of USE flags corresponding to all AMDGPU targets in this ROCm
+# version. The value depends on ${PV}. Architectures and devices map:
+# https://www.coelacanth-dream.com/posts/2019/12/30/did-rid-product-matome-p2
+
+# @ECLASS_VARIABLE: OFFICIAL_AMDGPU_TARGETS
+# @INTERNAL
+# @DESCRIPTION:
+# The list of USE flags corresponding to all officially supported AMDGPU
+# targets in this ROCm version, documented at
+#
https://docs.amd.com/bundle/ROCm-Installation-Guide-v${PV}/page/Prerequisite_Actions.html.
+# USE flag of these architectures will be default on. Depends on ${PV}.
+
+# @ECLASS_VARIABLE: ROCM_REQUIRED_USE
+# @OUTPUT_VARIABLE
+# @DESCRIPTION:
+# Requires at least one AMDGPU target to be compiled.
+# Example use for ROCm libraries:
+# @CODE
+# REQUIRED_USE="${ROCM_REQUIRED_USE}"
+# @CODE
+# Example use for packages that depend on ROCm libraries
+# @CODE
+# IUSE="rocm"
+# REQUIRED_USE="rocm? ( ${ROCM_REQUIRED_USE} )"
+# @CODE
+
+# @ECLASS_VARIABLE: ROCM_USEDEP
+# @OUTPUT_VARIABLE
+# @DESCRIPTION:
+# This is an eclass-generated USE-dependency string which can be used to
+# depend on another ROCm package being built for the same AMDGPU architecture.
+#
+# The generated USE-flag list is compatible with
[gentoo-dev] [PATCH v6 0/2] rocm.eclass: new eclass
The v6 fixes several issues raised in Github PR:
https://github.com/gentoo/gentoo/pull/26784
Changelog against v5:
1. Update outdated examples and comments
2. QA fixes
3. Rename rocm_src_{test,configure} to rocm-{test,configure} to avoid
confusion
4. Simplify rocm-test function
5. Change the reference of AMDGPU targets and GPU product matching.
Yiyang Wu (2):
rocm.eclass: new eclass
profiles/desc: add amdgpu_targets.desc for USE_EXPAND
eclass/rocm.eclass| 284 ++
profiles/base/make.defaults | 2 +-
profiles/desc/amdgpu_targets.desc | 17 ++
3 files changed, 302 insertions(+), 1 deletion(-)
create mode 100644 eclass/rocm.eclass
create mode 100644 profiles/desc/amdgpu_targets.desc
Interdiff against v5:
diff --git a/eclass/rocm.eclass b/eclass/rocm.eclass
index 679b1af54e0a..1866d6b7cc94 100644
--- a/eclass/rocm.eclass
+++ b/eclass/rocm.eclass
@@ -18,13 +18,16 @@
#
# Most ROCm packages use cmake as build system, so this eclass does not export
# phase functions which overwrites the phase functions in cmake.eclass. Ebuild
-# should explicitly call rocm_src_* in src_configure and src_test.
+# should explicitly call rocm-{configure,test} in src_configure and src_test.
#
# @EXAMPLE:
-# # Example for ROCm packages in https://github.com/ROCmSoftwarePlatform
# @CODE
+# # Example ebuild for ROCm library in https://github.com/ROCmSoftwarePlatform
+# # whcih depends on rocBLAS
# inherit cmake rocm
+# # ROCm libraries SRC_URI is usually in form of:
#
SRC_URI="https://github.com/ROCmSoftwarePlatform/${PN}/archive/rocm-${PV}.tar.gz
-> ${P}.tar.gz"
+# S=${WORKDIR}/${PN}-rocm-${PV}
# SLOT="0/$(ver_cut 1-2)"
# IUSE="test"
# REQUIRED_USE="${ROCM_REQUIRED_USE}"
@@ -35,17 +38,15 @@
# sci-libs/rocBLAS:${SLOT}[${ROCM_USEDEP}]
# "
#
-# S=${WORKDIR}/${PN}-rocm-${PV}
-#
# src_configure() {
# local mycmakeargs=(
# -DBUILD_CLIENTS_TESTS=$(usex test ON OFF)
# )
-# rocm_src_configure
+# rocm-configure
# }
#
# src_test() {
-# rocm_src_test
+# rocm-test
# }
# @CODE
#
@@ -53,7 +54,7 @@
# # rocBLAS, and use comma seperated ${HCC_AMDGPU_TARGET} to determine GPU
# # architecture to compile. Requires ROCm version >5.
# @CODE
-# ROCM_VERSION=5
+# ROCM_VERSION=5.1
# inherit rocm
# IUSE="rocm"
# REQUIRED_USE="rocm? ( ${ROCM_REQUIRED_USE} )"
@@ -206,25 +207,31 @@ get_amdgpu_flags() {
# @FUNCTION: check_rw_permission
# @USAGE: check_rw_permission
# @DESCRIPTION:
-# check read and write permissions on specific files.
-# allow using wildcard, for example check_rw_permission /dev/dri/render*
+# check read and write permissions on a specific file, die if no permission.
+# @EXAMPLE:
+# @CODE
+# check_rw_permission /dev/kfd
+# CODE
check_rw_permission() {
- [[ -r $1 ]] && [[ -w $1 ]] || die \
- "Portage do not have read or write permissions on $1! \n Make
sure both are in render group and check the permissions."
+ if [[ ! -r $1 ]] || [[ ! -w $1 ]]; then
+ eerror "Portage do not have read or write permissions on $1!"
+ eerror "Make sure both are in render group and check the
permissions."
+ die "No permissions on $1"
+ fi
}
# == phase functions ==
-# @FUNCTION: rocm_src_configure
+# @FUNCTION: rocm-configure
# @DESCRIPTION:
-# configure rocm packages, and setting common cmake arguments
-rocm_src_configure() {
- # allow acces to hardware
+# configure rocm packages, and setting common cmake arguments. Only for ROCm
+# libraries in https://github.com/ROCmSoftwarePlatform using cmake.
+rocm-configure() {
+ # avoid sandbox violation
addpredict /dev/kfd
addpredict /dev/dri/
mycmakeargs+=(
- -DCMAKE_INSTALL_PREFIX="${EPREFIX}/usr"
-DAMDGPU_TARGETS="$(get_amdgpu_flags)"
-DCMAKE_SKIP_RPATH=TRUE
)
@@ -232,46 +239,45 @@ rocm_src_configure() {
CXX="hipcc" cmake_src_configure
}
-# @FUNCTION: rocm_src_test
+# @FUNCTION: rocm-test
# @DESCRIPTION:
-# Test whether valid GPU device is present. If so, find how to, and execute
test.
-# ROCm packages can have to test mechanism:
+# Test whether valid GPU device is present. If so, execute test.
+# @EXAMPLE:
+# ROCm packages can have two test scenarioes:
# 1. cmake_src_test. MAKEOPTS="-j1" ensures only one test on GPU at a time;
-# 2. one single gtest binary called "${PN,,}"-test;
-# 3. Some package like rocFFT have alternative test like rocfft-selftest;
-# 4. Custome testing binaries like dev-libs/rccl. Use ${ROCM_TESTS} to specify.
-rocm_src_test() {
+# @CODE
+# LD_LIBRARY_PATH= rocm-test --cmake
+# @CODE
+# 2. one gtest binary called "${PN,,}"-test in ${BUILD_DIR}/clients/staging;
+# @CODE
+# cd "${BUILD_DIR}"/clients/staging || die
+# LD_LIBRARY_PATH= rocm-test "${PN,,}"-test
+# @CODE
+# Some packages like rocFFT have two test binaries like rocfft-selftest;
+# packages like dev-libs/rccl have test
Re: [gentoo-dev] Add systemd/merged-usr profiles
Hi,
That really depends.
If the expectation is that everything in /usr/{bin,sbin,lib*} needs to
now fit on / rather than /usr we're queued to re-install a very, very
large number of hosts.
Kind Regards,
Jaco
On 2022/08/31 18:01, Jeff Gazso wrote:
> Just out of curiosity, how much pain is this likely to cause existing
> installations that will need to migrate from a split-usr setup to a
> merged-usr setup?
>
> On Tue, Aug 30, 2022 at 2:28 PM Mike Gilbert wrote:
>
> This patch series adds a "merged-usr" feature profile, and subprofiles
> for each systemd profile.
>
> As background: systemd upstream is preparing to drop support for
> split-usr systems soon. All systemd users on Gentoo will eventually
> need to migrate to a merged-usr system.
>
>
Re: [gentoo-dev] Add systemd/merged-usr profiles
On Wed, Aug 31, 2022 at 12:01 PM Jeff Gazso wrote: > > Just out of curiosity, how much pain is this likely to cause existing > installations that will need to migrate from a split-usr setup to a > merged-usr setup? We haven't deployed this to users, so feedback is limited thus far. At least a handful of Gentoo devs have successfully migrated real systems from split-usr to merged-usr without any major problems. It's a relatively simple process: move the existing files (see sys-apps/merge-usr), flip the split-usr USE flag off, and then run "emege --update --deep --changed-use @world". The only pain point I see is for users with /usr on a separate filesystem and that are not using an appropriate initramfs. This has been an "unsupported" configuration on Gentoo for several years, but there are probably some users who do it anyway.
Re: [gentoo-dev] Add systemd/merged-usr profiles
Just out of curiosity, how much pain is this likely to cause existing installations that will need to migrate from a split-usr setup to a merged-usr setup? On Tue, Aug 30, 2022 at 2:28 PM Mike Gilbert wrote: > This patch series adds a "merged-usr" feature profile, and subprofiles > for each systemd profile. > > As background: systemd upstream is preparing to drop support for > split-usr systems soon. All systemd users on Gentoo will eventually > need to migrate to a merged-usr system. > > >
