Re: [gentoo-dev] [PATCH v2 1/2] rocm.eclass: new eclass
> On Wed, 17 Aug 2022, wuyy wrote:
> Also, I have some long sentences in code, for throwing information. How
> do I nicely wrap such long strings in bash? By incremental assigning it
> to variables I guess?
It's not an absolute rule. Avoid long lines where possible, but don't
jump through hoops to e.g. break long strings in assignments.
>> No double empty lines please. (Pkgcheck should have complained about
>> this.)
> OK. But `pkgcheck scan rocm.eclass` (version 0.10.12) does not
> complains. A missing feature maybe?
Hm, it warns about it in ebuilds but not in eclasses. This looks like a
bug in pkgcheck to me.
>> Please don't use internal Portage variables.
> OK, although I can't find such warnings on
> https://devmanual.gentoo.org/eclass-writing. Maybe an entry should be
> added?
This is in PMS section 12.3.17 "Reserved commands and variables":
https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-13700012.3.17
Let's add it to the devmanual as well:
https://github.com/gentoo/devmanual/pull/301/commits/616323bd33f6aee5b579c24e31a4605c36e08b53
>> > + elif [[ -d "${BUILD_DIR}"/clients/staging ]]; then
>>
>> Quotation marks aren't necessary here.
> What if BUILD_DIR contains spaces?
That's not a problem because word splitting isn't performed inside [[ ]]:
https://www.gnu.org/software/bash/manual/html_node/Conditional-Constructs.html#index-_005b_005b
Ulrich
signature.asc
Description: PGP signature
Re: [gentoo-dev] [PATCH v2 1/2] rocm.eclass: new eclass
Thank you very much for pointing out so much problems I missed!
On Tue, Aug 16, 2022 at 12:41:47AM +0200, Ulrich Mueller wrote:
> Please wrap lines at below 80 character positions. (This applies both to
> documentation and to code.)
>
Thanks. Seems that I forgot yo run re-wrapping in a final round. I don't
see any `pkgcheck scan` complains about this, maybe such check can be
added?
Also, I have some long sentences in code, for throwing information. How
do I nicely wrap such long strings in bash? By incremental assigning it
to variables I guess?
> @EXAMPLE is read as a paragraph, i.e. lines will be wrapped and this is
> how it will look when rendered as eclass manpage:
>
># Example for ROCm packages in https://github.com/ROCmSoftwarePlatform
>inherit cmake rocmSRC_URI="https://github.com/ROCmSoftwarePlat‐
>form/${PN}/archive/rocm-${PV}.tar.gz -> ${P}.tar.gz" SLOT="0/$(ver_cut
>1-2)" IUSE="test" REQUIRED_USE="${ROCM_REQUIRED_USE}" RESTRICT="!test?
>( test )"
>
>[...]
>
> So, you may want to include the whole example in a pair of @CODE tokens.
>
Thanks for pointing out. This is my first eclass and I didn't know how
documents are rendered before.
> > +
> > +if [[ ! ${_ROCM_ECLASS} ]]; then
> > +
> > +case "${EAPI:-0}" in
>
> This should be just ${EAPI} without a fallback to 0. Also, the
> quotation marks are not necessary.
>
> > + 7|8)
> > + ;;
> > + *)
> > + die "Unsupported EAPI=${EAPI} for ${ECLASS}"
>
> Whereas here it should be ${EAPI:-0}.
>
> > + ;;
> > +esac
>
> Or even better, follow standard usage as it can be found in
> multilib-minimal or toolchain-funcs:
>
> case ${EAPI} in
> 7|8) ;;
> *) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;;
> esac
>
I'll take the suggestion. Thanks!
> > +
> > +inherit edo
> > +
> > +
>
> No double empty lines please. (Pkgcheck should have complained about
> this.)
OK. But `pkgcheck scan rocm.eclass` (version 0.10.12) does not
complains. A missing feature maybe?
> > +# @DESCRIPTION:
> > +# The list of USE flags corresponding to all officlially supported AMDGPU
>
> "officially"
>
> > + ROCM_REQUIRED_USE+=" || ("
> > + for gpu_target in ${ALL_AMDGPU_TARGETS[@]}; do
>
> Add a pair of double quotes here.
OK, I'll polish here in v2.
> > + if [[ " ${OFFICIAL_AMDGPU_TARGETS[*]} " =~ " ${gpu_target} "
> > ]]; then
> So OFFICIAL_AMDGPU_TARGETS is an array, but then it's flattened to a
> string for the test? Either use "has" for the test, or don't use an
> array in the first place.
Yes, I should have used a better approach.
> > +get_amdgpu_flags() {
> > + local AMDGPU_TARGET_FLAGS
> > + for gpu_target in ${ALL_AMDGPU_TARGETS[@]}; do
>
> Double quotes please.
>
Thanks for finding such problem once again.
> > +# @FUNCTION: check_rw_permission
> > +# @USAGE: check_rw_permission
> > +# @DESCRIPTION:
> > +# check read and write permissions on specific files.
> > +# allow using wildcard, for example check_rw_permission /dev/dri/render*
> > +check_rw_permission() {
> > + [[ -r "$1" ]] && [[ -w "$1" ]] || die \
>
> Quotation marks aren't necessary here.
Yes, and after I perform another check, this function is implemented
incorrectly -- it is designed to accept wildcards (later in
rocm_src_test, there is check_rw_permission /dev/dri/render*) but when
bash pass the expanded argument to list, $1 is only the first matching
file. So I should think of a correct implementation, or stop using
wildcards. This is a bug (rare to encounter however), and I need to fix
it in v2.
>
> > + "${PORTAGE_USERNAME} do not have read or write permissions on
> > $1! \n Make sure ${PORTAGE_USERNAME} is in render group and check the
> > permissions."
>
> Please don't use internal Portage variables.
OK, although I can't find such warnings on
https://devmanual.gentoo.org/eclass-writing. Maybe an entry should be
added?
> > + elif [[ -d "${BUILD_DIR}"/clients/staging ]]; then
>
> Quotation marks aren't necessary here.
What if BUILD_DIR contains spaces?
> > + cd "${BUILD_DIR}/clients/staging" || die "Test directory not
> > found!"
> > + for test_program in "${PN,,}-"*test; do
> > + if [[ -x ${test_program} ]]; then
> > + edob ./${test_program}
> > + else
> > + die "The test program ${test_program} does not
> > exist or cannot be excuted!"
> > + fi
> > + done
> > + elif [[ ! -z "${ROCM_TESTS}" ]]; then
>
> Again, no quotation marks.
>
> Also, why the double negation? IMHO "-n" would be better readable.
Yeah, I don't understand myself either.
>
> > + for test_program in "${ROCM_TESTS}"; do
>
> What is this supposed to do? The loop will be executed exactly once,
> whatever ROCM_TESTS contains.
Well, I hope ROCM_TESTS can have multiple executables, so that's why
it have the
Re: [gentoo-dev] [PATCH v2 1/2] rocm.eclass: new eclass
> On Mon, 15 Aug 2022, Yiyang Wu wrote:
> +_rocm_set_globals() {
> + case ${ROCM_VERSION:-${PV}} in
> + 4*)
What will happen when there's a version 40 in the future?
signature.asc
Description: PGP signature
Re: [gentoo-dev] [PATCH v2 1/2] rocm.eclass: new eclass
> On Tue, 16 Aug 2022, Ulrich Mueller wrote:
>> +IUSE+=" ${gpu_target/#/+amdgpu_targets_}"
> Why "+="? I don't see any previous assignment of IUSE.
Please ignore this. I had missed that it's inside a loop.
signature.asc
Description: PGP signature
Re: [gentoo-dev] [PATCH v2 1/2] rocm.eclass: new eclass
> On Mon, 15 Aug 2022, Yiyang Wu wrote:
> diff --git a/eclass/rocm.eclass b/eclass/rocm.eclass
> new file mode 100644
> index ..8ca2c051ddce
> --- /dev/null
> +++ b/eclass/rocm.eclass
> @@ -0,0 +1,275 @@
> +# Copyright 2022 Gentoo Authors
> +# Distributed under the terms of the GNU General Public License v2
> +
> +# @ECLASS: rocm.eclass
> +# @MAINTAINER:
> +# Gentoo Science Project
> +# @AUTHOR:
> +# Yiyang Wu
> +# @SUPPORTED_EAPIS: 7 8
> +# @BLURB: Common functions and variables for ROCm packages written in HIP
> +# @DESCRIPTION:
> +# ROCm packages such as sci-libs/* can utilize functions in this
> eclass.
> +# Currently, it handles the AMDGPU_TARGETS variable via USE_EXPAND, so user
> can
> +# use USE flag to control which GPU architecture to compile, and ensure
> coherence
> +# among dependencies. It also specify CXX=hipcc, to let hipcc compile.
> Another
> +# important function is src_test, which checks whether a valid KFD device
> exists
> +# for testing, and then execute the test program.
Please wrap lines at below 80 character positions. (This applies both to
documentation and to code.)
> +#
> +# Most ROCm packages use cmake as build system, so this eclass does not
> export
> +# phase functions which overwrites the phase functions in cmake.eclass.
> Ebuild
> +# should explicitly call rocm_src_* in src_configure and src_test.
> +#
> +# @EXAMPLE:
> +# # Example for ROCm packages in https://github.com/ROCmSoftwarePlatform
> +# inherit cmake rocm
> +#
> SRC_URI="https://github.com/ROCmSoftwarePlatform/${PN}/archive/rocm-${PV}.tar.gz
> -> ${P}.tar.gz"
> +# SLOT="0/$(ver_cut 1-2)"
> +# IUSE="test"
> +# REQUIRED_USE="${ROCM_REQUIRED_USE}"
> +# RESTRICT="!test? ( test )"
> +#
> +# RDEPEND="
> +# dev-util/hip
> +# sci-libs/rocBLAS:${SLOT}[${ROCM_USEDEP}]
> +# "
> +#
> +# S=${WORKDIR}/${PN}-rocm-${PV}
> +#
> +# src_configure() {
> +# local mycmakeargs=(
> +# -DBUILD_CLIENTS_TESTS=$(usex test ON OFF)
> +# )
> +# rocm_src_configure
> +# }
> +#
> +# src_test() {
> +# rocm_src_test
> +# }
> +#
> +# # Example for packages depend on ROCm libraries -- a package depend on
> +# # rocBLAS, and use comma seperated ${HCC_AMDGPU_TARGET} to determine GPU
> +# # architecture to compile. Requires ROCm version >5.
> +# ROCM_VERSION=5
> +# inherit rocm
> +# IUSE="rocm"
> +# REQUIRED_USE="rocm? ( ${ROCM_REQUIRED_USE} )"
> +# DEPEND="rocm? ( >=dev-util/hip-${ROCM_VERSION}
> +# >=sci-libs/rocBLAS-${ROCM_VERSION}[${ROCM_USEDEP}] )"
> +#
> +# src_configure() {
> +# if use rocm; then
> +# local AMDGPU_FLAGS=$(get_amdgpu_flags)
> +# export HCC_AMDGPU_TARGET=${AMDGPU_FLAGS//;/,}
> +# fi
> +# default
> +# }
@EXAMPLE is read as a paragraph, i.e. lines will be wrapped and this is
how it will look when rendered as eclass manpage:
# Example for ROCm packages in https://github.com/ROCmSoftwarePlatform
inherit cmake rocmSRC_URI="https://github.com/ROCmSoftwarePlat‐
form/${PN}/archive/rocm-${PV}.tar.gz -> ${P}.tar.gz" SLOT="0/$(ver_cut
1-2)" IUSE="test" REQUIRED_USE="${ROCM_REQUIRED_USE}" RESTRICT="!test?
( test )"
[...]
So, you may want to include the whole example in a pair of @CODE tokens.
> +
> +if [[ ! ${_ROCM_ECLASS} ]]; then
> +
> +case "${EAPI:-0}" in
This should be just ${EAPI} without a fallback to 0. Also, the
quotation marks are not necessary.
> + 7|8)
> + ;;
> + *)
> + die "Unsupported EAPI=${EAPI} for ${ECLASS}"
Whereas here it should be ${EAPI:-0}.
> + ;;
> +esac
Or even better, follow standard usage as it can be found in
multilib-minimal or toolchain-funcs:
case ${EAPI} in
7|8) ;;
*) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;;
esac
> +
> +inherit edo
> +
> +
No double empty lines please. (Pkgcheck should have complained about
this.)
> +# @ECLASS_VARIABLE: ROCM_VERSION
> +# @DEFAULT_UNSET
> +# @PRE_INHERIT
> +# @DESCRIPTION:
> +# The ROCm version of current package. Default is ${PV}, but for other
> packages
> +# that depend on ROCm libraries, this can be set to match the version of
> +# required ROCm libraries.
> +
> +# @ECLASS_VARIABLE: ALL_AMDGPU_TARGETS
> +# @INTERNAL
> +# @DESCRIPTION:
> +# The list of USE flags corresponding to all AMDGPU targets in this ROCm
> +# version. The value depends on ${PV}. Architectures and devices map:
> +# https://www.coelacanth-dream.com/posts/2019/12/30/did-rid-product-matome-p2
> +
> +# @ECLASS_VARIABLE: OFFICIAL_AMDGPU_TARGETS
> +# @INTERNAL
> +# @DESCRIPTION:
> +# The list of USE flags corresponding to all officlially supported AMDGPU
"officially"
> +# targets in this ROCm version, documented at
> +#
> https://docs.amd.com/bundle/ROCm-Installation-Guide-v${PV}/page/Prerequisite_Actions.html.
> +# USE flag of these architectures will be default on. Depends on ${PV}.
> +
> +# @ECLASS_VARIABLE: ROCM_REQUIRED_USE
> +# @OUTPUT_VARIABLE
> +# @DESCRIPTION:
> +#
[gentoo-dev] [PATCH v2 1/2] rocm.eclass: new eclass
This eclass provides utilities for ROCm libraries in
https://github.com/ROCmSoftwarePlatform, e.g. rocBLAS, rocFFT.
It contains a USE_EXPAND, amdgpu_targets_*, which handles the GPU
architecture to compile, and keep targets coherent among dependencies.
Packages that depend on ROCm libraries, like cupy, can also make use of
this eclass, mainly specify GPU architecture and it's corresponding
dependencies via USE_EXPAND.
Closes: https://bugs.gentoo.org/810619
Signed-off-by: YiyangWu
---
eclass/rocm.eclass | 275
profiles/base/make.defaults | 2 +-
2 files changed, 276 insertions(+), 1 deletion(-)
create mode 100644 eclass/rocm.eclass
diff --git a/eclass/rocm.eclass b/eclass/rocm.eclass
new file mode 100644
index ..8ca2c051ddce
--- /dev/null
+++ b/eclass/rocm.eclass
@@ -0,0 +1,275 @@
+# Copyright 2022 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+# @ECLASS: rocm.eclass
+# @MAINTAINER:
+# Gentoo Science Project
+# @AUTHOR:
+# Yiyang Wu
+# @SUPPORTED_EAPIS: 7 8
+# @BLURB: Common functions and variables for ROCm packages written in HIP
+# @DESCRIPTION:
+# ROCm packages such as sci-libs/* can utilize functions in this
eclass.
+# Currently, it handles the AMDGPU_TARGETS variable via USE_EXPAND, so user can
+# use USE flag to control which GPU architecture to compile, and ensure
coherence
+# among dependencies. It also specify CXX=hipcc, to let hipcc compile. Another
+# important function is src_test, which checks whether a valid KFD device
exists
+# for testing, and then execute the test program.
+#
+# Most ROCm packages use cmake as build system, so this eclass does not export
+# phase functions which overwrites the phase functions in cmake.eclass. Ebuild
+# should explicitly call rocm_src_* in src_configure and src_test.
+#
+# @EXAMPLE:
+# # Example for ROCm packages in https://github.com/ROCmSoftwarePlatform
+# inherit cmake rocm
+#
SRC_URI="https://github.com/ROCmSoftwarePlatform/${PN}/archive/rocm-${PV}.tar.gz
-> ${P}.tar.gz"
+# SLOT="0/$(ver_cut 1-2)"
+# IUSE="test"
+# REQUIRED_USE="${ROCM_REQUIRED_USE}"
+# RESTRICT="!test? ( test )"
+#
+# RDEPEND="
+# dev-util/hip
+# sci-libs/rocBLAS:${SLOT}[${ROCM_USEDEP}]
+# "
+#
+# S=${WORKDIR}/${PN}-rocm-${PV}
+#
+# src_configure() {
+# local mycmakeargs=(
+# -DBUILD_CLIENTS_TESTS=$(usex test ON OFF)
+# )
+# rocm_src_configure
+# }
+#
+# src_test() {
+# rocm_src_test
+# }
+#
+# # Example for packages depend on ROCm libraries -- a package depend on
+# # rocBLAS, and use comma seperated ${HCC_AMDGPU_TARGET} to determine GPU
+# # architecture to compile. Requires ROCm version >5.
+# ROCM_VERSION=5
+# inherit rocm
+# IUSE="rocm"
+# REQUIRED_USE="rocm? ( ${ROCM_REQUIRED_USE} )"
+# DEPEND="rocm? ( >=dev-util/hip-${ROCM_VERSION}
+# >=sci-libs/rocBLAS-${ROCM_VERSION}[${ROCM_USEDEP}] )"
+#
+# src_configure() {
+# if use rocm; then
+# local AMDGPU_FLAGS=$(get_amdgpu_flags)
+# export HCC_AMDGPU_TARGET=${AMDGPU_FLAGS//;/,}
+# fi
+# default
+# }
+
+if [[ ! ${_ROCM_ECLASS} ]]; then
+
+case "${EAPI:-0}" in
+ 7|8)
+ ;;
+ *)
+ die "Unsupported EAPI=${EAPI} for ${ECLASS}"
+ ;;
+esac
+
+inherit edo
+
+
+# @ECLASS_VARIABLE: ROCM_VERSION
+# @DEFAULT_UNSET
+# @PRE_INHERIT
+# @DESCRIPTION:
+# The ROCm version of current package. Default is ${PV}, but for other packages
+# that depend on ROCm libraries, this can be set to match the version of
+# required ROCm libraries.
+
+# @ECLASS_VARIABLE: ALL_AMDGPU_TARGETS
+# @INTERNAL
+# @DESCRIPTION:
+# The list of USE flags corresponding to all AMDGPU targets in this ROCm
+# version. The value depends on ${PV}. Architectures and devices map:
+# https://www.coelacanth-dream.com/posts/2019/12/30/did-rid-product-matome-p2
+
+# @ECLASS_VARIABLE: OFFICIAL_AMDGPU_TARGETS
+# @INTERNAL
+# @DESCRIPTION:
+# The list of USE flags corresponding to all officlially supported AMDGPU
+# targets in this ROCm version, documented at
+#
https://docs.amd.com/bundle/ROCm-Installation-Guide-v${PV}/page/Prerequisite_Actions.html.
+# USE flag of these architectures will be default on. Depends on ${PV}.
+
+# @ECLASS_VARIABLE: ROCM_REQUIRED_USE
+# @OUTPUT_VARIABLE
+# @DESCRIPTION:
+# Requires at least one AMDGPU target to be compiled.
+# Example use for ROCm libraries:
+# REQUIRED_USE="${ROCM_REQUIRED_USE}"
+# Example use for packages that depend on ROCm libraries
+# IUSE="rocm"
+# REQUIRED_USE="rocm? ( ${ROCM_REQUIRED_USE} )"
+
+# @ECLASS_VARIABLE: ROCM_USEDEP
+# @OUTPUT_VARIABLE
+# @DESCRIPTION:
+# This is an eclass-generated USE-dependency string which can be used to
+# depend on another ROCm package being built for the same AMDGPU architecture.
+#
+# The generated USE-flag list is compatible with packages using rocm.eclass.
+#
+# Example use:
+# @CODE
+# DEPEND="sci-libs/rocBLAS[${ROCM_USEDEP}]"
+# @CODE
+
+# @FUNCTION: _rocm_set_globals
+#
