Re: [gentoo-dev] [PATCH] cargo.eclass: Optimize crate unpacking
On Sun, 2024-05-12 at 19:22 +0200, Florian Schmaus wrote: > On 12/05/2024 04.26, Michał Górny wrote: > > Unpack crates in parallel using xargs to utilize multicore systems > > better. Perform checksumming via a single sha256sum invocation. > > > > For dev-python/watchfiles, this speeds up unpacking on my machine > > from 2.6 s to 0.75 s (warm cache). > > > > Signed-off-by: Michał Górny > > --- > > eclass/cargo.eclass | 56 ++--- > > 1 file changed, 33 insertions(+), 23 deletions(-) > > > > diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass > > index 0f2da982f60c..5a16d3a30528 100644 > > --- a/eclass/cargo.eclass > > +++ b/eclass/cargo.eclass > > @@ -329,40 +329,50 @@ _cargo_gen_git_config() { > > cargo_src_unpack() { > > debug-print-function ${FUNCNAME} "$@" > > > > - mkdir -p "${ECARGO_VENDOR}" || die > > - mkdir -p "${S}" || die > > + mkdir -p "${ECARGO_VENDOR}" "${S}" || die > > > > local archive shasum pkg > > + local crates=() > > for archive in ${A}; do > > case "${archive}" in > > *.crate) > > - # when called by pkgdiff-mg, do not unpack > > crates > > - [[ ${PKGBUMPING} == ${PVR} ]] && continue > > - > > - ebegin "Loading ${archive} into Cargo registry" > > - tar -xf "${DISTDIR}"/${archive} -C > > "${ECARGO_VENDOR}/" || die > > - # generate sha256sum of the crate itself as > > cargo needs this > > - shasum=$(sha256sum "${DISTDIR}"/${archive} | > > cut -d ' ' -f 1) > > - pkg=$(basename ${archive} .crate) > > - cat <<- EOF > > > ${ECARGO_VENDOR}/${pkg}/.cargo-checksum.json > > - { > > - "package": "${shasum}", > > - "files": {} > > - } > > - EOF > > - # if this is our target package we need it in > > ${WORKDIR} too > > - # to make ${S} (and handle any revisions too) > > - if [[ ${P} == ${pkg}* ]]; then > > - tar -xf "${DISTDIR}"/${archive} -C > > "${WORKDIR}" || die > > - fi > > - eend $? > > + crates+=( "${archive}" ) > > ;; > > *) > > - unpack ${archive} > > + unpack "${archive}" > > ;; > > esac > > done > > > > + if [[ ${PKGBUMPING} != ${PVR} ]]; then > > + pushd "${DISTDIR}" >/dev/null || die > > + > > + ebegin "Unpacking crates" > > + printf '%s\0' "${crates[@]}" | > > + xargs -0 -P "$(makeopts_jobs)" -n 1 -- \ > > Consider using get_makeopts_jobs instead of makeopts_jobs, as it > searches more variables for --jobs. Whose bright idea was to add a second similarly named function that does roughly the same thing but apparently differently? It can hardly get more confusing. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [PATCH] cargo.eclass: Optimize crate unpacking
On 12/05/2024 04.26, Michał Górny wrote: Unpack crates in parallel using xargs to utilize multicore systems better. Perform checksumming via a single sha256sum invocation. For dev-python/watchfiles, this speeds up unpacking on my machine from 2.6 s to 0.75 s (warm cache). Signed-off-by: Michał Górny --- eclass/cargo.eclass | 56 ++--- 1 file changed, 33 insertions(+), 23 deletions(-) diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass index 0f2da982f60c..5a16d3a30528 100644 --- a/eclass/cargo.eclass +++ b/eclass/cargo.eclass @@ -329,40 +329,50 @@ _cargo_gen_git_config() { cargo_src_unpack() { debug-print-function ${FUNCNAME} "$@" - mkdir -p "${ECARGO_VENDOR}" || die - mkdir -p "${S}" || die + mkdir -p "${ECARGO_VENDOR}" "${S}" || die local archive shasum pkg + local crates=() for archive in ${A}; do case "${archive}" in *.crate) - # when called by pkgdiff-mg, do not unpack crates - [[ ${PKGBUMPING} == ${PVR} ]] && continue - - ebegin "Loading ${archive} into Cargo registry" - tar -xf "${DISTDIR}"/${archive} -C "${ECARGO_VENDOR}/" || die - # generate sha256sum of the crate itself as cargo needs this - shasum=$(sha256sum "${DISTDIR}"/${archive} | cut -d ' ' -f 1) - pkg=$(basename ${archive} .crate) - cat <<- EOF > ${ECARGO_VENDOR}/${pkg}/.cargo-checksum.json - { - "package": "${shasum}", - "files": {} - } - EOF - # if this is our target package we need it in ${WORKDIR} too - # to make ${S} (and handle any revisions too) - if [[ ${P} == ${pkg}* ]]; then - tar -xf "${DISTDIR}"/${archive} -C "${WORKDIR}" || die - fi - eend $? + crates+=( "${archive}" ) ;; *) - unpack ${archive} + unpack "${archive}" ;; esac done + if [[ ${PKGBUMPING} != ${PVR} ]]; then + pushd "${DISTDIR}" >/dev/null || die + + ebegin "Unpacking crates" + printf '%s\0' "${crates[@]}" | + xargs -0 -P "$(makeopts_jobs)" -n 1 -- \ Consider using get_makeopts_jobs instead of makeopts_jobs, as it searches more variables for --jobs. N.B.: If this where asking for a load-average limit, then using get_makeopts_loadavg would be the ideal way to pick up portage's default wrt --load-average. Therefore we should IMHO encourage using the get_makeopts_* functions over the (legacy?) non-get_ variants. + tar -x -C "${ECARGO_VENDOR}" -f + assert + eend $? + + while read -d '' -r shasum archive; do + pkg=${archive%.crate} + cat <<- EOF > ${ECARGO_VENDOR}/${pkg}/.cargo-checksum.json || die + { + "package": "${shasum}", + "files": {} + } + EOF + + # if this is our target package we need it in ${WORKDIR} too + # to make ${S} (and handle any revisions too) + if [[ ${P} == ${pkg}* ]]; then + tar -xf "${archive}" -C "${WORKDIR}" || die + fi + done < <(sha256sum -z "${crates[@]}" || die) + + popd >/dev/null || die + fi + cargo_gen_config } OpenPGP_0x8CAC2A9678548E35.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH 3/3] ruby-ng-gnome2.eclass: allow EAPI 8
On Sun, 2024-05-12 at 10:12 +0200, Ulrich Mueller wrote: > > > > > > On Sun, 12 May 2024, Hans de Graaff wrote: > > > -# @SUPPORTED_EAPIS: 7 > > +# @SUPPORTED_EAPIS: 7, 8 > > No comma there. Fixed, thanks! Hans signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Mysterious behavior of app-text/qpdfview
On Sun, 12 May 2024, netfab wrote: View -> Convert to grayscale That's it! I'm absolutely sure that I never enabled this option on purpose. Probably, some accidental mouse click. Sorry for the noise, Andrey
Re: [gentoo-dev] Mysterious behavior of app-text/qpdfview
Hi, Le 12/05/24 à 10:02, Andrey Grozin a tapoté : > Any ideas where to start investigating what has become broken? Are you sure that grayscale option is not enabled ? View -> Convert to grayscale
[gentoo-dev] Mysterious behavior of app-text/qpdfview
Hello *, The behavior of app-text/qpdfview has changed recently (approx. durung the last week) in a mysterious way. qpdfview itself has not changed (the revent -0.5_p1 is irrelevant - I'm discussing the stable version 0.5). Until recently, qpdfview displayed color pdf files as expected - in color. Now it displays them as black-and-white (more exactly, as levels of grey). The source of qpdfview itself has not changed - it's the same version 0.5 which worked fine a week ago. So, it's the result of an upgrade of some library on which it depends. It depends on a number of Qt5 libraries (some of them were upgraded recently); on mesa (indirectly); on poppler; and on some things which don't seem relevant for the color/black-and-white issue. At the moment I have no idea where to start investigating the problem. Just to check: xpdf, mupdf, gv, zathura display these color pdf files correctly, in color. They depend on many of the same libraries; if some of the recently upgraded libraries suddently has become broken, this does not influence these other pdf viewers. Any ideas where to start investigating what has become broken? Thanks in advance, Andrey
Re: [gentoo-dev] [PATCH 3/3] ruby-ng-gnome2.eclass: allow EAPI 8
> On Sun, 12 May 2024, Hans de Graaff wrote: > -# @SUPPORTED_EAPIS: 7 > +# @SUPPORTED_EAPIS: 7, 8 No comma there. signature.asc Description: PGP signature