Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package pdsh for openSUSE:Factory checked in at 2023-01-03 15:05:49 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/pdsh (Old) and /work/SRC/openSUSE:Factory/.pdsh.new.1563 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "pdsh" Tue Jan 3 15:05:49 2023 rev:46 rq:1046438 version:2.34 Changes: -------- --- /work/SRC/openSUSE:Factory/pdsh/pdsh.changes 2022-11-04 17:42:10.111068326 +0100 +++ /work/SRC/openSUSE:Factory/.pdsh.new.1563/pdsh.changes 2023-01-03 15:06:15.478822449 +0100 @@ -1,0 +2,19 @@ +Tue Jan 3 07:41:31 UTC 2023 - Egbert Eich <e...@suse.com> + +- Backport a number of features and fixes from the git master branch + (boo#1206795): + * Add '-C' option on Slrum plugin to restrict selected nodes to ones + with the specified features present: + slurm-add-C-features-constraint.patch + slurm-add-documentation-for-C.patch + * Add option '-k' to the ssh plugin to fail faster on connection failures: + fail-fast-on-ssh-errors-or-non-zero-return-code.patch + doc-fast-fail-update.patch + * Fix use of strchr + hostlist-fix-use-of-strchr.patch + * Dshbak: Fix uninitialized use of $tag on empty input + dshbak-fix-uninitialized-use-of-tag-on-empty-input.patch + * Dsh: Release a lock that is no longer used in dsh() + Release-a-lock-that-is-no-longer-used-in-dsh.patch + +------------------------------------------------------------------- New: ---- Release-a-lock-that-is-no-longer-used-in-dsh.patch doc-fast-fail-update.patch dshbak-fix-uninitialized-use-of-tag-on-empty-input.patch fail-fast-on-ssh-errors-or-non-zero-return-code.patch hostlist-fix-use-of-strchr.patch slurm-add-C-features-constraint.patch slurm-add-documentation-for-C.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ pdsh.spec ++++++ --- /var/tmp/diff_new_pack.OemlHp/_old 2023-01-03 15:06:15.934825110 +0100 +++ /var/tmp/diff_new_pack.OemlHp/_new 2023-01-03 15:06:15.942825157 +0100 @@ -1,7 +1,7 @@ # # spec file # -# Copyright (c) 2022 SUSE LLC +# Copyright (c) 2023 SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -62,6 +62,13 @@ License: GPL-2.0-or-later Group: Productivity/Clustering/Computing Source: https://github.com/chaos/%{pname}/releases/download/%{pname}-%{version}/%{pname}-%{version}.tar.gz +Patch1: slurm-add-C-features-constraint.patch +Patch2: slurm-add-documentation-for-C.patch +Patch3: hostlist-fix-use-of-strchr.patch +Patch4: dshbak-fix-uninitialized-use-of-tag-on-empty-input.patch +Patch5: Release-a-lock-that-is-no-longer-used-in-dsh.patch +Patch6: fail-fast-on-ssh-errors-or-non-zero-return-code.patch +Patch7: doc-fast-fail-update.patch %description Pdsh is a multithreaded remote shell client which executes commands on @@ -124,6 +131,13 @@ %prep %setup -q -n %{pname}-%{version} +%patch1 -p1 +%patch2 -p1 +%patch3 -p1 +%patch4 -p1 +%patch5 -p1 +%patch6 -p1 +%patch7 -p1 %build export CFLAGS="%{optflags} -fno-strict-aliasing" ++++++ Release-a-lock-that-is-no-longer-used-in-dsh.patch ++++++ From: ycaibb <yca...@gmail.com> Date: Thu Sep 9 10:52:21 2021 +0800 Subject: Release a lock that is no longer used in dsh() Patch-mainline: Not yet Git-repo: https://github.com/chaos/pdsh Git-commit: 5bf75e2b4ad6178eacad426fb76addcbfd7a8841 References: In dsh(), the threadcount_mutex is never released. While technically not a bug, this could cause a problem or confusion in the future if a lock is attempted on the mutex. Thus, we fix it by releasing the lock at the end of dsh(). Signed-off-by: ycaibb <yca...@gmail.com> Signed-off-by: Egbert Eich <e...@suse.de> --- src/pdsh/dsh.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/pdsh/dsh.c b/src/pdsh/dsh.c index a2c9e0a..1957421 100644 --- a/src/pdsh/dsh.c +++ b/src/pdsh/dsh.c @@ -1192,6 +1192,7 @@ int dsh(opt_t * opt) dsh_mutex_lock(&threadcount_mutex); while (threadcount > 0) pthread_cond_wait(&threadcount_cond, &threadcount_mutex); + dsh_mutex_unlock(&threadcount_mutex); if (debug) _dump_debug_stats(rshcount); ++++++ _multibuild ++++++ --- /var/tmp/diff_new_pack.OemlHp/_old 2023-01-03 15:06:15.982825391 +0100 +++ /var/tmp/diff_new_pack.OemlHp/_new 2023-01-03 15:06:15.986825414 +0100 @@ -6,6 +6,6 @@ <package>20_03</package> <package>20_11</package> <package>22_05</package> - --> +--> </multibuild> ++++++ doc-fast-fail-update.patch ++++++ From: Jerry Mannil <jman...@amazon.com> Date: Mon Jun 13 11:26:39 2022 -0700 Subject: doc: fast-fail update Patch-mainline: Not yet Git-repo: https://github.com/chaos/pdsh Git-commit: 36f2f561d336abd07b587baf91a02835d68d7ea7 References: Add '-k' option for fast-fail Signed-off-by: Egbert Eich <e...@suse.de> --- doc/pdsh.1.in | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/pdsh.1.in b/doc/pdsh.1.in index ccd529d..d4d7f0d 100644 --- a/doc/pdsh.1.in +++ b/doc/pdsh.1.in @@ -144,10 +144,13 @@ as if they had been given to \fB\-w\fR and preceeded with the minus `-' character. .SH "Standard pdsh options" -.TP +.TP .I "-S" Return the largest of the remote command return values. .TP +.I "-k" +Fail fast on connect failure or non-zero return code. +.TP .I "-h" Output usage menu and quit. A list of available rcmd modules will also be printed at the end of the usage message. ++++++ dshbak-fix-uninitialized-use-of-tag-on-empty-input.patch ++++++ From: Mark A. Grondona <mark.grond...@gmail.com> Date: Wed Aug 11 15:26:10 2021 -0700 Subject: dshbak: fix uninitialized use of $tag on empty input Patch-mainline: Not yet Git-repo: https://github.com/chaos/pdsh Git-commit: b32d9d312b52447a68d1ee52fbbe0394f857d87a References: Problem: The fix for hostnames of `0` in dshbak, i.e. the following commit commit dee6adf334376da560afa0f8beebe4cae36d8e7c Author: Mark A. Grondona <mgrond...@llnl.gov> Date: Thu Jun 25 13:37:51 2015 -0700 dshbak: Handle hostname of "0" @garlick found that dshbak can't handle the smallest of numbers. This is because dshbak skips lines where $tag resolves to false. But in perl, 0 is also false. What we really want to check for is an "empty" tag, so fix up the test to reflect that. Resolves #70. breaks handling of empty input to dshbak, since now the variable "$tag" can be used when it is uninitialized. The proper fix for issue #70 was to check if the variable was either uninitialized or an empty string. Fixes #132 Signed-off-by: Egbert Eich <e...@suse.de> --- scripts/dshbak | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/dshbak b/scripts/dshbak index a6e17eb..fbb4e71 100755 --- a/scripts/dshbak +++ b/scripts/dshbak @@ -112,7 +112,7 @@ sub process_lines while (<>) { my ($tag, $data) = m/^\s*(\S+?)\s*: ?(.*\n)$/; # Ignore lines that aren't prefixed with a hostname: - next unless "$tag" ne ""; + next unless defined $tag and "$tag" ne ""; push(@{$lines{$tag}}, $data); } return %lines; ++++++ fail-fast-on-ssh-errors-or-non-zero-return-code.patch ++++++ From: Jerry Mannil <jman...@amazon.com> Date: Fri Jun 10 00:03:06 2022 -0700 Subject: fail fast on ssh errors or non-zero return code Patch-mainline: Not yet Git-repo: https://github.com/chaos/pdsh Git-commit: df2c464f246fbc5179f090873214df4c7c16a79b References: * Added "-k" option to enable fail fast option * Fail fast on non-zero error code Fixes #139 Signed-off-by: Egbert Eich <e...@suse.de> --- src/pdsh/dsh.c | 2 +- src/pdsh/opt.c | 10 +++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/src/pdsh/dsh.c b/src/pdsh/dsh.c index 1957421..cad8d41 100644 --- a/src/pdsh/dsh.c +++ b/src/pdsh/dsh.c @@ -753,7 +753,7 @@ static void *_rsh_thread(void *args) a->rc = rv; /* if a single qshell thread fails, terminate whole job */ - if (a->kill_on_fail && a->state == DSH_FAILED) { + if (a->kill_on_fail && ((a->state == DSH_FAILED) || (a->rc > 0))) { _fwd_signal(SIGTERM); errx("%p: terminating all processes\n"); } diff --git a/src/pdsh/opt.c b/src/pdsh/opt.c index 790fa2d..44b85dc 100644 --- a/src/pdsh/opt.c +++ b/src/pdsh/opt.c @@ -66,7 +66,8 @@ #define OPT_USAGE_DSH "\ Usage: pdsh [-options] command ...\n\ --S return largest of remote command return values\n" +-S return largest of remote command return values\n\ +-k fail fast on connect failure or non-zero return code\n" /* -s option only useful on AIX */ #if HAVE_MAGIC_RSHELL_CLEANUP @@ -113,9 +114,9 @@ Usage: rpdcp [-options] src [src2...] dir\n\ /* undocumented "-K" option - keep domain name in output */ #if HAVE_MAGIC_RSHELL_CLEANUP -#define DSH_ARGS "sS" +#define DSH_ARGS "sSk" #else -#define DSH_ARGS "S" +#define DSH_ARGS "Sk" #endif #define PCP_ARGS "pryzZe:" #define GEN_ARGS "hLNKR:M:t:cqf:w:x:l:u:bI:dVT:Q" @@ -691,6 +692,9 @@ void opt_args(opt_t * opt, int argc, char *argv[]) else goto test_module_option; break; + case 'k': + opt->kill_on_fail = true; + break; default: test_module_option: if (mod_process_opt(opt, c, optarg) < 0) _usage(opt); ++++++ hostlist-fix-use-of-strchr.patch ++++++ From: Mark A. Grondona <mark.grond...@gmail.com> Date: Thu Aug 12 18:51:55 2021 -0700 Subject: hostlist: fix use of strchr() Patch-mainline: Not yet Git-repo: https://github.com/chaos/pdsh Git-commit: 8ad5f4ff5dcccb87d6723cb36aa8097b27fdae6a References: Problem: Comparison of strchr() result to NUL results in compiler warnings. Test strchr() result against NULL instead to avoid the compiler warnings. Signed-off-by: Egbert Eich <e...@suse.de> --- src/common/hostlist.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/common/hostlist.c b/src/common/hostlist.c index d7c9bb9..69edc43 100644 --- a/src/common/hostlist.c +++ b/src/common/hostlist.c @@ -369,7 +369,7 @@ static char * _next_tok(char *sep, char **str) int level = 0; /* push str past any leading separators */ - while (**str != '\0' && strchr(sep, **str) != '\0') + while (**str != '\0' && strchr(sep, **str)) (*str)++; if (**str == '\0') @@ -378,7 +378,7 @@ static char * _next_tok(char *sep, char **str) /* assign token ptr */ tok = *str; - while ( **str != '\0' && + while ( **str != '\0' && (level != 0 || strchr(sep, **str) == NULL) ) { if ( **str == '[' ) level++; else if ( **str == ']' ) level--; @@ -386,7 +386,7 @@ static char * _next_tok(char *sep, char **str) } /* nullify consecutive separators and push str beyond them */ - while (**str != '\0' && strchr(sep, **str) != '\0') + while (**str != '\0' && strchr(sep, **str) != NULL) *(*str)++ = '\0'; return tok; ++++++ slurm-add-C-features-constraint.patch ++++++ From: Dylan Simon <dy...@dylex.net> Date: Mon Nov 9 20:28:23 2020 -0500 Subject: slurm: add -C features constraint Patch-mainline: Not yet Git-repo: https://github.com/chaos/pdsh Git-commit: 4955a92987bf42aa1edeb83702a268c0fc8119d9 References: Restrict existing hostlist to nodes with one of the given features (like srun -C) Signed-off-by: Egbert Eich <e...@suse.de> --- src/modules/slurm.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 73 insertions(+), 3 deletions(-) diff --git a/src/modules/slurm.c b/src/modules/slurm.c index a2dfada..5594c4f 100644 --- a/src/modules/slurm.c +++ b/src/modules/slurm.c @@ -74,10 +74,12 @@ static int mod_slurm_wcoll(opt_t *opt); static int mod_slurm_exit(void); static hostlist_t _slurm_wcoll(List jobids); static hostlist_t _slurm_wcoll_partition(List partitions); +static hostlist_t _slurm_wcoll_constraint(hostlist_t, List constraint); static int slurm_process_opt(opt_t *, int opt, char *arg); static List job_list = NULL; static List partition_list = NULL; +static List constraint_list = NULL; /* * Export generic pdsh module options @@ -103,6 +105,10 @@ struct pdsh_module_option slurm_module_options[] = "Run on nodes contained in SLURM partition", DSH | PCP, (optFunc) slurm_process_opt }, + { 'C', "feature,...", + "Limit to SLURM nodes with any of the specified features", + DSH | PCP, (optFunc) slurm_process_opt + }, PDSH_OPT_TABLE_END }; @@ -155,6 +161,9 @@ slurm_process_opt(opt_t *pdsh_opts, int opt, char *arg) case 'P': partition_list = list_split_append (partition_list, ",", arg); break; + case 'C': + constraint_list = list_split_append (constraint_list, ",", arg); + break; default: break; } @@ -171,6 +180,9 @@ mod_slurm_exit(void) if (partition_list) list_destroy (partition_list); + if (constraint_list) + list_destroy (constraint_list); + return (0); } @@ -188,7 +200,7 @@ static void _append_hostlist (hostlist_t *hl1, hostlist_t hl2) */ static int mod_slurm_wcoll(opt_t *opt) { - hostlist_t hlp = NULL, hlj = NULL; + hostlist_t hlp = NULL, hlj = NULL, hlc = NULL; /* * If a list of partitions is supplied gather the hosts @@ -215,6 +227,12 @@ static int mod_slurm_wcoll(opt_t *opt) hostlist_destroy (hlj); } + if (constraint_list) { + hlc = _slurm_wcoll_constraint(opt->wcoll, constraint_list); + hostlist_destroy(opt->wcoll); + opt->wcoll = hlc; + } + return 0; } @@ -258,6 +276,20 @@ static int _alljobids_requested (List l) return (list_delete_all (l, (ListFindF)_find_str, all)); } +static int _features_include (const char *f, const char *c) { + int l; + + l = strlen(c); + while (f) { + while (*f == ',') + f++; + if (strncmp(f, c, l) == 0 && (f[l] == '\0' || f[l] == ',')) + return (1); + f = strchr(f, ','); + } + return (0); +} + static hostlist_t _hl_append (hostlist_t hl, char *nodes) { if (hl == NULL) @@ -278,7 +310,7 @@ static hostlist_t _slurm_wcoll (List joblist) if ((joblist == NULL) && (envjobid = _slurm_jobid()) < 0) return (NULL); - if (slurm_load_jobs((time_t) NULL, &msg, 1) < 0) + if (slurm_load_jobs((time_t) NULL, &msg, SHOW_ALL) < 0) errx ("Unable to contact slurm controller: %s\n", slurm_strerror (errno)); @@ -327,7 +359,7 @@ static hostlist_t _slurm_wcoll_partition (List partitionlist) partition_info_t * p; ListIterator li; - if (slurm_load_partitions((time_t) NULL, &msg, 1) < 0) + if (slurm_load_partitions((time_t) NULL, &msg, SHOW_ALL) < 0) errx ("Unable to contact slurm controller: %s\n", slurm_strerror (errno)); @@ -360,6 +392,44 @@ static hostlist_t _slurm_wcoll_partition (List partitionlist) return (hl); } +static hostlist_t _slurm_wcoll_constraint (hostlist_t wl, List constraintlist) +{ + int i; + hostlist_t hl; + node_info_msg_t * msg; + node_info_t * n; + char *f; + char *c; + ListIterator li; + + if (slurm_load_node((time_t) NULL, &msg, SHOW_ALL) < 0) + errx ("Unable to contact slurm controller: %s\n", + slurm_strerror (errno)); + + li = list_iterator_create(constraintlist); + hl = hostlist_create(""); + for (i = 0; i < msg->record_count; i++){ + n = &msg->node_array[i]; + + if (hostlist_find(wl, n->name) < 0) + continue; + + f = n->features_act ? n->features_act : n->features; + if (!f) + continue; + + list_iterator_reset(li); + while ((c = list_next(li))){ + if (_features_include(f, c)) { + hostlist_push_host(hl, n->name); + continue; + } + } + } + + return (hl); +} + /* * vi: tabstop=4 shiftwidth=4 expandtab */ ++++++ slurm-add-documentation-for-C.patch ++++++ From: Dylan Simon <dy...@dylex.net> Date: Mon Nov 9 20:41:27 2020 -0500 Subject: slurm: add documentation for -C Patch-mainline: Not yet Git-repo: https://github.com/chaos/pdsh Git-commit: 9ed9f978598b8a18820d41d8af3779a7a52817a7 References: Signed-off-by: Egbert Eich <e...@suse.de> --- doc/pdsh.1.in | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/pdsh.1.in b/doc/pdsh.1.in index 1f0c1ff..ccd529d 100644 --- a/doc/pdsh.1.in +++ b/doc/pdsh.1.in @@ -303,6 +303,12 @@ argument "all" can be used to target all nodes running SLURM jobs, e.g. Target list of nodes containing in the SLURM partition \fIpartition\fR. This option may be used multiple times to target multiple SLURM partitions and/or partitions may be given in a comma-delimited list. +.TP +.I "-C feature[,feature,...]" +Restrict selected nodes to those SLURM nodes that have one of the specified +\fIfeature\fRs active. This option should be used in combination with other +node selectors, and filters the resulting list. Any nodes that do not have +one of the specified features (or is not a SLURM node) will be excluded. .SH "torque module options" The \fBtorque\fI module allows \fBpdsh\fR to target nodes based on
