Hello community,
here is the log from the commit of package gnu_parallel for openSUSE:Factory
checked in at 2020-08-03 14:16:47
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/gnu_parallel (Old)
and /work/SRC/openSUSE:Factory/.gnu_parallel.new.3592 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "gnu_parallel"
Mon Aug 3 14:16:47 2020 rev:57 rq:823963 version:20200722
Changes:
--------
--- /work/SRC/openSUSE:Factory/gnu_parallel/gnu_parallel.changes
2020-06-05 20:29:13.069349481 +0200
+++ /work/SRC/openSUSE:Factory/.gnu_parallel.new.3592/gnu_parallel.changes
2020-08-03 14:17:30.052626334 +0200
@@ -1,0 +2,6 @@
+Thu Jul 30 13:01:42 UTC 2020 - Jan Engelhardt <jeng...@inai.de>
+
+- Update to release 20200722
+ * Bug fixes and man page updates.
+
+-------------------------------------------------------------------
Old:
----
parallel-20200522.tar.bz2
parallel-20200522.tar.bz2.sig
New:
----
parallel-20200722.tar.bz2
parallel-20200722.tar.bz2.sig
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ gnu_parallel.spec ++++++
--- /var/tmp/diff_new_pack.HjqkaV/_old 2020-08-03 14:17:34.684630980 +0200
+++ /var/tmp/diff_new_pack.HjqkaV/_new 2020-08-03 14:17:34.688630984 +0200
@@ -17,7 +17,7 @@
Name: gnu_parallel
-Version: 20200522
+Version: 20200722
Release: 0
Summary: Shell tool for executing jobs in parallel
License: GPL-3.0-or-later
++++++ parallel-20200522.tar.bz2 -> parallel-20200722.tar.bz2 ++++++
++++ 1605 lines of diff (skipped)
++++ retrying with extended exclude list
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/Makefile.am new/parallel-20200722/Makefile.am
--- old/parallel-20200522/Makefile.am 2019-11-23 17:22:59.000000000 +0100
+++ new/parallel-20200722/Makefile.am 2020-05-24 14:06:04.000000000 +0200
@@ -123,4 +123,25 @@
mkdir -p urls
cd urls && grep -v '(dead)' ../src/* | grep -h -Po 'https?://[^
$$<>")}]+' | perl -pe
's/(>|\{).*//;s/\\-/-/g;s/\\n//g;s/&/&/g;s/&#.*//;'"s/'.*//" | grep -Ev
'parallel-(20)?$$|coolwebsite.biz' | sort -u | egrep -v 'example.com|##|\*\(' |
parallel -j0 --timeout 33 --bar --tag --joblog joblog --retries 3 neno wget -m
-l1 -Q1 '{=$$_=Q($$_)=}'
+reconf:
+ rm -fr autom4te.cache aclocal.m4 config.h config.h.in config.log
Makefile.in missing install-sh
+ rm -rf src/Makefile.in
+ autoreconf --install -W gnu
+ ./configure
+ make -j
+ sudo make install
+
+pack_unpack_and_test_build:
+ echo '### Building tar.bz2'
+ ./configure
+ make dist
+ make dist-bzip2
+ echo "### Unpack parallel-$(YYYYMMDD).tar.bz2"
+ cp parallel-$(YYYYMMDD).tar.bz2 /tmp
+ cd /tmp && \
+ tar xjf parallel-$(YYYYMMDD).tar.bz2 && \
+ cd parallel-$(YYYYMMDD) && \
+ ./configure && make -j && sudo make -j install
+
+
EXTRA_DIST = CITATION CREDITS cc-by-sa.txt fdl.txt
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/NEWS new/parallel-20200722/NEWS
--- old/parallel-20200522/NEWS 2020-05-23 19:19:47.000000000 +0200
+++ new/parallel-20200722/NEWS 2020-07-22 22:50:20.000000000 +0200
@@ -1,3 +1,64 @@
+20200722
+
+New in this release:
+
+* No new functionality
+
+* Bug fixes and man page updates.
+
+News about GNU Parallel:
+
+* Tips for Running Jobs on your VM
+
https://support.ehelp.edu.au/support/solutions/articles/6000089713-tips-for-running-jobs-on-your-vm#parallel
+
+* Introduction to GNU Parallel: Parallelizing Massive Individual Tasks
+ https://www.youtube.com/watch?v=QSFLTEyCKvo
+
+* Slides for same
+
http://www.hpc.lsu.edu/training/weekly-materials/2020-Spring/GNUParallel_Spring_2020.pdf
+
+* Linux-Fu: Parallel Universe
+ https://hackaday.com/2020/06/29/linux-fu-parallel-universe/
+
+* Resumable batch jobs with gnu parallel https://igor.io/parallel/
+
+* Introducing Parallel into Shell
+ https://www.petelawson.com/post/parallel-in-shell/
+
+* AV1 encoding parallelization with GNU Parallel
+
https://www.reddit.com/r/AV1/comments/fjvqp9/av1_encoding_parallelization_with_gnu_parallel/
+
+* Beschleunigte Befehlsbearbeitung mit GNU Parallel
+
https://www.linux-community.de/ausgaben/linuxuser/2013/06/beschleunigte-befehlsbearbeitung-mit-gnu-parallel/
+
+
+20200622
+
+New in this release:
+
+* No new functionality
+
+* Bug fixes and man page updates.
+
+News about GNU Parallel:
+
+* GNU Parallel is used at NERSC:
https://www.nersc.gov/assets/Uploads/08-Workflows-20200616.pdf
+
+* Resumable batch jobs with GNU Parallel https://igor.io/parallel/
+
+* Lab Meeting technical talk
https://www.slideshare.net/hoffmanlab/gnu-parallel-194030490
+
+* Open Sky Software Website Credits
https://www.openskysoftware.com/site-credits.htm
+
+* Speed up your Python Scripts using Ubuntu and GNU Parallel
https://ecce.esri.ca/wpecce/2019/12/24/speed-up-your-python-scripts-using-ubuntu-and-gnu-parallel/
+
+* Indispensable command-line tools
https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
+
+* GNU Parallel for simulations
http://pdebuyl.be/blog/2020/gnu-parallel-for-simulations.html
+
+* Introduction to GNU parallel
https://bioinformaticsworkbook.org/Appendix/GNUparallel/GNU_parallel_examples.html
+
+
20200522
New in this release:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/README new/parallel-20200722/README
--- old/parallel-20200522/README 2020-05-23 20:32:43.000000000 +0200
+++ new/parallel-20200722/README 2020-07-22 22:55:10.000000000 +0200
@@ -57,11 +57,11 @@
Full installation of GNU Parallel is as simple as:
- wget https://ftpmirror.gnu.org/parallel/parallel-20200522.tar.bz2
- wget https://ftpmirror.gnu.org/parallel/parallel-20200522.tar.bz2.sig
- gpg parallel-20200522.tar.bz2.sig
- bzip2 -dc parallel-20200522.tar.bz2 | tar xvf -
- cd parallel-20200522
+ wget https://ftpmirror.gnu.org/parallel/parallel-20200722.tar.bz2
+ wget https://ftpmirror.gnu.org/parallel/parallel-20200722.tar.bz2.sig
+ gpg parallel-20200722.tar.bz2.sig
+ bzip2 -dc parallel-20200722.tar.bz2 | tar xvf -
+ cd parallel-20200722
./configure && make && sudo make install
@@ -70,11 +70,11 @@
If you are not root you can add ~/bin to your path and install in
~/bin and ~/share:
- wget https://ftpmirror.gnu.org/parallel/parallel-20200522.tar.bz2
- wget https://ftpmirror.gnu.org/parallel/parallel-20200522.tar.bz2.sig
- gpg parallel-20200522.tar.bz2.sig
- bzip2 -dc parallel-20200522.tar.bz2 | tar xvf -
- cd parallel-20200522
+ wget https://ftpmirror.gnu.org/parallel/parallel-20200722.tar.bz2
+ wget https://ftpmirror.gnu.org/parallel/parallel-20200722.tar.bz2.sig
+ gpg parallel-20200722.tar.bz2.sig
+ bzip2 -dc parallel-20200722.tar.bz2 | tar xvf -
+ cd parallel-20200722
./configure --prefix=$HOME && make && make install
Or if your system lacks 'make' you can simply copy src/parallel
@@ -122,8 +122,8 @@
When using programs that use GNU Parallel to process data for
publication please cite:
- Tange, O. (2020, May 22). GNU Parallel 20200522 ('Kraftwerk').
- Zenodo. https://doi.org/10.5281/zenodo.3841377
+ Tange, O. (2020, July 22). GNU Parallel 20200722 ('Privacy Shield').
+ Zenodo. https://doi.org/10.5281/zenodo.3956817
= New versions =
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/configure.ac new/parallel-20200722/configure.ac
--- old/parallel-20200522/configure.ac 2020-05-23 20:32:43.000000000 +0200
+++ new/parallel-20200722/configure.ac 2020-07-22 22:55:10.000000000 +0200
@@ -1,4 +1,4 @@
-AC_INIT([parallel], [20200522], [bug-paral...@gnu.org])
+AC_INIT([parallel], [20200722], [bug-paral...@gnu.org])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/missing new/parallel-20200722/missing
--- old/parallel-20200522/missing 2020-05-23 20:33:16.000000000 +0200
+++ new/parallel-20200722/missing 2020-07-22 22:55:39.000000000 +0200
@@ -1,9 +1,9 @@
#! /bin/sh
# Common wrapper for a few potentially missing GNU programs.
-scriptversion=2013-10-28.13; # UTC
+scriptversion=2018-03-07.03; # UTC
-# Copyright (C) 1996-2014 Free Software Foundation, Inc.
+# Copyright (C) 1996-2018 Free Software Foundation, Inc.
# Originally written by Fran,cois Pinard <pin...@iro.umontreal.ca>, 1996.
# This program is free software; you can redistribute it and/or modify
@@ -17,7 +17,7 @@
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
# As a special exception to the GNU General Public License, if you
# distribute this file as part of a program that contains a
@@ -101,9 +101,9 @@
exit $st
fi
-perl_URL=http://www.perl.org/
-flex_URL=http://flex.sourceforge.net/
-gnu_software_URL=http://www.gnu.org/software
+perl_URL=https://www.perl.org/
+flex_URL=https://github.com/westes/flex
+gnu_software_URL=https://www.gnu.org/software
program_details ()
{
@@ -207,9 +207,9 @@
exit $st
# Local variables:
-# eval: (add-hook 'write-file-hooks 'time-stamp)
+# eval: (add-hook 'before-save-hook 'time-stamp)
# time-stamp-start: "scriptversion="
# time-stamp-format: "%:y-%02m-%02d.%02H"
-# time-stamp-time-zone: "UTC"
+# time-stamp-time-zone: "UTC0"
# time-stamp-end: "; # UTC"
# End:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/env_parallel.1
new/parallel-20200722/src/env_parallel.1
--- old/parallel-20200522/src/env_parallel.1 2020-02-22 20:04:55.000000000
+0100
+++ new/parallel-20200722/src/env_parallel.1 2020-07-04 11:32:19.000000000
+0200
@@ -1,4 +1,4 @@
-.\" Automatically generated by Pod::Man 4.09 (Pod::Simple 3.35)
+.\" Automatically generated by Pod::Man 4.11 (Pod::Simple 3.40)
.\"
.\" Standard preamble:
.\" ========================================================================
@@ -54,16 +54,20 @@
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
-.if !\nF .nr F 0
-.if \nF>0 \{\
-. de IX
-. tm Index:\\$1\t\\n%\t"\\$2"
+.nr rF 0
+.if \n(.g .if rF .nr rF 1
+.if (\n(rF:(\n(.g==0)) \{\
+. if \nF \{\
+. de IX
+. tm Index:\\$1\t\\n%\t"\\$2"
..
-. if !\nF==2 \{\
-. nr % 0
-. nr F 2
+. if !\nF==2 \{\
+. nr % 0
+. nr F 2
+. \}
. \}
.\}
+.rr rF
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear. Run. Save yourself. No user-serviceable parts.
@@ -129,7 +133,7 @@
.\" ========================================================================
.\"
.IX Title "ENV_PARALLEL 1"
-.TH ENV_PARALLEL 1 "2020-01-27" "20200122" "parallel"
+.TH ENV_PARALLEL 1 "2020-07-04" "20200622" "parallel"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
@@ -209,7 +213,7 @@
than the maximal length of a command and smaller than half of the max
if running remotely. E.g. The max size of Bash's command is 128 \s-1KB,\s0 so
\&\fBenv_parallel\fR will fail if '\fBset | wc \-c\fR' is bigger than 128
-\&\s-1KB.\s0 Technically the limit is in \fIexecve\fR\|(1) which IPC::open3
uses.
+\&\s-1KB.\s0 Technically the limit is in \fBexecve\fR\|(1) which IPC::open3
uses.
.PP
Bash completion functions are well-known for taking up well over 128
\&\s-1KB\s0 of environment space and the primary reason for causing
@@ -390,7 +394,8 @@
.IX Subsection "BUGS"
.PP
Due to a bug in Bash, aliases containing newlines must be followed by
-a newline in the command.
+a newline in the command. Some systems are not affected by this bug,
+but will print a warning anyway.
.SS "csh"
.IX Subsection "csh"
\&\fBenv_parallel\fR for \fBcsh\fR breaks \fB\f(CB$PARALLEL\fB\fR, so do not
use
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/env_parallel.bash
new/parallel-20200722/src/env_parallel.bash
--- old/parallel-20200522/src/env_parallel.bash 2020-01-27 18:17:31.000000000
+0100
+++ new/parallel-20200722/src/env_parallel.bash 2020-07-04 10:45:58.000000000
+0200
@@ -58,7 +58,7 @@
}
_ignore_HARDCODED() {
# These names cannot be detected
- echo
'(_|TIMEOUT|GROUPS|FUNCNAME|DIRSTACK|PIPESTATUS|USERNAME|BASH_[A-Z_]+)'
+ echo
'(_|TIMEOUT|GROUPS|FUNCNAME|DIRSTACK|PIPESTATUS|USERNAME|BASHPID|BASH_[A-Z_]+)'
}
_ignore_READONLY() {
readonly | perl -e '@r = map {
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/env_parallel.fish
new/parallel-20200722/src/env_parallel.fish
--- old/parallel-20200522/src/env_parallel.fish 2020-01-27 18:17:31.000000000
+0100
+++ new/parallel-20200722/src/env_parallel.fish 2020-06-21 16:02:55.000000000
+0200
@@ -157,7 +157,7 @@
($name,$val)=split(/ /,$_,2);
# Ignore read-only vars
$name=~/^(HOME|USER|COLUMNS|FISH_VERSION|LINES|PWD|SHLVL|_|
- history|status|version)$/x and next;
+ fish_pid|history|hostname|status|version)$/x and next;
# Single quote $val
if($val =~ /[^-_.+a-z0-9\/]/i) {
$val =~ s/\047/\047"\047"\047/g; # "-quote single quotes
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/env_parallel.html
new/parallel-20200722/src/env_parallel.html
--- old/parallel-20200522/src/env_parallel.html 2020-05-23 14:40:35.000000000
+0200
+++ new/parallel-20200722/src/env_parallel.html 2020-07-04 11:32:20.000000000
+0200
@@ -353,7 +353,7 @@
<h3 id="BUGS">BUGS</h3>
-<p>Due to a bug in Bash, aliases containing newlines must be followed by a
newline in the command.</p>
+<p>Due to a bug in Bash, aliases containing newlines must be followed by a
newline in the command. Some systems are not affected by this bug, but will
print a warning anyway.</p>
<h2 id="csh">csh</h2>
Binary files old/parallel-20200522/src/env_parallel.pdf and
new/parallel-20200722/src/env_parallel.pdf differ
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/env_parallel.pod
new/parallel-20200722/src/env_parallel.pod
--- old/parallel-20200522/src/env_parallel.pod 2020-01-27 18:17:31.000000000
+0100
+++ new/parallel-20200722/src/env_parallel.pod 2020-07-04 11:21:35.000000000
+0200
@@ -261,7 +261,8 @@
=head3 BUGS
Due to a bug in Bash, aliases containing newlines must be followed by
-a newline in the command.
+a newline in the command. Some systems are not affected by this bug,
+but will print a warning anyway.
=head2 csh
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/env_parallel.texi
new/parallel-20200722/src/env_parallel.texi
--- old/parallel-20200522/src/env_parallel.texi 2020-04-21 19:43:26.000000000
+0200
+++ new/parallel-20200722/src/env_parallel.texi 2020-07-04 11:21:47.000000000
+0200
@@ -352,7 +352,8 @@
@subsection BUGS
Due to a bug in Bash, aliases containing newlines must be followed by
-a newline in the command.
+a newline in the command. Some systems are not affected by this bug,
+but will print a warning anyway.
@node csh
@section csh
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/niceload new/parallel-20200722/src/niceload
--- old/parallel-20200522/src/niceload 2020-05-23 20:32:43.000000000 +0200
+++ new/parallel-20200722/src/niceload 2020-07-22 22:55:10.000000000 +0200
@@ -23,7 +23,7 @@
use strict;
use Getopt::Long;
$Global::progname="niceload";
-$Global::version = 20200522;
+$Global::version = 20200722;
Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage();
if($opt::version) {
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel new/parallel-20200722/src/parallel
--- old/parallel-20200522/src/parallel 2020-05-23 20:32:43.000000000 +0200
+++ new/parallel-20200722/src/parallel 2020-07-22 22:55:10.000000000 +0200
@@ -948,7 +948,7 @@
my $two_gb = 2**31-1;
my $blocksize = $Global::blocksize;
my $in = *STDIN;
- my $timeout = ::multiply_time_units($opt::blocktimeout);
+ my $timeout = $Global::blocktimeout;
my $header = find_header(\$buf,$in);
my $anything_written;
@@ -1762,6 +1762,13 @@
if(defined $opt::max_args) {
$Global::max_number_of_args = $opt::max_args;
}
+ if(defined $opt::blocktimeout) {
+ $Global::blocktimeout = int(multiply_time_units($opt::blocktimeout));
+ if($Global::blocktimeout < 1) {
+ ::error("--block-timeout must be at least 1");
+ wait_and_exit(255);
+ }
+ }
if(defined $opt::timeout) {
$Global::timeoutq = TimeoutQueue->new($opt::timeout);
}
@@ -2150,7 +2157,7 @@
sub init_globals() {
# Defaults:
- $Global::version = 20200522;
+ $Global::version = 20200722;
$Global::progname = 'parallel';
$::name = "GNU Parallel";
$Global::infinity = 2**31;
@@ -4535,6 +4542,7 @@
((defined $opt::linebuffer) ? "--linebuffer" : ""),
((defined $opt::max_chars) ? "--max-chars ".$opt::max_chars : ""),
((defined $opt::plain) ? "--plain" : ""),
+ ((defined $opt::plus) ? "--plus" : ""),
((defined $opt::retries) ? "--retries ".$opt::retries : ""),
((defined $opt::timeout) ? "--timeout ".$opt::timeout : ""),
((defined $opt::ungroup) ? "-u" : ""),
@@ -4852,6 +4860,9 @@
if($Global::unkilled_sqlworker) {
waitpid($Global::unkilled_sqlworker,0);
}
+ # Avoid: Warning: unable to close filehandle properly: No space
+ # left on device during global destruction.
+ $SIG{__WARN__} = sub {};
exit($error);
}
@@ -4896,8 +4907,8 @@
"If you use programs that use GNU Parallel to process data for an
article in a",
"scientific publication, please cite:",
"",
- " Tange, O. (2020, May 22). GNU Parallel 20200522 ('Kraftwerk').",
- " Zenodo. https://doi.org/10.5281/zenodo.3841377";,
+ " Tange, O. (2020, July 22). GNU Parallel 20200722 ('Privacy
Shield').",
+ " Zenodo. https://doi.org/10.5281/zenodo.3956817";,
"",
# Before changing this line, please read
#
https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice
@@ -4927,8 +4938,8 @@
"If you use programs that use GNU Parallel to process data for an
article in a",
"scientific publication, please cite:",
"",
- " Tange, O. (2020, May 22). GNU Parallel 20200522 ('Kraftwerk').",
- " Zenodo. https://doi.org/10.5281/zenodo.3841377";,
+ " Tange, O. (2020, July 22). GNU Parallel 20200722 ('Privacy
Shield').",
+ " Zenodo. https://doi.org/10.5281/zenodo.3956817";,
"",
# Before changing this line, please read
#
https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and
@@ -5015,11 +5026,11 @@
"GNU $Global::progname $Global::version",
"Copyright (C) 2007-2020 Ole Tange, http://ole.tange.dk and Free
Software",
"Foundation, Inc.",
- "License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>",
+ "License GPLv3+: GNU GPL version 3 or later
<https://gnu.org/licenses/gpl.html>",
"This is free software: you are free to change and redistribute it.",
"GNU $Global::progname comes with no warranty.",
"",
- "Web site: http://www.gnu.org/software/${Global::progname}\n";,
+ "Web site: https://www.gnu.org/software/${Global::progname}\n";,
"When using programs that use GNU Parallel to process data for
publication",
"please cite as described in 'parallel --citation'.\n",
);
@@ -5041,20 +5052,20 @@
"If you use programs that use GNU Parallel to process data for an
article in a",
"scientific publication, please cite:",
"",
- "\@software{tange_2020_3841377,",
+ "\@software{tange_2020_3956817,",
" author = {Tange, Ole},",
- " title = {GNU Parallel 20200522 ('Kraftwerk')},",
- " month = May,",
+ " title = {GNU Parallel 20200722 ('Privacy Shield')},",
+ " month = Jul,",
" year = 2020,",
" note = {{GNU Parallel is a general parallelizer to run",
" multiple serial command line programs in
parallel",
" without changing them.}},",
" publisher = {Zenodo},",
- " doi = {10.5281/zenodo.3841377},",
- " url = {https://doi.org/10.5281/zenodo.3841377}";,
+ " doi = {10.5281/zenodo.3956817},",
+ " url = {https://doi.org/10.5281/zenodo.3956817}";,
"}",
"",
- "(Feel free to use \\nocite{tange_2020_3841377})",
+ "(Feel free to use \\nocite{tange_2020_3956817})",
"",
# Before changing this line, please read
#
https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and
@@ -5160,7 +5171,7 @@
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
-# along with this program; if not, see <http://www.gnu.org/licenses/>
+# along with this program; if not, see <https://www.gnu.org/licenses/>
# or write to the Free Software Foundation, Inc., 51 Franklin St,
# Fifth Floor, Boston, MA 02110-1301 USA
";
@@ -8582,12 +8593,13 @@
unlink;
rmdir;
}
- if($bash=~s/h//) {
+ if($bash=~s/(\d+)h/$1/) {
exit $bash;
}
exit $csh;
}).
- "' ".'"$?h" "$status" ';
+ # `echo \$?h` is needed to make fish not complain
+ "' ".'"`echo \\\\\\\\\$?h`" "$status" ';
}
return $script
}
@@ -12291,8 +12303,13 @@
my $dbh;
if($driver eq "CSV") {
# CSV does not use normal dsn
- $dbh = DBI->connect("dbi:CSV:", "", "", { f_dir => "$database", })
- or die $DBI::errstr;
+ if(-d $database) {
+ $dbh = DBI->connect("dbi:CSV:", "", "", { f_dir => "$database", })
+ or die $DBI::errstr;
+ } else {
+ ::error("$database is not a directory.");
+ ::wait_and_exit(255);
+ }
} else {
$dbh = DBI->connect($dsn, $userid, $password,
{ RaiseError => 1, AutoInactiveDestroy => 1 })
@@ -12303,7 +12320,6 @@
$dbh->{'RaiseError'} = 1;
$dbh->{'ShowErrorStatement'} = 1;
$dbh->{'HandleError'} = sub {};
-
if(not defined $options{'table'}) {
::error("The DBURL ($dburl) must contain a table.");
::wait_and_exit(255);
@@ -12603,7 +12619,7 @@
my %vartype = (
"Oracle" => { "BIGINT" => "NUMBER(19,0)",
"TEXT" => "CLOB", },
- "mysql" => { "TEXT" => "LONGTEXT", },
+ "mysql" => { "TEXT" => "BLOB", },
"CSV" => { "BIGINT" => "INT",
"FLOAT" => "REAL", },
);
@@ -12642,26 +12658,58 @@
0, @$record_ref[1..$#$record_ref]);
}
+
sub get_record($) {
my $self = shift;
my @retval;
my $table = $self->table();
my @v_cols = map { ", V$_" } (1..$self->max_number_of_args());
- my $v = $self->get("SELECT Seq, Command @v_cols FROM $table ".
- "WHERE Exitval = -1000 ORDER BY Seq LIMIT 1;");
- if($v->[0]) {
- my $val_ref = $v->[0];
- # Mark record as taken
- my $seq = shift @$val_ref;
- # Save the sequence number to use when running the job
- $SQL::next_seq = $seq;
- $self->update("SET Exitval = ? WHERE Seq = ".$seq, -1220);
- my @command = split /\257 /, shift @$val_ref;
- $SQL::command_ref = \@command;
- for (@$val_ref) {
- push @retval, Arg->new($_);
+ my $rand = "Reserved-".$$.rand();
+ my $v;
+ my $more_pending;
+
+ do {
+ if($self->{'driver'} eq "CSV") {
+ # Sub SELECT is not supported in CSV
+ # So to minimize the race condition below select a job at random
+ my $r = $self->get("SELECT Seq, Command @v_cols FROM $table ".
+ "WHERE Exitval = -1000 LIMIT 100;");
+ $v = [ sort { rand() > 0.5 } @$r ];
+ } else {
+ # Avoid race condition where multiple workers get the same job
+ # by setting Stdout to a unique string
+ # (SELECT * FROM (...) AS dummy) is needed due to sillyness in MySQL
+ $self->update("SET Stdout = ?,Exitval = ? ".
+ "WHERE Seq = (".
+ " SELECT * FROM (".
+ " SELECT min(Seq) FROM $table WHERE Exitval =
-1000".
+ " ) AS dummy".
+ ") AND Exitval = -1000;", $rand, -1210);
+ # If a parallel worker overwrote the unique string this will get
nothing
+ $v = $self->get("SELECT Seq, Command @v_cols FROM $table ".
+ "WHERE Stdout = ?;", $rand);
+ }
+ if($v->[0]) {
+ my $val_ref = $v->[0];
+ # Mark record as taken
+ my $seq = shift @$val_ref;
+ # Save the sequence number to use when running the job
+ $SQL::next_seq = $seq;
+ $self->update("SET Exitval = ? WHERE Seq = ".$seq, -1220);
+ # Command is encoded with '\257 space' as splitting char
+ my @command = split /\257 /, shift @$val_ref;
+ $SQL::command_ref = \@command;
+ for (@$val_ref) {
+ push @retval, Arg->new($_);
+ }
+ } else {
+ # If the record was updated by another job in parallel,
+ # then we may not be done, so see if there are more jobs pending
+ $more_pending =
+ $self->get("SELECT Seq FROM $table WHERE Exitval = ?;", -1210);
}
- }
+ } while (not $v->[0] and $more_pending->[0]);
+
if(@retval) {
return \@retval;
} else {
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel.1 new/parallel-20200722/src/parallel.1
--- old/parallel-20200522/src/parallel.1 2020-05-23 14:40:34.000000000
+0200
+++ new/parallel-20200722/src/parallel.1 2020-06-07 21:58:05.000000000
+0200
@@ -129,7 +129,7 @@
.\" ========================================================================
.\"
.IX Title "PARALLEL 1"
-.TH PARALLEL 1 "2020-05-23" "20200422" "parallel"
+.TH PARALLEL 1 "2020-06-06" "20200522" "parallel"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
@@ -2409,10 +2409,13 @@
\& sql:oracle://scott:ti...@ora.example.com/xe/parjob
\& postgresql://scott:ti...@pg.example.com/pgdb/parjob
\& pg:///parjob
-\& sqlite3:///pardb/parjob.sqlite
-\& csv:///%2Ftmp%2Fmydir/jobtable
+\& sqlite3:///%2Ftmp%2Fpardb.sqlite/parjob
+\& csv:///%2Ftmp%2Fpardb/parjob
.Ve
.Sp
+Notice how / in the path of sqlite and \s-1CVS\s0 must be encoded as
+\&\f(CW%2F\fR. Except the last / in \s-1CSV\s0 which must be a /.
+.Sp
It can also be an alias from ~/.sql/aliases:
.Sp
.Vb 1
@@ -3182,6 +3185,17 @@
.PP
If \fBmy_program\fR fails a red \s-1FAIL\s0 will be printed followed by the
failing
command; otherwise a green \s-1OK\s0 will be printed followed by the command.
+.SH "EXAMPLE: Continously show the latest line of output"
+.IX Header "EXAMPLE: Continously show the latest line of output"
+It can be useful to monitor the output of running jobs.
+.PP
+This shows the most recent output line until a job finishes. After
+which the output of the job is printed in full:
+.PP
+.Vb 2
+\& parallel \*(Aq{} | tee >(cat >&3)\*(Aq ::: \*(Aqcommand 1\*(Aq
\*(Aqcommand 2\*(Aq \e
+\& 3> >(perl \-ne \*(Aq$|=1;chomp;printf"%.\*(Aq$COLUMNS\*(Aqs\er",$_."
"x100\*(Aq)
+.Ve
.SH "EXAMPLE: Log rotate"
.IX Header "EXAMPLE: Log rotate"
Log rotation renames a logfile to an extension with a higher number:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel.html new/parallel-20200722/src/parallel.html
--- old/parallel-20200522/src/parallel.html 2020-05-23 14:40:34.000000000
+0200
+++ new/parallel-20200722/src/parallel.html 2020-06-07 21:58:05.000000000
+0200
@@ -39,6 +39,7 @@
<li><a
href="#EXAMPLE:-Composed-command-with-multiple-input-sources">EXAMPLE: Composed
command with multiple input sources</a></li>
<li><a href="#EXAMPLE:-Calling-Bash-functions">EXAMPLE: Calling Bash
functions</a></li>
<li><a href="#EXAMPLE:-Function-tester">EXAMPLE: Function tester</a></li>
+ <li><a href="#EXAMPLE:-Continously-show-the-latest-line-of-output">EXAMPLE:
Continously show the latest line of output</a></li>
<li><a href="#EXAMPLE:-Log-rotate">EXAMPLE: Log rotate</a></li>
<li><a
href="#EXAMPLE:-Removing-file-extension-when-processing-files">EXAMPLE:
Removing file extension when processing files</a></li>
<li><a href="#EXAMPLE:-Removing-strings-from-the-argument">EXAMPLE: Removing
strings from the argument</a></li>
@@ -2281,8 +2282,10 @@
sql:oracle://scott:ti...@ora.example.com/xe/parjob
postgresql://scott:ti...@pg.example.com/pgdb/parjob
pg:///parjob
- sqlite3:///pardb/parjob.sqlite
- csv:///%2Ftmp%2Fmydir/jobtable</code></pre>
+ sqlite3:///%2Ftmp%2Fpardb.sqlite/parjob
+ csv:///%2Ftmp%2Fpardb/parjob</code></pre>
+
+<p>Notice how / in the path of sqlite and CVS must be encoded as %2F. Except
the last / in CSV which must be a /.</p>
<p>It can also be an alias from ~/.sql/aliases:</p>
@@ -2969,6 +2972,15 @@
<p>If <b>my_program</b> fails a red FAIL will be printed followed by the
failing command; otherwise a green OK will be printed followed by the
command.</p>
+<h1 id="EXAMPLE:-Continously-show-the-latest-line-of-output">EXAMPLE:
Continously show the latest line of output</h1>
+
+<p>It can be useful to monitor the output of running jobs.</p>
+
+<p>This shows the most recent output line until a job finishes. After which
the output of the job is printed in full:</p>
+
+<pre><code> parallel '{} | tee >(cat >&3)' ::: 'command
1' 'command 2' \
+ 3> >(perl -ne
'$|=1;chomp;printf"%.'$COLUMNS's\r",$_."
"x100')</code></pre>
+
<h1 id="EXAMPLE:-Log-rotate">EXAMPLE: Log rotate</h1>
<p>Log rotation renames a logfile to an extension with a higher number: log.1
becomes log.2, log.2 becomes log.3, and so on. The oldest log is removed. To
avoid overwriting files the process starts backwards from the high number to
the low number. This will keep 10 old versions of the log:</p>
Binary files old/parallel-20200522/src/parallel.pdf and
new/parallel-20200722/src/parallel.pdf differ
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel.pod new/parallel-20200722/src/parallel.pod
--- old/parallel-20200522/src/parallel.pod 2020-05-23 14:40:16.000000000
+0200
+++ new/parallel-20200722/src/parallel.pod 2020-06-06 22:28:29.000000000
+0200
@@ -2380,8 +2380,11 @@
sql:oracle://scott:ti...@ora.example.com/xe/parjob
postgresql://scott:ti...@pg.example.com/pgdb/parjob
pg:///parjob
- sqlite3:///pardb/parjob.sqlite
- csv:///%2Ftmp%2Fmydir/jobtable
+ sqlite3:///%2Ftmp%2Fpardb.sqlite/parjob
+ csv:///%2Ftmp%2Fpardb/parjob
+
+Notice how / in the path of sqlite and CVS must be encoded as
+%2F. Except the last / in CSV which must be a /.
It can also be an alias from ~/.sql/aliases:
@@ -2910,7 +2913,6 @@
=back
-
=head1 EXAMPLE: Working as xargs -n1. Argument appending
GNU B<parallel> can work similar to B<xargs -n1>.
@@ -3144,6 +3146,17 @@
command; otherwise a green OK will be printed followed by the command.
+=head1 EXAMPLE: Continously show the latest line of output
+
+It can be useful to monitor the output of running jobs.
+
+This shows the most recent output line until a job finishes. After
+which the output of the job is printed in full:
+
+ parallel '{} | tee >(cat >&3)' ::: 'command 1' 'command 2' \
+ 3> >(perl -ne '$|=1;chomp;printf"%.'$COLUMNS's\r",$_." "x100')
+
+
=head1 EXAMPLE: Log rotate
Log rotation renames a logfile to an extension with a higher number:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel.texi new/parallel-20200722/src/parallel.texi
--- old/parallel-20200522/src/parallel.texi 2020-05-23 14:40:39.000000000
+0200
+++ new/parallel-20200722/src/parallel.texi 2020-06-07 21:58:07.000000000
+0200
@@ -25,6 +25,7 @@
* EXAMPLE@asis{:} Composed command with multiple input sources::
* EXAMPLE@asis{:} Calling Bash functions::
* EXAMPLE@asis{:} Function tester::
+* EXAMPLE@asis{:} Continously show the latest line of output::
* EXAMPLE@asis{:} Log rotate::
* EXAMPLE@asis{:} Removing file extension when processing files::
* EXAMPLE@asis{:} Removing strings from the argument::
@@ -2674,10 +2675,13 @@
sql:oracle://scott:ti...@ora.example.com/xe/parjob
postgresql://scott:ti...@pg.example.com/pgdb/parjob
pg:///parjob
- sqlite3:///pardb/parjob.sqlite
- csv:///%2Ftmp%2Fmydir/jobtable
+ sqlite3:///%2Ftmp%2Fpardb.sqlite/parjob
+ csv:///%2Ftmp%2Fpardb/parjob
@end verbatim
+Notice how / in the path of sqlite and CVS must be encoded as
+%2F. Except the last / in CSV which must be a /.
+
It can also be an alias from ~/.sql/aliases:
@verbatim
@@ -3542,6 +3546,19 @@
If @strong{my_program} fails a red FAIL will be printed followed by the failing
command; otherwise a green OK will be printed followed by the command.
+@node EXAMPLE: Continously show the latest line of output
+@chapter EXAMPLE: Continously show the latest line of output
+
+It can be useful to monitor the output of running jobs.
+
+This shows the most recent output line until a job finishes. After
+which the output of the job is printed in full:
+
+@verbatim
+ parallel '{} | tee >(cat >&3)' ::: 'command 1' 'command 2' \
+ 3> >(perl -ne '$|=1;chomp;printf"%.'$COLUMNS's\r",$_." "x100')
+@end verbatim
+
@node EXAMPLE: Log rotate
@chapter EXAMPLE: Log rotate
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel_alternatives.7
new/parallel-20200722/src/parallel_alternatives.7
--- old/parallel-20200522/src/parallel_alternatives.7 2020-05-23
18:03:47.000000000 +0200
+++ new/parallel-20200722/src/parallel_alternatives.7 2020-05-24
13:39:34.000000000 +0200
@@ -2935,6 +2935,8 @@
https://pypi.org/project/papply/ (Last checked: 2020\-04)
.SS "Todo"
.IX Subsection "Todo"
+https://gitlab.com/netikras/bthread
+.PP
https://github.com/JeiKeiLim/simple_distribute_job
.PP
https://github.com/reggi/pkgrun
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel_alternatives.html
new/parallel-20200722/src/parallel_alternatives.html
--- old/parallel-20200522/src/parallel_alternatives.html 2020-05-23
18:03:47.000000000 +0200
+++ new/parallel-20200722/src/parallel_alternatives.html 2020-07-04
11:32:24.000000000 +0200
@@ -2432,6 +2432,8 @@
<h2 id="Todo">Todo</h2>
+<p>https://gitlab.com/netikras/bthread</p>
+
<p>https://github.com/JeiKeiLim/simple_distribute_job</p>
<p>https://github.com/reggi/pkgrun</p>
Binary files old/parallel-20200522/src/parallel_alternatives.pdf and
new/parallel-20200722/src/parallel_alternatives.pdf differ
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel_alternatives.pod
new/parallel-20200722/src/parallel_alternatives.pod
--- old/parallel-20200522/src/parallel_alternatives.pod 2020-05-23
18:02:54.000000000 +0200
+++ new/parallel-20200722/src/parallel_alternatives.pod 2020-05-24
01:22:51.000000000 +0200
@@ -2616,6 +2616,8 @@
=head2 Todo
+https://gitlab.com/netikras/bthread
+
https://github.com/JeiKeiLim/simple_distribute_job
https://github.com/reggi/pkgrun
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/parallel_alternatives.texi
new/parallel-20200722/src/parallel_alternatives.texi
--- old/parallel-20200522/src/parallel_alternatives.texi 2020-05-23
18:03:52.000000000 +0200
+++ new/parallel-20200722/src/parallel_alternatives.texi 2020-05-24
13:39:35.000000000 +0200
@@ -3106,6 +3106,8 @@
@node Todo
@section Todo
+https://gitlab.com/netikras/bthread
+
https://github.com/JeiKeiLim/simple_distribute_job
https://github.com/reggi/pkgrun
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/sem new/parallel-20200722/src/sem
--- old/parallel-20200522/src/sem 2020-05-23 20:32:43.000000000 +0200
+++ new/parallel-20200722/src/sem 2020-07-22 22:55:10.000000000 +0200
@@ -948,7 +948,7 @@
my $two_gb = 2**31-1;
my $blocksize = $Global::blocksize;
my $in = *STDIN;
- my $timeout = ::multiply_time_units($opt::blocktimeout);
+ my $timeout = $Global::blocktimeout;
my $header = find_header(\$buf,$in);
my $anything_written;
@@ -1762,6 +1762,13 @@
if(defined $opt::max_args) {
$Global::max_number_of_args = $opt::max_args;
}
+ if(defined $opt::blocktimeout) {
+ $Global::blocktimeout = int(multiply_time_units($opt::blocktimeout));
+ if($Global::blocktimeout < 1) {
+ ::error("--block-timeout must be at least 1");
+ wait_and_exit(255);
+ }
+ }
if(defined $opt::timeout) {
$Global::timeoutq = TimeoutQueue->new($opt::timeout);
}
@@ -2150,7 +2157,7 @@
sub init_globals() {
# Defaults:
- $Global::version = 20200522;
+ $Global::version = 20200722;
$Global::progname = 'parallel';
$::name = "GNU Parallel";
$Global::infinity = 2**31;
@@ -4535,6 +4542,7 @@
((defined $opt::linebuffer) ? "--linebuffer" : ""),
((defined $opt::max_chars) ? "--max-chars ".$opt::max_chars : ""),
((defined $opt::plain) ? "--plain" : ""),
+ ((defined $opt::plus) ? "--plus" : ""),
((defined $opt::retries) ? "--retries ".$opt::retries : ""),
((defined $opt::timeout) ? "--timeout ".$opt::timeout : ""),
((defined $opt::ungroup) ? "-u" : ""),
@@ -4852,6 +4860,9 @@
if($Global::unkilled_sqlworker) {
waitpid($Global::unkilled_sqlworker,0);
}
+ # Avoid: Warning: unable to close filehandle properly: No space
+ # left on device during global destruction.
+ $SIG{__WARN__} = sub {};
exit($error);
}
@@ -4896,8 +4907,8 @@
"If you use programs that use GNU Parallel to process data for an
article in a",
"scientific publication, please cite:",
"",
- " Tange, O. (2020, May 22). GNU Parallel 20200522 ('Kraftwerk').",
- " Zenodo. https://doi.org/10.5281/zenodo.3841377";,
+ " Tange, O. (2020, July 22). GNU Parallel 20200722 ('Privacy
Shield').",
+ " Zenodo. https://doi.org/10.5281/zenodo.3956817";,
"",
# Before changing this line, please read
#
https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice
@@ -4927,8 +4938,8 @@
"If you use programs that use GNU Parallel to process data for an
article in a",
"scientific publication, please cite:",
"",
- " Tange, O. (2020, May 22). GNU Parallel 20200522 ('Kraftwerk').",
- " Zenodo. https://doi.org/10.5281/zenodo.3841377";,
+ " Tange, O. (2020, July 22). GNU Parallel 20200722 ('Privacy
Shield').",
+ " Zenodo. https://doi.org/10.5281/zenodo.3956817";,
"",
# Before changing this line, please read
#
https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and
@@ -5015,11 +5026,11 @@
"GNU $Global::progname $Global::version",
"Copyright (C) 2007-2020 Ole Tange, http://ole.tange.dk and Free
Software",
"Foundation, Inc.",
- "License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>",
+ "License GPLv3+: GNU GPL version 3 or later
<https://gnu.org/licenses/gpl.html>",
"This is free software: you are free to change and redistribute it.",
"GNU $Global::progname comes with no warranty.",
"",
- "Web site: http://www.gnu.org/software/${Global::progname}\n";,
+ "Web site: https://www.gnu.org/software/${Global::progname}\n";,
"When using programs that use GNU Parallel to process data for
publication",
"please cite as described in 'parallel --citation'.\n",
);
@@ -5041,20 +5052,20 @@
"If you use programs that use GNU Parallel to process data for an
article in a",
"scientific publication, please cite:",
"",
- "\@software{tange_2020_3841377,",
+ "\@software{tange_2020_3956817,",
" author = {Tange, Ole},",
- " title = {GNU Parallel 20200522 ('Kraftwerk')},",
- " month = May,",
+ " title = {GNU Parallel 20200722 ('Privacy Shield')},",
+ " month = Jul,",
" year = 2020,",
" note = {{GNU Parallel is a general parallelizer to run",
" multiple serial command line programs in
parallel",
" without changing them.}},",
" publisher = {Zenodo},",
- " doi = {10.5281/zenodo.3841377},",
- " url = {https://doi.org/10.5281/zenodo.3841377}";,
+ " doi = {10.5281/zenodo.3956817},",
+ " url = {https://doi.org/10.5281/zenodo.3956817}";,
"}",
"",
- "(Feel free to use \\nocite{tange_2020_3841377})",
+ "(Feel free to use \\nocite{tange_2020_3956817})",
"",
# Before changing this line, please read
#
https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and
@@ -5160,7 +5171,7 @@
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
-# along with this program; if not, see <http://www.gnu.org/licenses/>
+# along with this program; if not, see <https://www.gnu.org/licenses/>
# or write to the Free Software Foundation, Inc., 51 Franklin St,
# Fifth Floor, Boston, MA 02110-1301 USA
";
@@ -8582,12 +8593,13 @@
unlink;
rmdir;
}
- if($bash=~s/h//) {
+ if($bash=~s/(\d+)h/$1/) {
exit $bash;
}
exit $csh;
}).
- "' ".'"$?h" "$status" ';
+ # `echo \$?h` is needed to make fish not complain
+ "' ".'"`echo \\\\\\\\\$?h`" "$status" ';
}
return $script
}
@@ -12291,8 +12303,13 @@
my $dbh;
if($driver eq "CSV") {
# CSV does not use normal dsn
- $dbh = DBI->connect("dbi:CSV:", "", "", { f_dir => "$database", })
- or die $DBI::errstr;
+ if(-d $database) {
+ $dbh = DBI->connect("dbi:CSV:", "", "", { f_dir => "$database", })
+ or die $DBI::errstr;
+ } else {
+ ::error("$database is not a directory.");
+ ::wait_and_exit(255);
+ }
} else {
$dbh = DBI->connect($dsn, $userid, $password,
{ RaiseError => 1, AutoInactiveDestroy => 1 })
@@ -12303,7 +12320,6 @@
$dbh->{'RaiseError'} = 1;
$dbh->{'ShowErrorStatement'} = 1;
$dbh->{'HandleError'} = sub {};
-
if(not defined $options{'table'}) {
::error("The DBURL ($dburl) must contain a table.");
::wait_and_exit(255);
@@ -12603,7 +12619,7 @@
my %vartype = (
"Oracle" => { "BIGINT" => "NUMBER(19,0)",
"TEXT" => "CLOB", },
- "mysql" => { "TEXT" => "LONGTEXT", },
+ "mysql" => { "TEXT" => "BLOB", },
"CSV" => { "BIGINT" => "INT",
"FLOAT" => "REAL", },
);
@@ -12642,26 +12658,58 @@
0, @$record_ref[1..$#$record_ref]);
}
+
sub get_record($) {
my $self = shift;
my @retval;
my $table = $self->table();
my @v_cols = map { ", V$_" } (1..$self->max_number_of_args());
- my $v = $self->get("SELECT Seq, Command @v_cols FROM $table ".
- "WHERE Exitval = -1000 ORDER BY Seq LIMIT 1;");
- if($v->[0]) {
- my $val_ref = $v->[0];
- # Mark record as taken
- my $seq = shift @$val_ref;
- # Save the sequence number to use when running the job
- $SQL::next_seq = $seq;
- $self->update("SET Exitval = ? WHERE Seq = ".$seq, -1220);
- my @command = split /\257 /, shift @$val_ref;
- $SQL::command_ref = \@command;
- for (@$val_ref) {
- push @retval, Arg->new($_);
+ my $rand = "Reserved-".$$.rand();
+ my $v;
+ my $more_pending;
+
+ do {
+ if($self->{'driver'} eq "CSV") {
+ # Sub SELECT is not supported in CSV
+ # So to minimize the race condition below select a job at random
+ my $r = $self->get("SELECT Seq, Command @v_cols FROM $table ".
+ "WHERE Exitval = -1000 LIMIT 100;");
+ $v = [ sort { rand() > 0.5 } @$r ];
+ } else {
+ # Avoid race condition where multiple workers get the same job
+ # by setting Stdout to a unique string
+ # (SELECT * FROM (...) AS dummy) is needed due to sillyness in MySQL
+ $self->update("SET Stdout = ?,Exitval = ? ".
+ "WHERE Seq = (".
+ " SELECT * FROM (".
+ " SELECT min(Seq) FROM $table WHERE Exitval =
-1000".
+ " ) AS dummy".
+ ") AND Exitval = -1000;", $rand, -1210);
+ # If a parallel worker overwrote the unique string this will get
nothing
+ $v = $self->get("SELECT Seq, Command @v_cols FROM $table ".
+ "WHERE Stdout = ?;", $rand);
+ }
+ if($v->[0]) {
+ my $val_ref = $v->[0];
+ # Mark record as taken
+ my $seq = shift @$val_ref;
+ # Save the sequence number to use when running the job
+ $SQL::next_seq = $seq;
+ $self->update("SET Exitval = ? WHERE Seq = ".$seq, -1220);
+ # Command is encoded with '\257 space' as splitting char
+ my @command = split /\257 /, shift @$val_ref;
+ $SQL::command_ref = \@command;
+ for (@$val_ref) {
+ push @retval, Arg->new($_);
+ }
+ } else {
+ # If the record was updated by another job in parallel,
+ # then we may not be done, so see if there are more jobs pending
+ $more_pending =
+ $self->get("SELECT Seq FROM $table WHERE Exitval = ?;", -1210);
}
- }
+ } while (not $v->[0] and $more_pending->[0]);
+
if(@retval) {
return \@retval;
} else {
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/sql new/parallel-20200722/src/sql
--- old/parallel-20200522/src/sql 2020-05-23 20:32:43.000000000 +0200
+++ new/parallel-20200722/src/sql 2020-07-22 22:55:10.000000000 +0200
@@ -574,7 +574,7 @@
exit ($err);
sub parse_options {
- $Global::version = 20200522;
+ $Global::version = 20200722;
$Global::progname = 'sql';
# This must be done first as this may exec myself
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude
config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4
--exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh
old/parallel-20200522/src/sql.1 new/parallel-20200722/src/sql.1
--- old/parallel-20200522/src/sql.1 2020-05-23 20:33:18.000000000 +0200
+++ new/parallel-20200722/src/sql.1 2020-07-22 22:55:40.000000000 +0200
@@ -1,4 +1,4 @@
-.\" Automatically generated by Pod::Man 4.09 (Pod::Simple 3.35)
+.\" Automatically generated by Pod::Man 4.11 (Pod::Simple 3.40)
.\"
.\" Standard preamble:
.\" ========================================================================
@@ -54,16 +54,20 @@
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
-.if !\nF .nr F 0
-.if \nF>0 \{\
-. de IX
-. tm Index:\\$1\t\\n%\t"\\$2"
+.nr rF 0
+.if \n(.g .if rF .nr rF 1
+.if (\n(rF:(\n(.g==0)) \{\
+. if \nF \{\
+. de IX
+. tm Index:\\$1\t\\n%\t"\\$2"
..
-. if !\nF==2 \{\
-. nr % 0
-. nr F 2
+. if !\nF==2 \{\
+. nr % 0
+. nr F 2
+. \}
. \}
.\}
+.rr rF
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear. Run. Save yourself. No user-serviceable parts.
@@ -129,7 +133,7 @@
.\" ========================================================================
.\"
.IX Title "SQL 1"
-.TH SQL 1 "2020-05-23" "20200522" "parallel"
+.TH SQL 1 "2020-07-22" "20200722" "parallel"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
Binary files old/parallel-20200522/src/sql.pdf and
new/parallel-20200722/src/sql.pdf differ
++++++ parallel-20200522.tar.bz2.sig -> parallel-20200722.tar.bz2.sig ++++++
--- /work/SRC/openSUSE:Factory/gnu_parallel/parallel-20200522.tar.bz2.sig
2020-06-05 20:29:13.237349977 +0200
+++
/work/SRC/openSUSE:Factory/.gnu_parallel.new.3592/parallel-20200722.tar.bz2.sig
2020-08-03 14:17:34.144630438 +0200
@@ -2,7 +2,7 @@
# To check the signature run:
# echo | gpg
-# gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve
parallel-20200522.tar.bz2.sig
+# gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve
parallel-20200722.tar.bz2.sig
echo | gpg 2>/dev/null
gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve $0
@@ -10,32 +10,32 @@
-----BEGIN PGP SIGNATURE-----
-iQUHBAABCgAdFiEEzaAaQgjE90UGEH570atFFoiIiIgFAl7JbI0ACgkQ0atFFoiI
-iIgx/yagx60zqVyc6TKNteaZ+vU3aIL5yB6kpxwzxy0QrA2RBV9/v2f1fjroorda
-w33MIFEXkqKhpVM07jTlsZy05ZK/llI1lUKmyh1jB9qVncZNl5vwZuemZfDigscO
-xFwc+5NdI1MQpE64xYoc0g1eFH0ZJzZ3GYjLTRJ1Sn6L0WNUUXXMl+qoy67MRFAS
-NFPZ1uCEhdW3T6d5bJdb5c5tK+u6yf0OFKTfAwcFf8BdTBfC2lIx69pWPsedv+XS
-naojvnqC3PkTn3Gt/O3/eL36i6yBd+AytVpJBxFDdBWa1t9SzcrokfXvJFj1YcLu
-o+U271in5wCX4jblEpkVRRLzVs97UfIMM2QYVpQljTC0iXyDxgb/6eqo9IDP6NRp
-AIGStCyy26Ttps73PbtrtKBeSZww1HbrAfZxn/o6zzuRLNwHWApC40JUrYxvhBv1
-JNLa2kSvcLeQ5Yey4RO4/qdeV/bhiLOJZB0/0gVb6Q7vjW7PzYKeLEVwfzJ4kljY
-jG8kTSHeKXfCq63ouuZEKE5oeE3GDg/pmq0G32PUNIHezcPF3gk0A5lyy6UOHHRV
-V5Y+bFir0BCC+BFHobFaIfSIj6Ov8YK5fsC/C4YsZsv6MKzsPhOKNyTWlT6z/CQz
-l8MgHFAFB9f1b5qWeD5e+c/xWXsyKSYorRswY42kFx3cYf0vwKX1t4aPFXEueKVs
-rqkKVxX/IbDMYN89BT7ONYpglPLKqWr136JeHiTE1Zk+uCv9car4HjDdgT+VZ0Aa
-bMtHOcY1n4Q8JBVzq31PZ2nRWHZXlSJOfzGLvGtfweueg6TZbXPaPhirwtpgE0Eo
-zKWFP4JYbYPLVRZVfWntFU5O+mvb683rABs1k6ZGrKkyiNkOmjRd3KJk2Gl9xvgp
-YekWEqh8RlORSMn6M9QyfZGLGuL/ND7zp256qWSXVOObjQuh3rG4f3yYYlaKgQbA
-INaQ7Cj8ahZoCDWfR+7r5ZmubCz0udZA8jFKO1Mbik8YL3id0ItPvavOn7o9002K
-axKtb7arY1P++rEElQ10j5ODwVnhPGx7G952bUzdusf9c0p5evzcnJaEAQNsnvFL
-ZwKwg26LEwjw0YOJ57ZXzT1kUrdPQNC/Gj9XsuSOEc97GBl2d6SNQ145mtYjVaho
-80BU2mzYmPPi1xZaZB5JNpOkRhu5PoZRl0Sxpx+8Bmu/aaINspR5rhdxDXoi+H0O
-92DllIWmLofqGkg9RK9/8h+WEDZoyr6Y4xXeilm8Xa27Y1mwqwuDnyvrf2D2I0PY
-Qo/umAjNjYRdiJgRxfE+Fcuo6ubpaqwEF/YZj7pSEurjXPsUVQGN+TC1XEgq/fih
-EFkSCk0+v6W1NBOZioGPMet0jfeuvYcBYawq3H5SZ+2OPzAbSS3Ve1ixqDB7Pzpo
-N87sV8JPJyoWpvT68gGTqoef0TVk/Q1gHDTjLSbkC3Ra5XyuG53zS8dQZZUEPLkW
-41VXO1Pr+1KUgMNjmLhVNhCUKD8rHuQSomeaKXFAYW2oi1krR6U0W+V7sI02XUwK
-RmODYG2K1nmkMUNEkmUCZhXjRJRWBZyPI1Vcv8JDK/xvXMvrTvMSWpe+HFDbc0ES
-tfVlnrrzY0pb0PRP/1mn4YluvBnTFJuTgzFNknl1GdMq6TFR+tyuypPh
-=qcHS
+iQUHBAABCgAdFiEEzaAaQgjE90UGEH570atFFoiIiIgFAl8YqBIACgkQ0atFFoiI
+iIgb7yagtq4p8frx0/OinsuCZrGkkARAhRPdSVU9KndJ0nHqmHcuy87fMtRAgoA4
++RVIUlYu/ctbn96PmfOWky9AHO0G+9qwI0KEapP8cWJloRH+Nsnr+njLulb3GAHw
+EukalhBlRiQ0dUolbHBRQR1RU75gnagIBePsPanc2hKAD2Z/QFbx80i6m8KfGoZF
+IFNhsXaysM2mVt4G9bzY8U1CFrjH2v80u9voYMbtU7K6Kpdaxc0uYNI1yHJvXgX4
+EXyJeGXL05ZliPFezi3OoVdk2WBQh529LIXof3I5btHtRrlr4ueFbIx/wk/LI8ND
+pd8BPghVGKuTx/Q7v93JWLEc4MfEJGseD4XSq4WI34Dl3aPux8ueMUDru6hYagl+
+BxiWVh2xHkaTpuBothuqNC+C92ZQB4WLd+EFJde2OPkw2v7ME/A9hnE3qxoMRd7S
+U955RtfIeL93hCGvVfAuNv1V6olQIwMhG8zL8KUVtoNRjW6QC3qX+PrgFVpKT80V
+w3i1uFmYOZ4MElqR5VYfeWOjBlHeOuBRLD6Jh1Z8NDwr/G6dP6zHvA2e5+OwJBU/
+WvxzH9TJ+7sj95S9LCUenAW3xf7t/VJhutA9PA/uyqCXglQ3x7zKamndA8pXNp9L
+oujxNBya5034KaVdN/ah4v2Q85xaM9aE4g1b/HZeImYPWuzqrjcNsvqDLeG4YEJ4
+i+1JzMagqoxOVxr+YxpMIoY47+MsspscnG7p5i+Pm0nT8jtVli070nCnarZH4lqQ
+GqOG1NFh9SzRGo86vRrKR/tcE3cKPqYUgRDa4uj2GXBhVgRyMsNrVgfgc3ItUtqP
+6SjRTGhHz9Xz45e4VI4dpKf2rXw2DS7y1XFrhpbUN28I/KOFs/6fWxL4MdimnnEY
+1YICdJ9FPTgIZC2FB1n5TfiqF5ESKKjDqIAUtKNvKOSn6mWL4JEVwrVYcO6eexIg
+tsbq18vq0Ntp6kH/OXR3a0M7unFIBmF9pqMQQ+BgeirmbRdXRjKbvJNm9x3SIpZz
+c2GGY9d74Pqn889avc49PkeCOOFH7QF6dUPaXXvE3Phpz9bI/Fwgmn3BUUFE5pGr
+xx2OavPP5bwUo6cznQO3oF2YIdpDeh2XufFQ+xArPEWP2PgqXAWiDXoon7kAwovO
+TvzNQjPJZvvGBMz95mcsH0GCwQWyjH5aCuh+mKD0uV1GsWRBHTw4c4tF/QJFzlxE
+/b7aSfhj/EgY3WQ53rMUUhgMX/fQUrWX/BORAAfZgsF0oMOsplz5sVk51oK07bL9
+IgTBGsY3cOw+VsudLPqhFKaczIHRCjnVh6OZQnslHRxGRXGL309kDLW1atBsLl4I
+0arIu391JVpA/Oa/aQi6Zc+jp656gzEzh9NhFBcF5kDitnh7BZjvzD+OYNfBbPcc
+ED1Z4gDsl8D6/I0wCgCt6v0PxAJU3neVjMgXaMx2pbjHq81/KwiizL5La4MXeoeX
+eK1fdUihfZeWsL11qpTy+7F4nOKfII5u2kB9NwuQx9rNns4AaHYMfDD1eB3pSYUW
+Ekm81fHgFWu0VFvBdRPf0dqMEHr+BnJi2FZ2ErD/RxmRhJySFUjTtX+fE3uvSQG7
+EgyiSjPNwBpJlsFFuwcup8bNfd7bGBNEpVnI4MgYdTcBQXFIZJ1EG9ez
+=pW61
-----END PGP SIGNATURE-----
