Re: Regularize shift $((OPTIND - 1)) after getopts

[Andreas Kusalananda Kähäri]

On Wed, Aug 30, 2023 at 12:31:08PM -0700, Simon Branch wrote:
> Hello tech@,
> 
> In an arithmetic substitution -- that is, the $(( ... )) construct in a
> shell script -- one can reference variables either with ($a) or without
> (a) the dollar sign.  However, the latter is slightly better: the
> variable is interpreted as an integer, and then the integer is used
> inside the expression.  With the former, the variable's contents are
> spliced into the expression before it is parsed.  For example:
> 
>   $ a='1+5'
>   $ echo $((a * 2))
>   12
>   $ echo $(($a * 2))
>   11
> 
> Another gotcha with $(($a)):
> 
>   $ echo $((a=2, a))
>   2
>   $ echo $((a=3, $a))  # oops, $a was spliced in before the assignment
>   2
> 
> All this to say, $(($a)) should be reserved for where it is required,
> not used as the default form.
> 
> The following idiom is used throughout the source tree, including in
> sh.1 and ksh.1:
> 
>   while getopts ...
> ...
>   done
>   shift $(($OPTIND - 1))
> 
> and it has a couple of variations:
> 
>   shift $((OPTIND - 1))  my preferred option
>   shift OPTIND-1 works because ksh is magical
>   shift $(( OPTIND -1 )) and other silly whitespace conventions
>   shift "$(($OPTIND - 1))"   unnecessary: $(()) evaluates to an integer

About that last one:  If it wasn't for the fact that OpenBSD ksh and sh
are a bit magical, there would be an issue if $IFS contained a digit.
So to protect the code in the pathological case (non-OpenBSD ksh/sh, and
$IFS containing digits), the quotes are needed.  Now, that's never going
to be an issue in shell script in the OpenBSD source tree obviously, but
still... I'm an academic.

> 
> My patch changes all of these to the $((OPTIND - 1)) usage, which
> encourages good shell practice and is clearly understandable.  It
> ignores src/gnu since I believe those files are vendored in, and there's
> no reason to introduce conflicts when pulling them from upstream.
> 
> Reasonable?

Reasonable.  But I'm not anyone you need to listen to.



-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: [PATCH] Add -executable, -readable, and -writable options to /usr/bin/find

[Andreas Kusalananda Kähäri]

On Sun, Apr 02, 2023 at 10:39:00AM +0200, Kusalananda Kähäri wrote:
> On Sat, Apr 01, 2023 at 08:18:56AM +0200, Solène Rapenne wrote:
> > Le Fri, 31 Mar 2023 19:40:45 -0700,
> > Jared Harper  a écrit :
> > 
> > > I have put together a patch that adds -executable, -readable, and
> > > -writable to /usr/bin/find.
> > > 
> > > When I first started working on this patch, I implemented the access
> > > check by checking the stat of the file like so:
> > > 
> > 
> > this doesn't add much value IMO, we already have -perm that can be used
> > to return paths matching the permission, or only a bit.
> > 
> > find . -executable can be written find . -type f -perm -100
> > find . -writable can   be written find . -type f -perm -200
> > find . -readable can   be written find . -type f -perm -400
> > 
> > on linux, those flags make sense to have because they also take care of
> > ACLs, while their -perm doesn't. OpenBSD doesn't have ACLs.
> 
> 
> Note that e.g. -executable tests whether the current item is readable

s/readable/executable


duh

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: [PATCH] Add -executable, -readable, and -writable options to /usr/bin/find

[Andreas Kusalananda Kähäri]

On Sat, Apr 01, 2023 at 08:18:56AM +0200, Solène Rapenne wrote:
> Le Fri, 31 Mar 2023 19:40:45 -0700,
> Jared Harper  a écrit :
> 
> > I have put together a patch that adds -executable, -readable, and
> > -writable to /usr/bin/find.
> > 
> > When I first started working on this patch, I implemented the access
> > check by checking the stat of the file like so:
> > 
> 
> this doesn't add much value IMO, we already have -perm that can be used
> to return paths matching the permission, or only a bit.
> 
> find . -executable can be written find . -type f -perm -100
> find . -writable can   be written find . -type f -perm -200
> find . -readable can   be written find . -type f -perm -400
> 
> on linux, those flags make sense to have because they also take care of
> ACLs, while their -perm doesn't. OpenBSD doesn't have ACLs.


Note that e.g. -executable tests whether the current item is readable
by the current user, a test that is quite difficult to implement using
-perm (you would have to involve -user and possibly -group too, as well
as $LOGNAME).

With -executable as an example, you could avoid entering directories
that you don't have access to in a convenient way

find . -type d ! -executable -prune -o ...

GNU extensions to standard utilities are almost exclusively about
providing convenient shortcuts and convinient functionality to the lazy
user.  Usually, this extra functionality is found in other standard
tools, or it can trivially be implemented by a short script written
in the shell or using awk etc., so their extensions often seems to be
running counter to the Unix philsophy of "do one thing and do it well".
But then again, "GNU is not Unix".

However, in this particular case, the extension is *useful* and not
at all trivial to implement with existing predicates in find(1) or by
calling other utilities.  The above example could be implemented with
"find . -type d ! -exec cd {} \; -prune -o ..." on systems where cd(1)
is an external utility, which it is not on OpenBSD, so we would use
something silly like "! -exec sh -c 'cd "$1"' sh {} \;" instead, unless
we got either really creative or just ignored the fact that we can't
enter that directory and redirect all diagnostic messages to /dev/null.


Cheers,

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: Adding POSIX c99 compiler wrapper script

[Andreas Kusalananda Kähäri]

On Fri, Feb 11, 2022 at 10:30:26PM +, Stuart Henderson wrote:
> On 2022/02/11 16:13, Joe Nelson wrote:
> > 
> > I noticed that OpenBSD lacks the POSIX "c99" compiler wrapper.
> > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/c99.html
> > 
> > Is it missing because the community feels it's ill conceived, or just
> > because nobody stepped up to implement it? If the latter, I can
> > contribute a patch to add it as a wrapper around cc.
> 
> AFAIK we've never seen anything which uses it. At least not in ports.

I don't want to take sides, but the yash shell, which is in ports, has a
configure script that says

cc="${CC-c99}"

It's likely that CC is set in the environment when the port is built,
but still, it would default to c99 if it wasn't set.


> 
> > A few questions about the desired behavior:
> > 
> > * If the user passes options not listed by POSIX, should the wrapper
> >   die with a usage error, or pass them silently to the underlying
> >   compiler? The FreeBSD implementation [0] does strict checking, while
> >   NetBSD [1] and GCC on Debian [2] pass along all parameters.
> > 
> > * Should it add -pedantic as well as -std=c99?
> > 
> > * Is /usr/bin/c99 something that should go in OpenBSD base, or in
> >   compiler ports?
> 
> If it's added at all, base is probably the place for it.
> 
> > 0: https://github.com/freebsd/freebsd-src/blob/main/usr.bin/c99/c99.c
> 
> Funny, it strips -lrt!
> 
> > 1: https://github.com/NetBSD/src/blob/trunk/usr.bin/c99/c99.sh
> > 2: https://salsa.debian.org/toolchain-team/gcc-defaults/-/blob/master/c99
> > 

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: man sed(1) diff

[Andreas Kusalananda Kähäri]

On Sun, Sep 05, 2021 at 02:01:37PM +0200, Ingo Schwarze wrote:
> Hello,
> 
> i see where Ian's confusion is coming from, even though arguably,
> the existing text is accurate.  But it is not a good idea to insert
> exceptions as parenthetic remarks in the middle of an enumeration
> of steps that is already somewhat long and complicated.
> 
> I think it is better to explain the special rules for D processing
> in the paragraph describing D (surprise, surprise) rather than in
> the middle of the general description of what a "cycle" is.
> 
> 
> Consider the following minimal example:
> 
>$ printf 'axb' | sed -n 'y/x/\n/;s/^b/c/;P;D'
>a
>c
> 
> The D deletes "a\n" from the pattern space.
> After that, the next cycle is entered with "b" in the pattern space,
> without copying a new line into the pattern space.
> 
> 
> OK?
>   Ingo
> 
> 
> Index: sed.1
> ===
> RCS file: /cvs/src/usr.bin/sed/sed.1,v
> retrieving revision 1.48
> diff -u -r1.48 sed.1
> --- sed.1 17 Mar 2016 05:27:10 -  1.48
> +++ sed.1 5 Sep 2021 11:46:48 -
> @@ -140,9 +140,6 @@
>  cyclically copies a line of input, not including its terminating newline
>  character, into a
>  .Em pattern space ,
> -(unless there is something left after a
> -.Ic D
> -function),
>  applies all of the commands with addresses that select that pattern space,
>  copies the pattern space to the standard output, appending a newline, and
>  deletes the pattern space.
> @@ -331,7 +328,8 @@
>  Delete the pattern space and start the next cycle.
>  .It [2addr] Ns Ic D
>  Delete the initial segment of the pattern space through the first
> -newline character and start the next cycle.
> +newline character and start the next cycle without copying the next
> +line of input into the pattern space.
>  .It [2addr] Ns Ic g
>  Replace the contents of the pattern space with the contents of the
>  hold space.


This is an improvement.  This should be compared to the POSIX text:

If the pattern space contains no , delete the pattern
space and start a normal new cycle as if the d command was
issued.  Otherwise, delete the initial segment of the pattern
space through the first , and start the next cycle with
the resultant pattern space and without reading any new input.

The difference is that the POSIX text explicitly says that "D" should
act like "d" if there is no newline character in the pattern space.  The
OpenBSD text does not actually say what happens if there is no newline
character in the pattern space (I believe that OpenBSD sed behaves as
POSIX describes).

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: man sed(1) diff

[Andreas Kusalananda Kähäri]

On Sun, Sep 05, 2021 at 09:51:44AM +0100, ropers wrote:
> I.
> 
> Not to engage in pointless bikeshedding, but I find this clearer and
> --if I understand things correctly-- also more technically accurate:
> 
> Index: sed.1
> ===
> RCS file: /cvs/src/usr.bin/sed/sed.1,v
> retrieving revision 1.60
> diff -C8 -u -r1.60 sed.1
> cvs server: conflicting specifications of output style
> --- sed.1 8 Mar 2021 02:47:28 -   1.60
> +++ sed.1 5 Sep 2021 08:23:25 -
> @@ -141,19 +141,19 @@
>  Normally,
>  .Nm
>  cyclically copies a line of input, not including its terminating newline
>  character, into a
>  .Em pattern space ,
>  (unless there is something left after a
>  .Ic D
>  function),
> -applies all of the commands with addresses that select that pattern space,
> -copies the pattern space to the standard output, appending a newline, and
> -deletes the pattern space.
> +applies all of the commands with addresses that select that pattern,

An address does not select a pattern, it selects the pattern space.

> +copies the pattern space contents to the standard output, appending a

The word "contents" is not needed.

> +newline, and deletes them from the pattern space.

It's unclear what "them" refers to here (possibly "the contents"?).

The proposed text seems to want to separate "the pattern space" from
"the contents of the pattern space".  I don't think this distinction
is helpful, just like it's not helpful ta make a distinction between
"a string" and "the contents of a string".  The manual (and the POSIX
standard text) refers to the data that is read from the input and
currently subject to the commands of the editing script as "the pattern
space" throughout, not just in this introductory paragraph.


>  .Pp
>  Some of the functions use a
>  .Em hold space
>  to save all or part of the pattern space for subsequent retrieval.
>  .Sh SED ADDRESSES
>  An address is not required, but if specified must be a number (that counts
>  input lines
>  cumulatively across input files), a dollar character
> 
> (I used the diff -C 8 option to show a little more context.)
> 
> 
> II.
> 
> [Link for easier reading: <https://man.openbsd.org/sed#pattern>]
> 
> Also, does the "(unless there is something left after a D function)"
> part really relate to the preceding parenthetical clause of "not
> including its terminating newline character"?  Should it be moved to
> directly follow that instead of following the "into a pattern space"
> part?
> Alternatively, would it be better to move the "(...)" part to a
> separate subsequent sentence like this:
> > (A newline character may be present in the pattern space
> > if left behind after a
> > .Ic D
> > function.)
> Is it even important to include that information in the man page?  Is
> it ever relevant that there may technically be some string and a
> newline left in the pattern space?
> 
> 
> Thank you,
> Ian

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: sed(1): line addresses and next cycle

[Andreas Kusalananda Kähäri]

On Sun, Aug 15, 2021 at 11:34:22PM +0200, Martijn van Duren wrote:
> Andreas Kähäri gave a nice example on misc@ on how our sed addressing
> implemenation differs from gsed[0][1]. While writing my reply I noticed
> that POSIX doesn't state how "next cycle" should be interpreted when it
> comes to address ranges. So I can't state that our implementation is
> wrong per se. However, I do think that gsed's interpretation is more
> intuitive, since a numeric address is not dependent on the context of
> the pattern space and thus should register as "in range".

But note that this comes out of a discussion on how to do '0,/re/'
addressing with OpenBSD sed.  Your changes appears to remove one way of
actually handling a match of '/re/' on the first line without giving us
another.  It would be better to have a clean way of doing the equivalent
of '0,/re/' than to remove a way to do this.  Interestingly (?), the sed
in plan9port works the same as our native sed.

Andreas

> 
> Diff below changes program parsing to more closely match gsed in this
> regard:
> $ printf 'test1\nbla1\ntest2\nbla2\n' | sed -e '1 { /^test/d; }' -e 
> '1,/^test/d'  
> bla1
> test2
> bla2
> $ printf 'test1\nbla1\ntest2\nbla2\n' | ./obj/sed -e '1 { /^test/d; }' -e 
> '1,/^test/d'  
> bla2
> $ printf 'bla0\ntest1\nbla1\ntest2\nbla2\n' | ./obj/sed -e '1 { /^test/d; }' 
> -e '1,/^test/d'
> bla1
> test2
> bla2
> $ printf 'test1\nbla1\ntest2\nbla2\n' | gsed -e '1 { /^test/d; }' -e 
> '1,/^test/d' 
> bla2
> $ printf 'bla0\ntest1\nbla1\ntest2\nbla2\n' | gsed -e '1 { /^test/d; }' -e 
> '1,/^test/d'
> bla1
> test2
> bla2
> 
> The diff passes regress, but hasn't had a lot of scrutiny. Just checking
> for general interest in changing this functionality. As soon as I
> know that it's something we might want I'll spend more braincycles on
> it.
> 
> martijn@
> 
> [0] https://marc.info/?l=openbsd-misc=162896537001890=2
> [1] https://marc.info/?l=openbsd-misc=162905748428954=2
> 
> Index: process.c
> ===
> RCS file: /cvs/src/usr.bin/sed/process.c,v
> retrieving revision 1.34
> diff -u -p -r1.34 process.c
> --- process.c 14 Nov 2018 10:59:33 -  1.34
> +++ process.c 15 Aug 2021 21:30:22 -
> @@ -89,14 +89,16 @@ process(void)
>   SPACE tspace;
>   size_t len, oldpsl;
>   char *p;
> + int nextcycle;
>  
>   for (linenum = 0; mf_fgets(, REPLACE);) {
>   pd = 0;
> + nextcycle = 0;
>  top:
>   cp = prog;
>  redirect:
>   while (cp != NULL) {
> - if (!applies(cp)) {
> + if (!applies(cp) || nextcycle) {
>   cp = cp->next;
>   continue;
>   }
> @@ -127,14 +129,16 @@ redirect:
>   break;
>   case 'd':
>   pd = 1;
> - goto new;
> + nextcycle = 1;
> + break;
>   case 'D':
>   if (pd)
>   goto new;
>   if (psl == 0 ||
>   (p = memchr(ps, '\n', psl)) == NULL) {
>   pd = 1;
> - goto new;
> + nextcycle = 1;
> + break;
>   } else {
>   psl -= (p + 1) - ps;
>   memmove(ps, p + 1, psl);
> @@ -267,8 +271,9 @@ new:  if (!nflag && !pd)
>   * (lastline, linenumber, ps).
>   */
>  #define  MATCH(a)\
> - (a)->type == AT_RE ? regexec_e((a)->u.r, ps, 0, 1, 0, psl) :\
> - (a)->type == AT_LINE ? linenum == (a)->u.l : lastline()
> + (a)->type == AT_LINE ? linenum == (a)->u.l :\
> + (a)->type == AT_LAST ? lastline() : \
> + pd ? 0 : regexec_e((a)->u.r, ps, 0, 1, 0, psl)
>  
>  /*
>   * Return TRUE if the command applies to the current line.  Sets the inrange
> 

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: /etc/netstart argument handling

[Andreas Kusalananda Kähäri]

On Fri, Apr 30, 2021 at 09:17:47PM +0200, Tilo Stritzky wrote:
> 
> /etc/netstart contains the following getopts handler:
> 
> while getopts ":n" opt; do
> 
> That colon is just totally bogus, isn't it?


The colon at the start of the optstring has the effect that makes the
getopts utility silent when unsupported options are used.  Instead of
emitting a diagnostic message ("unknown option"), you'll get a "?" in
$opt that you can choose to handle yourself (this is taken care of by
the "*)" case in the code, which calls the "usage" function).

So it's not "totally bogus".  The question is whether you want to see
diagnostic messages from getopts when you call /etc/netstart with
unsupported options in addition to the usage info or not, and it seems
you're saying you do want to see such a message.

> 
> 
> tilo
> 
> Index: netstart
> ===
> RCS file: /cvs/src/etc/netstart,v
> retrieving revision 1.211
> diff -u -p -r1.211 netstart
> --- netstart  23 Dec 2020 17:22:07 -  1.211
> +++ netstart  30 Apr 2021 19:14:01 -
> @@ -233,7 +233,7 @@ PRINT_ONLY=false
>  V4_DHCPCONF=false
>  V6_AUTOCONF=false
> 
> -while getopts ":n" opt; do
> +while getopts n opt; do
>   case $opt in
>   n)  PRINT_ONLY=true;;
>   *)  usage;;

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: ksh(1): '[' and ']' chars disturbing substitution?

[Andreas Kusalananda Kähäri]

On Sat, Dec 26, 2020 at 01:29:58PM +0100, Alessandro De Laurenzis wrote:
> Greetings,
> 
> While extracting the first two characters from a string I noticed the
> following:
> 
> > $ str="-- foo bar blahblahblah"
> > $ echo "First 2 chars: ${str%${str#??}}"
> > First 2 chars: --
> > $ str="-- foo bar blahblahblah [foo]"
> > $ echo "First 2 chars: ${str%${str#??}}"
> > First 2 chars: -- foo bar blahblahblah [foo]
> > $
> 
> It seems that the presence of '[' and ']' is somehow disturbing the
> substitution...
> 
> Quite similar behavior in bash (the only difference is that the substitution
> works there when no chars are present in between the square brackets).
> 
> Is this expected?

The main issue with your code is that you need to quote the pattern to
have it be interpeted as a string an not as a shell globbing pattern:

${str%"${str#??}"}

You may also want to use printf to output variable data (it doesn't
matter in this particular case though, but would matter with bash with
the xpg_echo shell option set, if the two characters happens to be
something like \n or \t).

printf 'First 2 chars: %s\n' "${str%"${str#??}"}"

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: syspatch exit state

[Andreas Kusalananda Kähäri]

On Sun, Dec 06, 2020 at 02:19:05PM +, SW wrote:
> Hi,
> I've been looking to have syspatch give me a quick indication of whether
> a reboot is likely to be required. As a quick and dirty check, I've just
> been treating "Were patches applied?" as the indicator.
> 
> The following diff will cause syspatch to exit when applying patches
> with status code 0 only if patches were actually applied.
> 
> My biggest concern is that it does cause a change in behaviour, so
> perhaps this either needs making into an option or a different approach
> entirely?
> 
> --- syspatch    Sun Dec  6 14:11:12 2020
> +++ syspatch    Sun Dec  6 14:10:23 2020
> @@ -323,3 +323,9 @@ if ((OPTIND == 1)); then
>     _PATCH_APPLIED=true
>     done
>  fi
> +
> +if [ "$_PATCH_APPLIED" = "true" ]; then
> +   exit 0
> +else
> +   exit 1
> +fi
> 
> Thanks,
> S

$_PATCH_APPLIED will be "true" or "false", which means that you could
use it as is done elsewhere in the script:

$_PATCH_APPLIED

If that is the last line of the script, it (true or false) will supply
the exit status of the script.  No need for an if statement or explicit
exit.

Returning a non-zero exit status is generally a sign of some sort of
failure.  Not applying a patch due to already having the patch applied
is not really a failure.

Instead, you could run syspatch -c from a regular cron job and
patch+reboot whenever that tells you there's new patches available.

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: find -exec util {} arg + confusion

[Andreas Kusalananda Kähäri]

On Mon, Nov 16, 2020 at 09:15:11AM +0100, Paul de Weerd wrote:
> Hi Andreas,
> 
> On Mon, Nov 16, 2020 at 08:53:36AM +0100, Andreas Kusalananda Kähäri wrote:
> | On Thu, Nov 12, 2020 at 08:51:22PM +0100, Paul de Weerd wrote:
> | > Hi all,
> | > 
> | > I misread find(1) and did:
> | > 
> | > [weerdpom] $ find path/to/cam -name \*.JPG -exec cp {} path/to/store +
> | > find: -exec no terminating ";" or "+"
> | 
> | Not really what you're asking for, but...
> | 
> | What you seem to want to do can be done with
> | 
> | find path/to/cam -name '*.JPG' -exec sh -c 'cp "$@" path/to/store' sh 
> {} +
> 
> Thanks, I've solved this in the past with small scripts in my homedir
> or going to `find | xargs -J`.  I'll add your suggestion to my list.
> 
> | Or, with GNU coreutils installed,
> | 
> | find path/to/cam -name '*.JPG' -exec gcp -t path/to/store {} +
> 
> Ugh, installing GNU stuff for something like this... :)  Besides, the
> problem is more generic than just cp(1).  Do all GNU tools that (by
> default) have the target argument as the last argument support -t?  I
> mean, I know cp, mv and ln do, but do they all?
> 

That was just given as a suggestion for *if* you happened to have
coreutils installed.  Personally, I would opt for the "-exec sh -c"
variant as it's portable to any sh shell and find/cp implementation (I'm
often working on several different Unices).

The coreutils' info documentation says that the non-standard convenience
option -t (or --target-directory) is implemented for cp, install, ln,
and mv.

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: find -exec util {} arg + confusion

[Andreas Kusalananda Kähäri]

On Thu, Nov 19, 2020 at 03:26:28PM -0800, Jordan Geoghegan wrote:
> 
> 
> On 11/16/20 12:15 AM, Paul de Weerd wrote:
> > Hi Andreas,
> > 
> > On Mon, Nov 16, 2020 at 08:53:36AM +0100, Andreas Kusalananda Kähäri wrote:
> > | On Thu, Nov 12, 2020 at 08:51:22PM +0100, Paul de Weerd wrote:
> > | > Hi all,
> > | >
> > | > I misread find(1) and did:
> > | >
> > | > [weerdpom] $ find path/to/cam -name \*.JPG -exec cp {} path/to/store +
> > | > find: -exec no terminating ";" or "+"
> > |
> > | Not really what you're asking for, but...
> > |
> > | What you seem to want to do can be done with
> > |
> > |   find path/to/cam -name '*.JPG' -exec sh -c 'cp "$@" path/to/store' sh 
> > {} +
> > 
> > Thanks, I've solved this in the past with small scripts in my homedir
> > or going to `find | xargs -J`.  I'll add your suggestion to my list.
> > 
> > | Or, with GNU coreutils installed,
> > |
> > |   find path/to/cam -name '*.JPG' -exec gcp -t path/to/store {} +
> > 
> > Ugh, installing GNU stuff for something like this... :)  Besides, the
> > problem is more generic than just cp(1).  Do all GNU tools that (by
> > default) have the target argument as the last argument support -t?  I
> > mean, I know cp, mv and ln do, but do they all?
> > 
> > | I quite like Alexander's proposed smaller diff.
> > 
> > Making it more clear to the user that + must follow {} is the thing
> > I'd like to achieve, so yeah :)  No strong feelings over his vs my
> > diff.
> > 
> > Cheers,
> > 
> > Paul
> > 
> 
> Hi Paul,
> 
> I've stumbled on this issue a number of times myself. The easiest workaround
> I've found is to do something like this:
> 
> find /tmp/1 -type f -name "*" -print0 | xargs -0 -I{} cp {} /tmp/2/
> 
> No GNU stuff needed.

This would execute cp once for each found name rather than for batches
of found names.  Also, your -name test is a no-op as it matches every
name.


-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: find -exec util {} arg + confusion

[Andreas Kusalananda Kähäri]

On Thu, Nov 12, 2020 at 08:51:22PM +0100, Paul de Weerd wrote:
> Hi all,
> 
> I misread find(1) and did:
> 
> [weerdpom] $ find path/to/cam -name \*.JPG -exec cp {} path/to/store +
> find: -exec no terminating ";" or "+"

Not really what you're asking for, but...

What you seem to want to do can be done with

find path/to/cam -name '*.JPG' -exec sh -c 'cp "$@" path/to/store' sh 
{} +

Or, with GNU coreutils installed,

find path/to/cam -name '*.JPG' -exec gcp -t path/to/store {} +

> 
> That was somewhat surprising - there is a terminating "+".  The error
> really is that the "+" doesn't follow immediately after the "{}"
> (which the manpage of course told me).  Although it would be nice to
> be able to use tools like cp and mv to -exec with +, I suspect there
> to be dragons.  So I'm proposing to change the error to point this out
> to the unsuspecting user.
> 
> Cheers,
> 
> Paul 'WEiRD' de Weerd
> 
> Index: function.c
[cut]


I quite like Alexander's proposed smaller diff.

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: Document that openlog's first paramater must continue to point to valid data

[Andreas Kusalananda Kähäri]

On Mon, Feb 03, 2020 at 09:28:15PM +0100, Ingo Schwarze wrote:
> Hi,
> 
> since our manual page doesn't explain the details of how openlog(3)
> uses *ident, it seems reasonable for users to conclude that it is
> safest to neither free nor modify it.
> 
> Then again, given that in our implementation, freeing it may even
> pose a security hazard, i might seem friendly to give more details.
> 
> POSIX has the same wording as our manual page (and so do 4.4BSD-Lite2,
> FreeBSD, NetBSD, and Oracle Solaris 11).  As far as i understand,
> the wording being in POSIX implies behaviour is unspecified if the
> memory becomes invalid or if its content is changed.  That's also
> how the Linux man-pages project documents it.  I don't think
> researching history is needed for this; knowing that it's unspecified
> feels sufficient.
> 
> Given that our implementation chooses to use-after-free (as it is
> permitted to) if the memory becomes invalid, i prefer the Theo's
> strong wording "must persist" to the possibly less discouraging
> "unspecified" - foremost, we are documenting *our* implementation.
> 
> Regarding changes of the content, i consider it friendly to mention
> that it is unspecified - otherwise, people might mistakenly assume
> that our behaviour were required by POSIX.
> 
> While here, add the missing pointer to POSIX, correct HISTORY,
> drop redundant verbiage from RETURN VALUES, and garbage collect .Tn.
> Admittedly, that's more than one change in one patch, but all of
> it is fairly standard, so why waste time splitting it.
> (Still, feel free to OK only parts, of course.)

For what it's worth:

Related: https://www.austingroupbugs.net/view.php?id=1244

A proposal seems to have been accepted (if I'm reading it correctly) in
November of last year to change the wording in POSIX from

The ident argument is a string that is prepended to every
message.

to

The ident argument is a pointer to a null-terminated identifier
that shall be prepended (without the null terminator) to every
message. The application shall ensure that the string pointed
to by ident remains valid during the syslog() calls that will
prepend this identifier; however, it is unspecified whether
changes made to the string will change the identifier prepended
by later syslog() calls.


Regards,

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: Teach du(1) the -m flag, disk usage in megabytes

[Andreas Kusalananda Kähäri]

On Wed, Jan 29, 2020 at 06:38:31PM +0100, Ingo Schwarze wrote:
> Hi Andreas,
> 
> Andreas Kusalananda wrote on Wed, Jan 29, 2020 at 06:09:54PM +0100:
> > On Wed, Jan 29, 2020 at 05:48:44PM +0100, Ingo Schwarze wrote:
> 
> >> +.Dl du -sh * .??* | sort -h
> 
> > Why the ".??*"? That would miss single (well, double) letter hidden
> > directory names like .a, .b etc.
> 
> True, but those are very uncommon in practice.
> This is not supposed to be something that is perfect for each
> and every occasion, but just an example that is easy to type
> in interactive use and usually good enough.  Also, it is quite
> obvious that .??* won't match ".a".
> 
> > As far as I know, neither ksh(1) nor sh(1) expands .* to ..
> > (which I assume the ?? in .??* is a precaution against).
> 
> I have no intention of ensnaring users of other shells:
> 
>$ uname -a
>   OpenBSD isnote.usta.de 6.6 GENERIC.MP#583 amd64
>$ csh -c 'echo .*'
>   . ..

Ah, you are correct.  I see that it's a slight mischance that I've never
had to use csh(1).

Regards,

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: Teach du(1) the -m flag, disk usage in megabytes

[Andreas Kusalananda Kähäri]

On Wed, Jan 29, 2020 at 05:48:44PM +0100, Ingo Schwarze wrote:
> Hi Jason,
> 
> Jason McIntyre wrote on Wed, Jan 29, 2020 at 04:26:42PM +:
> 
> > i don;t think it would be such a bad thing for du to have an example or
> > two, so i'm ok with this.
> > 
> >   so sth like:
> > 
> > Display a summary of files and folders in the current directory,
> > sorted by size:
> > 
> > $ blah
> 
> That wording seems very nice to me, see below for a patch.
> 
> I also included .??* because having large hidden directories around
> and desperately searching for the waste of disk space elsewhere
> seems like a common trap to me.
> 
> > - i guess if you have a lot of stuff it makes sense to have the biggest
> >   files displayed at the end of the list. but generally wouldn;t you
> >   want your biggest files listed first? we could add -r to sort.
> 
> Actually, i consider it more versatile without the -r.
> You are right, it really matters for directories with lots of entries.
> But even with few entries, what's wrong with having the most relevant
> entries closest to the subsequent shell prompt?
> Everything is visible on the screen in that case either way.
> 
> Feel free to commit something like this (with or without the .??*,
> with ot without the -r) based on OK schwarze@, or send an OK to me.
> 
> Yours,
>   Ingo
> 
> 
> Index: du.1
> ===
> RCS file: /cvs/src/usr.bin/du/du.1,v
> retrieving revision 1.35
> diff -u -r1.35 du.1
> --- du.1  2 Sep 2019 21:18:41 -   1.35
> +++ du.1  29 Jan 2020 16:40:13 -
> @@ -147,6 +147,11 @@
>  .El
>  .Sh EXIT STATUS
>  .Ex -std du
> +.Sh EXAMPLES
> +Display a summary of files and folders in the current directory,
> +sorted by size:
> +.Pp
> +.Dl du -sh * .??* | sort -h

Why the ".??*"? That would miss single (well, double) letter hidden
directory names like .a, .b etc.  As far as I know, neither ksh(1) nor
sh(1) expands .* to .. (which I assume the ?? in .??* is a precaution
against).

$ ls -la
total 4
drwxr-xr-x  2 kk  wheel  512 Jan 29 18:03 .
drwxr-xr-x  3 kk  wheel  512 Jan 29 18:03 ..
$ echo .*
.*

>  .Sh SEE ALSO
>  .Xr df 1 ,
>  .Xr fts_open 3 ,

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Naive patch to allow amd(8) to mount NFS with wxallowed

[Andreas Kusalananda Kähäri]

I noticed that the following amd(8) map did not actually mount the
remote NFS filesystem with the wxallowed mount option:

/defaults   sublink:=${key}
*   host==eeyore;type:=link;fs:=/vol/local/${host}; \
host!=eeyore;type:=nfs;rhost:=eeyore;rfs:=/vol/local/${host};\
opts:=rw,tcp,intr,soft,nosuid,nodevs,wxallowed;

On a machine "box", this mounts "eeyore:/vol/local/box" as

eeyore:/vol/local/box on /tmp_mnt/eeyore/vol/local/box type nfs (nodev, nosuid, 
v2, tcp, soft, intr, timeo=100)

Looking at src/usr.sbin/amd/amd/mount_fs.c, this appears to be because
only a select set of mount options are being allowed.  These are "ro",
"nodev", "noexec", "nosuid", and "sync".

I tested the attached very naive patch on my amd64-current system, and
it *seems* to work:

eeyore:/vol/local/box on /tmp_mnt/eeyore/vol/local/box type nfs (nodev, nosuid, 
wxallowed, v2, tcp, soft, intr, timeo=100) 

Regards,

-- 
Andreae (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.
Index: mount_fs.c
===
RCS file: /extra/cvs/src/usr.sbin/amd/amd/mount_fs.c,v
retrieving revision 1.14
diff -u -p -r1.14 mount_fs.c
--- mount_fs.c  20 Oct 2014 06:55:59 -  1.14
+++ mount_fs.c  20 Jan 2020 22:34:32 -
@@ -50,6 +50,7 @@ struct opt_tab mnt_flags[] = {
{ "noexec", MNT_NOEXEC },
{ "nosuid", MNT_NOSUID },
{ "sync",   MNT_SYNCHRONOUS },
+   { "wxallowed",  MNT_WXALLOWED },
{ 0, 0 }
 };
 

Re: grep -R with no path

[Andreas Kusalananda Kähäri]

On Tue, Dec 03, 2019 at 01:00:36AM +0100, Klemens Nanni wrote:
> On Mon, Dec 02, 2019 at 08:31:02AM +, Miod Vallat wrote:
> > grep(1), when invoked with the -R option but no path, displays a
> > "recursive search of stdin" warning and acts as if -R had not been
> > specified.
> > 
> > GNU grep, in that case, will perform a recursive search in the current
> > directory, i.e. uses an implicit "." path if none is given.
> > 
> > This is IMO a much better behaviour. What about the following diff?
> Document this.
> 
> OK?
> 
> Index: grep.1
> ===
> RCS file: /cvs/src/usr.bin/grep/grep.1,v
> retrieving revision 1.49
> diff -u -p -r1.49 grep.1
> --- grep.17 Oct 2019 20:51:34 -   1.49
> +++ grep.12 Dec 2019 23:58:56 -
> @@ -246,6 +246,11 @@ will only search a file until a match ha
>  making searches potentially less expensive.
>  .It Fl R
>  Recursively search subdirectories listed.
> +If no
> +.Ar file
> +is given,
> +.Nm
> +searches the working directory.
>  .It Fl s
>  Silent mode.
>  Nonexistent and unreadable files are ignored

Could I suggest using "the current working directory" or "the current
directory" in place of "the working directory"?

Regards,

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

Re: find.1: Use -delete in EXAMPLES

[Andreas Kusalananda Kähäri]

On Thu, Aug 22, 2019 at 12:37:28PM +0200, Klemens Nanni wrote:
> We support -delete since the following and I see no reason to prefer the
> current examples for the very same reasons tedu already outlined:
> 
>   find.c revision 1.21
>   date: 2017/01/03 21:31:16;  author: tedu;  state: Exp;  lines: +10 -4;
>   add -delete option which can simplify the common case of wanting to 
> delete
>   lots of files without the arcane -exec or error prone xargs.
>   code from freebsd.
>   ok millert
> 
> CAVEATS even goes into detail wrt. removing special files, so no
> information seems to be lost with this diff.
> 
> Feedback? OK?

You may want to single quote the patterns as '*.jpg' and '*.gif' as is
done a few lines further up in the manual. Also, since the text says
"files" (assuming this means "regular files"), one may want to add -type f
to the tests in the command.


Regards,

> 
> Index: find.1
> ===
> RCS file: /cvs/src/usr.bin/find/find.1,v
> retrieving revision 1.96
> diff -u -p -r1.96 find.1
> --- find.16 Dec 2018 17:45:14 -   1.96
> +++ find.122 Aug 2019 10:31:14 -
> @@ -581,9 +581,7 @@ ending in a dot and single digit, but sk
>  Find and remove all *.jpg and *.gif files under the current working
>  directory:
>  .Pp
> -.Dl "$ find . \e( -name \e*.jpg -o -name \e*.gif \e) -exec rm {} \e;"
> -or
> -.Dl "$ find . \e( -name \e*.jpg -o -name \e*.gif \e) -print0 | xargs -0r rm"
> +.Dl "$ find . \e( -name \e*.jpg -o -name \e*.gif \e) -delete"
>  .Sh SEE ALSO
>  .Xr chflags 1 ,
>  .Xr chmod 1 ,

Re: Conditional sysupgrade

[Andreas Kusalananda Kähäri]

On Sat, Apr 27, 2019 at 01:23:20PM +0100, Marco Bonetti wrote:
> Hello folks,
> 
> First of all congratulations on a new OpenBSD release and thanks for
> introducing sysupgrade in -current.
> 
> Before sysupgrade, I was using a custom script for achieving the same
> result with only difference that I was checking if a new snapshot (or
> release) is available by looking at BUILDINFO before starting the
> upgrade process.
> 
> Patch below introduce the same behaviour using SHA256.sig as control
> file. If you believe there is a valid use case for reinstalling already
> applied sets to the running system please let me know and I can add a
> -f force option.
> 
> Cheers,
> Marco

I was going to suggest something similar.

See comment below.

> 
> Index: usr.sbin/sysupgrade/sysupgrade.8
> ===
> RCS file: /cvs/src/usr.sbin/sysupgrade/sysupgrade.8,v
> retrieving revision 1.2
> diff -u -p -u -r1.2 sysupgrade.8
> --- usr.sbin/sysupgrade/sysupgrade.8  26 Apr 2019 05:54:49 -  1.2
> +++ usr.sbin/sysupgrade/sysupgrade.8  27 Apr 2019 11:54:40 -
> @@ -28,7 +28,7 @@
>  .Nm
>  is a utility to upgrade
>  .Ox
> -to the next release or a new snapshot.
> +to the next release or a new snapshot if available.
>  .Pp
>  .Nm
>  downloads the necessary files to
> 
> Index: usr.sbin/sysupgrade/sysupgrade.sh
> ===
> RCS file: /cvs/src/usr.sbin/sysupgrade/sysupgrade.sh,v
> retrieving revision 1.6
> diff -u -p -u -r1.6 sysupgrade.sh
> --- usr.sbin/sysupgrade/sysupgrade.sh 26 Apr 2019 21:52:39 -  1.6
> +++ usr.sbin/sysupgrade/sysupgrade.sh 27 Apr 2019 11:54:48 -
> @@ -110,7 +110,19 @@ fi
>  
>  cd ${SETSDIR}
>  
> -unpriv -f SHA256.sig ftp -Vmo SHA256.sig ${URL}SHA256.sig
> +unpriv -f SHA256.sig.tmp ftp -Vmo SHA256.sig.tmp ${URL}SHA256.sig
> +TMP_SHA=$(sha256 -q SHA256.sig.tmp)
> +
> +unpriv touch SHA256.sig
> +CUR_SHA=$(sha256 -q SHA256.sig)
> +
> +if [[ "${TMP_SHA}" = "${CUR_SHA}" ]]; then

Why compare checksums?

if cmp -s SHA256.sig SHA256.sig.tmp; then

> + rm SHA256.sig.tmp
> + return 0
> +fi
> +
> +unpriv cat SHA256.sig.tmp >SHA256.sig
> +rm SHA256.sig.tmp
>  
>  _KEY=openbsd-${_KERNV[0]%.*}${_KERNV[0]#*.}-base.pub
>  _NEXTKEY=openbsd-${NEXT_VERSION%.*}${NEXT_VERSION#*.}-base.pub

-- 
Andreas Kusalananda Kähäri,
National Bioinformatics Infrastructure Sweden (NBIS),
Uppsala University, Sweden.

Re: bgpd use long long instead of int64_t

[Andreas Kusalananda Kähäri]

; -   int64_t sumq;
> > +   long long   num;
> > +   long long   min;
> > +   long long   max;
> > +   long long   sum;
> > +   long long   sumq;
> >  };
> >  
> >  #defineMRT_FILE_LEN512
> > Index: usr.sbin/bgpd/session.h
> > ===
> > RCS file: /cvs/src/usr.sbin/bgpd/session.h,v
> > retrieving revision 1.131
> > diff -u -p -r1.131 session.h
> > --- usr.sbin/bgpd/session.h 18 Feb 2019 09:58:19 -  1.131
> > +++ usr.sbin/bgpd/session.h 18 Feb 2019 20:53:38 -
> > @@ -149,22 +149,22 @@ struct ctl_conn {
> >  TAILQ_HEAD(ctl_conns, ctl_conn)ctl_conns;
> >  
> >  struct peer_stats {
> > -   u_int64_tmsg_rcvd_open;
> > -   u_int64_tmsg_rcvd_update;
> > -   u_int64_tmsg_rcvd_notification;
> > -   u_int64_tmsg_rcvd_keepalive;
> > -   u_int64_tmsg_rcvd_rrefresh;
> > -   u_int64_t        msg_sent_open;
> > -   u_int64_tmsg_sent_update;
> > -   u_int64_tmsg_sent_notification;
> > -   u_int64_tmsg_sent_keepalive;
> > -   u_int64_tmsg_sent_rrefresh;
> > -   u_int64_tprefix_rcvd_update;
> > -   u_int64_tprefix_rcvd_withdraw;
> > -   u_int64_tprefix_rcvd_eor;
> > -   u_int64_tprefix_sent_update;
> > -   u_int64_tprefix_sent_withdraw;
> > -   u_int64_tprefix_sent_eor;
> > +   unsigned long long   msg_rcvd_open;
> > +   unsigned long long   msg_rcvd_update;
> > +   unsigned long long   msg_rcvd_notification;
> > +   unsigned long long   msg_rcvd_keepalive;
> > +   unsigned long long   msg_rcvd_rrefresh;
> > +   unsigned long long   msg_sent_open;
> > +   unsigned long long   msg_sent_update;
> > +   unsigned long long   msg_sent_notification;
> > +   unsigned long long   msg_sent_keepalive;
> > +   unsigned long long   msg_sent_rrefresh;
> > +   unsigned long long   prefix_rcvd_update;
> > +   unsigned long long   prefix_rcvd_withdraw;
> > +   unsigned long long   prefix_rcvd_eor;
> > +   unsigned long long   prefix_sent_update;
> > +   unsigned long long   prefix_sent_withdraw;
> > +   unsigned long long   prefix_sent_eor;
> > time_t   last_updown;
> > time_t   last_read;
> > u_int32_tprefix_cnt;
> > 
> > 

-- 
Andreas Kusalananda Kähäri,
National Bioinformatics Infrastructure Sweden (NBIS),
Uppsala University, Sweden.

Re: man -> vim

[Andreas Kusalananda Kähäri]

On Fri, Feb 15, 2019 at 11:19:57AM +0100, Oleg Pahl wrote:
> Hi @all,
> 
> Today I try to work with man pages using *vim*.
> 
> *# man man | vim -*
> 
> why i see raw (a lot of special char.) data by default? is it ok?
> 
> *# ma man | less *
> 
> I see no special characters. looks very good.
> 
> Is it possible to convert *raw* data of man page to normal format (for
> *vim*) with one command?
> 
> BR,
> 
> -op

man man | col -b | vim -

See the col(1) manual.  This is also mentioned in the mandoc(1) manual,
in the "ASCII Output" section.


-- 
Andreas Kusalananda Kähäri,
National Bioinformatics Infrastructure Sweden (NBIS),
Uppsala University, Sweden.

Re: MySQL Server on OpenBSD

[Andreas Kusalananda Kähäri]

On Mon, Dec 11, 2017 at 03:34:47PM +, Luis Coronado wrote:
> Mysql was replaced by mariadb some time ago. I have not seen any problems
> with the transition but ymmv.
> 
> -luis
> 
> On Mon, Dec 11, 2017 at 9:21 AM Сергей Тарасов <tsv1...@yandex.ru> wrote:
> 
> > Hello.
> > 
> > Is there a package mysql-server in OpenBSD version 6.2 ?
> >
> > 
> > Sergey Tarasov
> >
> >


The only issue I've come across with MariaDB vs. MySQL is:

ERROR 1349 (HY000) at line 136: View's SELECT contains a subquery in the FROM 
clause

This is me trying to recreate a schema that I'm using at work, where
we use MySQL running on Ubuntu.  Not a showstopper for us as our main
development platform for this particular project isn't OpenBSD.  And
even if we were forced to use MariaDB we would probably quite easily
be able to rewrite the schema without the view's subquery in that FROM
clause.


Regards,

-- 
Andreas Kusalananda Kähäri,
National Bioinformatics Infrastructure Sweden (NBIS),
Uppsala University, Sweden.

Re: bc -l wording

[Andreas Kusalananda Kähäri]

For what it's worth, POSIX has

-l
(The letter ell.) Define the math functions and initialize scale
to 20, instead of the default zero; see the EXTENDED DESCRIPTION
section.

http://pubs.opengroup.org/onlinepubs/9699919799.2016edition/utilities/bc.html

Cheers,


On Fri, Oct 06, 2017 at 02:27:33PM +0200, Jan Stary wrote:
> Currently, the bc(1) manpage describes "-l" as
> 
>   Allow specification of an arbitrary precision math library
> 
> I am not a native speaker, but "specification of a library"
> seems unclear here. It loads /usr/share/misc/bc.library,
> not that the user could "specify" some other library to load.
> 
> Later in the manpage, "the functions available in _the_ library"
> are described, and FILES makes it explicit that this is the one.
> 
> Perhaps a native speaker will come up with some better wording.
> 
>   Jan
> 
> 
> 
> Index: bc.1
> ===
> RCS file: /cvs/src/usr.bin/bc/bc.1,v
> retrieving revision 1.32
> diff -u -p -r1.32 bc.1
> --- bc.1  17 Nov 2015 05:45:35 -  1.32
> +++ bc.1  6 Oct 2017 12:20:10 -
> @@ -76,7 +76,7 @@ If multiple
>  options are specified, they are processed in the order given,
>  separated by newlines.
>  .It Fl l
> -Allow specification of an arbitrary precision math library.
> +Load an arbitrary precision math library.
>  The definitions in the library are available to command line
>  expressions.
>  .El
> 

-- 
Andreas Kusalananda Kähäri,
National Bioinformatics Infrastructure Sweden (NBIS),
Uppsala University, Sweden.

Re: Can not allocate memory for KERN_MSGBUF

[Andreas Kusalananda Kähäri]

On Fri, Jun 24, 2016 at 01:19:32PM +0200, Alexander Bluhm wrote:
> On Fri, Jun 24, 2016 at 10:30:04AM +0200, Andreas Kusalananda K?h?ri wrote:
> > I'm following -current on amd64, running in VirtualBox.  I recompiled
> > the kernel today with fresh sources, like I do a few times a week on
> > slow days.
> 
> Have you also installed /usr/include/sys/msgbuf.h from
> /usr/src/sys/sys/msgbuf.h and recomiled /usr/src/sbin/dmesg ?
> A make build should do that.
> 
> bluhm
> 

I believe I have fallen into the trap of not recompiling the userland
binaries before reacting te seeing the dmesg command misbehave.

I was doing a regular update of the system so I recompiled the kernel
and rebooted.  I have a piece of shell code in my login scripts that
compares the dmsg output from the current boot with that of the previous
boot, and when I saw that there was no output from the current boot I
reacted before thinking.


I am sincerely sorry for the noise.

Regards,


signature.asc
Description: PGP signature

Can not allocate memory for KERN_MSGBUF

[Andreas Kusalananda Kähäri]

On Thu, Jun 23, 2016 at 07:15:21AM -0600, Alexander Bluhm wrote:
> CVSROOT:  /cvs
> Module name:  src
> Changes by:   bl...@cvs.openbsd.org   2016/06/23 07:15:21
> 
> Modified files:
>   sys/kern   : subr_log.c 
>   sys/sys: msgbuf.h 
> 
> Log message:
> It is annoying that the dmesg buffer can overflow and loose messages
> undetected during debugging.  To make clear what happens, count the
> dropped bytes and write message buffer full to syslogd.  This also
> helps to have a reliable log system.
> OK deraadt@ millert@ tedu@
> 

Hi,

I'm following -current on amd64, running in VirtualBox.  I recompiled
the kernel today with fresh sources, like I do a few times a week on
slow days.

After reboot:

$ dmesg
dmesg: sysctl: KERN_MSGBUF: Cannot allocate memory

I have no /etc/sysctl.conf and I don't set those variables by
hand.

The issue *seems* to have been introduced with the commit above
as backing it out fixes it.

dmesg output without the commit (or any later commit):

OpenBSD 6.0-beta (GENERIC.MP) #26: Fri Jun 24 10:16:28 CEST 2016
kk@uerfale:/sys/arch/amd64/compile/GENERIC.MP
real mem = 4278124544 (4079MB)
avail mem = 4143943680 (3951MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xe1000 (10 entries)
bios0: vendor innotek GmbH version "VirtualBox" date 12/01/2006
bios0: innotek GmbH VirtualBox
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP APIC SSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz, 3193.07 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,RDRAND,NXE,LONG,LAHF,ABM,ITSC
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: CPU supports MTRRs but not enabled by BIOS
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz, 3192.77 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,RDRAND,NXE,LONG,LAHF,ABM,ITSC
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
"PNP0303" at acpi0 not configured
"PNP0F03" at acpi0 not configured
acpiac0 at acpi0: AC unit online
acpivideo0 at acpi0: GFX0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel 0 
configured to compatibility, channel 1 configured to compatibility
wd0 at pciide0 channel 0 drive 0: 
wd0: 128-sector PIO, LBA, 51200MB, 104857600 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
atapiscsi0 at pciide0 channel 1 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0:  ATAPI 5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
vga1 at pci0 dev 2 function 0 "InnoTek VirtualBox Graphics Adapter" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
em0 at pci0 dev 3 function 0 "Intel 82543GC" rev 0x02: apic 2 int 19, address 
08:00:27:b1:89:72
"InnoTek VirtualBox Guest Service" rev 0x00 at pci0 dev 4 function 0 not 
configured
piixpm0 at pci0 dev 7 function 0 "Intel 82371AB Power" rev 0x08: SMBus disabled
em1 at pci0 dev 8 function 0 "Intel 82543GC" rev 0x02: apic 2 int 16, address 
08:00:27:ff:fb:ca
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on wd0a (22d20e8d1f94fa20.a) swap on wd0b dump on wd0b


signature.asc
Description: PGP signature

Re: tr.1 -s wording

[Andreas Kusalananda Kähäri]

On Mon, Apr 18, 2016 at 07:11:46PM -0400, Christian Heckendorf wrote:
> Hi,
> After reading the man page description of tr -s, I couldn't understand
> what it meant to "squeeze" characters. It could be that 'squeeze' is a
> mnemonic for '-s' but it's not helpful as a technical explanation.
> 
> --
> Christian

I prefer "removes consecutive occurrences" over "removes consecutive
repetitions".   "ABCABCABC" is a consecutive repetition of the
characters "ABC".



> 
> 
> Index: tr.1
> ===
> RCS file: /cvs/src/usr.bin/tr/tr.1,v
> retrieving revision 1.25
> diff -u -p -u -r1.25 tr.1
> --- tr.1  28 Feb 2015 21:51:57 -  1.25
> +++ tr.1  18 Apr 2016 22:53:28 -
> @@ -82,12 +82,12 @@ option causes characters to be deleted f
>  .It Fl s
>  The
>  .Fl s
> -option squeezes multiple occurrences of the characters listed in the last
> +option removes consecutive repetitions of characters listed in the last
>  operand (either
>  .Ar string1
>  or
>  .Ar string2 )
> -in the input into a single instance of the character.
> +from the input.
>  This occurs after all deletion and translation is completed.
>  .El
>  .Pp
> 

-- 
Andreas Kusalananda Kähäri, Bioinformatics Developer (NBIS)
Uppsala University, Uppsala, Sweden
NBIS: http://nbis.se OpenPGP: url=https://db.tt/2zaB1E7y; id=46082BDF



signature.asc
Description: PGP signature

Quoting ${CC} expansion in libiberty Makefile

[Andreas Kusalananda Kähäri]

Previously sent to misc@, but I was told to send it here instead.

Cheers,
Andreas

- Forwarded message from Andreas Kusalananda Kähäri 
<andreas.kah...@icm.uu.se> -

Date: Mon, 18 Jan 2016 16:33:34 +0100
From: Andreas Kusalananda Kähäri <andreas.kah...@icm.uu.se>
To: openbsd-misc <m...@openbsd.org>
Subject: Quoting ${CC} expansion in libiberty Makefile

Hi,

I tried running the base system build with CC="ccache cc" and it broke
while compiling libiberty due to an unquoted expansion of ${CC} in the
Makefile ("${MAKE} ${GNUCFLAGS} CC=${CC} needed-list").

I know I won't save much time by using ccache since the compiler is
rebuilt during the process, but I assume that a similar problem would
arise if one tried to use CC="distcc cc", for example.  Also, other
Makefiles in the tree quotes ${CC}, and this is *almost* the sole
instance where it's not quoted.  Other instances are
lib/libcrypto/crypto/arch/{mips64,sparc64}/Makefile.inc

Patch:

Index: Makefile.bsd-wrapper
===
RCS file: /cvs/src/gnu/lib/libiberty/Makefile.bsd-wrapper,v
retrieving revision 1.15
diff -u -p -u -r1.15 Makefile.bsd-wrapper
--- Makefile.bsd-wrapper31 Aug 2014 01:02:48 -  1.15
+++ Makefile.bsd-wrapper18 Jan 2016 15:28:44 -
@@ -32,7 +32,7 @@ CLEANFILES+=  Makefile config.cache confi
 depend:needed-list
 
 needed-list: config.status
-   ${MAKE} ${GNUCFLAGS} CC=${CC} needed-list
+   ${MAKE} ${GNUCFLAGS} CC="${CC}" needed-list
 
 config.status: Makefile.in configure 
    PATH="/bin:/usr/bin:/sbin:/usr/sbin" \



-- 
Andreas Kusalananda Kähäri, Bioinformatics Developer, Uppsala, Sweden
OpenPGP: url=https://db.tt/2zaB1E7y; id=46082BDF
------------



- End forwarded message -

-- 
Andreas Kusalananda Kähäri, Bioinformatics Developer, Uppsala, Sweden
OpenPGP: url=https://db.tt/2zaB1E7y; id=46082BDF



signature.asc
Description: PGP signature

ntpd pledge, needs "unix" to talk to ntpctl

[Andreas Kusalananda Kähäri]

Hi,

I noticed that ntpd would die if I tried to use ntpctl to check on it:

[...]
29946 ntpd CALL  poll(0xda8993ab5c0,4,1000)
29946 ntpd RET   poll 1
29946 ntpd CALL  kbind(0x7f7c2558,0x18,0x7bb3facd5f812ed9)
29946 ntpd RET   kbind 0
29946 ntpd CALL  accept(5,0x7f7c2630,0x7f7c262c)
29946 ntpd PLDG  accept, "unix", errno 1 Operation not permitted
29946 ntpd PSIG  SIGABRT SIG_DFL
[...]

I also get ntpd(): syscall 30 "unix" in the console.

Cheer,

ps. is tech@ the right list for these sorts of things?

-- 
:: Andreas Kusalananda Kähäri
:: Bioinformatics Developer
:: Uppsala, Sweden
::--

On console: newsyslog(3865): sycall 59 "exec"

[Andreas Kusalananda Kähäri]

Hi,

I wonder if this message that I'm seeing in my console is coming from
the 1.96 commit to usr.bin/newsyslog/newsyslog.c?

newsyslog(): syscall 59 "exec"

Running "newsyslog" as root produces no such output, in the console or
elsewhere that I can spot.  However, running "newsyslog -F" seems to set
off one of those messages per rotated log file(?).

My system is an amd64 system with sources from this morning.

Andreas

-- 
:: Andreas Kusalananda Kähäri
:: Bioinformatics Developer
:: Uppsala, Sweden
::--

Re: Pledge shutdown halt

[Andreas Kusalananda Kähäri]

On Tue, Nov 17, 2015 at 11:31:57PM -0700, Theo de Raadt wrote:
> What ttys are you guys rebooting from.  Is it seen on console?  Or... is it
> only seen from xterm...?
> 

I just now rebuilt my amd64 machine with a fresh checkout from
"anon...@anoncvs.eu.openbsd.org:/cvs".

The message displays in the terminal from which the command was
executed.  Rebooting as root in the console, I see the message in the
console.  Rebooting with "doas reboot" from logging in as a user with
SSH I get the same:

$ doas reboot
reboot: revoke: Inappropriate ioctl for device
Shared connection to xxx.xxx.xx.xxx closed.
Connection to xxx.xxx.xx.xxx closed by remote host.

I do not run X.

-- 
:: Andreas Kusalananda Kähäri
:: Bioinformatics Developer
:: Uppsala, Sweden
::--

[Patch] Quoting ${CC} in Makefile wrapper, libiberty

[Andreas Kusalananda Kähäri]

Hi,

The Makefile.bsd-wrapper for src/gnu/lib/libiberty is the only such
Makefile (that I could find) that does not quote ${CC} properly.

A trivial patch is attached.

Cheers,

-- 
:: Andreas Kusalananda Kähäri, Bioinformatics Developer, BILS,
:: Uppsala University, Sweden
::--
Index: Makefile.bsd-wrapper
===
RCS file: /cvs/src/gnu/lib/libiberty/Makefile.bsd-wrapper,v
retrieving revision 1.15
diff -u -r1.15 Makefile.bsd-wrapper
--- Makefile.bsd-wrapper31 Aug 2014 01:02:48 -  1.15
+++ Makefile.bsd-wrapper6 May 2015 18:03:30 -
@@ -32,7 +32,7 @@
 depend:needed-list
 
 needed-list: config.status
-   ${MAKE} ${GNUCFLAGS} CC=${CC} needed-list
+   ${MAKE} ${GNUCFLAGS} CC=${CC} needed-list
 
 config.status: Makefile.in configure 
PATH=/bin:/usr/bin:/sbin:/usr/sbin \