Re: Preferred TERM for pkg_add

[Nicholas Marriott]

>From a quick look it will happily fall back to carriage return and space
if the capabilities it would prefer (el and hpa/RI) are not present, so
probably anything would do.

Or you could check if the terminfo entry existed with something like
"tput el >/dev/null 2>&1" before running pkg_add.


On Thu, Jan 05, 2023 at 08:45:47AM +0100, Patrik Lundin wrote:
> Hello,
> 
> There was recently a bug report opened for the ansible module
> openbsd_pkg, where someone was getting errors due to pkg_add disliking
> their TERM setting:
> https://github.com/ansible-collections/community.general/issues/5738
> 
> Basically they are running TERM=xterm-kitty locally, which makes pkg_add
> disable the progress meter, easily testable like so:
> ```
> orbb# nmap
> ksh: nmap: not found
> orbb# TERM=xterm-kitty pkg_add nmap
> No progress meter: failed termcap lookup on xterm-kitty
> quirks-6.42 signed on 2022-12-09T12:59:38Z
> orbb# nmap
> Nmap 7.91 ( https://nmap.org )
> [...]
> ```
> 
> I can agree with the reporter that it would be better to have the
> ansible module "do the right thing" and set some lowest common
> denominator TERM instead of depending on the whims of a local
> environment. From what I can tell both "dumb" (as suggested by the
> reporter) and "unknown" seems to still print the progress bar:
> ```
> # TERM=unknown pkg_delete nmap
> nmap-7.91p2: ok
> Read shared items: ok
> ```
> 
> ... and:
> ```
> # TERM=dumb pkg_add nmap
> quirks-6.42 signed on 2022-12-09T12:59:38Z
> nmap-7.91p2: ok
> ```
> 
> Basically I am asking if any specific choice of a TERM would be
> preferred if we would make openbsd_pkg always set one of them for the
> different pkg_* tools.
> 
> Regards,
> Patrik Lundin
> 

Re: [patch 2/3] towards TERM=xterm

[Nicholas Marriott]

Reply inline.


On Sun, 8 Jan 2023 at 21:38, Crystal Kolipe 
wrote:

> On Sun, Jan 08, 2023 at 08:55:46PM +0000, Nicholas Marriott wrote:
> > This is still too big for one go so I think it needs split further. I
> > suggest that first we get three diffs with the very easy stuff:
> >
> > 1) The two bug fixes
> >
> > 2) HPA/VPA/INDN/RIN
>
> You didn't mention '3' :-)
>

Yes I saw that after I sent it, I changed my mind :-).


> But anyway, that's basically the first diff I sent and the two one-liners.
>
> I've attached that to the end of this mail.
>
> However, I think it's counter-productive to put just this in without the
> changes to the function keys, because as soon as people start testing the
> patch and setting TERM=xterm, then the first thing they are going to
> complain
> about is that F1 - F4 don't work.
>

Your full diff is 400 lines and makes 5 or 6 different changes in four
files, it cannot go in in one go.


>
> > Some other comments on the larger diff:
> >
> > - Why do you change RS_SCROLLBACK_SCREENS?
>
> To more closely mirror xterm behaviour.  The current scrollback buffer is a
> tiny fraction of what xterm offers by default.  It's not that important,
> but
> if we're telling people that the console is now like xterm, it's better
> that
> it really is.
>
>
This should be a separate change also.


> > - Wouldn't it be better to make underline match rather than moving bits
> > about at runtime? Is that not possible?
>
> I'm not sure what you mean here.
>
> The original underlining code is unchanged by my diff.  I only added double
> underline and strikethough.
>

> Are you talking about the rasops code that actually plots the extra
> underline
> pixels?  That's done in the same way as the existing code, so I'm not sure
> what you would prefer done differently?
>
> Or is it the part in wscons where the underline flag is moved to bit 0?
> That
> is a nasty hack that is in the original code.  I could take it out, but
> that
> would mean changing all of the bpp specific rasops code and removing the
> magic
> number 1 and replacing it with WSATTR_UNDERLINE.
>

Yes. WSATTR_UNDERLINE is 8 and you move it to bit 0 in rasops_pack_cattr,
so can it not be swapped with WSATTR_REVERSE so it is not necessary to move
it? You are changing rasops to use WSATTR flags in several places, so it
would be nice if it could use WSATTR_UNDERLINE directly instead of 1.

Obviously this should be done as a separate change also and it can wait
until later, there is enough here already.


>
> That sounds like a lot of churn that could potentially introduce new bugs
> and
> would require a lot of testing.  I prefer to leave the original hack in,
> because the rasops code has seen little change for many years, and with the
> features that I'm adding I don't think it will need to see much more any
> time
> soon.
>
> > - It seems odd that invisible will still show some elements like the
> > underline. I think unless we can support it fully it would be better not
> to
> > support it at all.
>
> This is the correct behaviour.  Xterm does the same thing.  Invisible
> characters retain underline, double underline, and strikethrough.
>

Hah, you are right. The terminals I tried do not do this but xterm does.

Invisible is useless in any case but yes if it is supported it should be
the same as xterm.


> It would actually be easier to implement invisible without the extra
> elements,
> but I put in the extra effort to mimick xterm behaviour :).
>
> Oh, by the way, I tested my idea of creating a bold font by over-printing
> the
> glyph one pixel to the right.  It looked good, and if we used this approach
> instead of setting a brighter colour for bold, then we could support bold
> in
> conjunction with all 256 colours instead of just the first eight.
>
> --- dev/wscons/wsemul_vt100_subr.c.orig Mon May 25 06:55:49 2020
> +++ dev/wscons/wsemul_vt100_subr.c  Sun Jan  8 18:28:04 2023
> @@ -231,7 +231,7 @@
> switch (A3(edp->modif1, edp->modif2, c)) {
> case A3('>', '\0', 'c'): /* DA secondary */
> wsdisplay_emulinput(edp->cbcookie, WSEMUL_VT_ID2,
> -   sizeof(WSEMUL_VT_ID2));
> +   sizeof(WSEMUL_VT_ID2)-1);
> break;
>
> case A3('\0', '\0', 'J'): /* ED selective erase in display */
> @@ -461,6 +461,9 @@
> edp->ccol -= min(DEF1_ARG(0), edp->ccol);
> edp->flags &= ~VTFL_LASTCHAR;
> break;
> +   case 'G': /* HPA */
> +   edp->ccol = min(DEF1_ARG(0)-1, edp->ncols-1);
> +   break;
> case '

Re: [patch 2/3] towards TERM=xterm

[Nicholas Marriott]

This is still too big for one go so I think it needs split further. I
suggest that first we get three diffs with the very easy stuff:

1) The two bug fixes

2) HPA/VPA/INDN/RIN

Some other comments on the larger diff:

- Why do you change RS_SCROLLBACK_SCREENS?

- Wouldn't it be better to make underline match rather than moving bits
about at runtime? Is that not possible?

- It seems odd that invisible will still show some elements like the
underline. I think unless we can support it fully it would be better not to
support it at all.



On Sat, 7 Jan 2023, 17:42 Crystal Kolipe, 
wrote:

> # New escape sequences and two one-line bugfixes
>
> --- dev/wscons/wsemul_vt100_subr.c.dist Mon May 25 06:55:49 2020
> +++ dev/wscons/wsemul_vt100_subr.c  Sat Jan  7 12:39:58 2023
> @@ -231,7 +231,7 @@
> switch (A3(edp->modif1, edp->modif2, c)) {
> case A3('>', '\0', 'c'): /* DA secondary */
> wsdisplay_emulinput(edp->cbcookie, WSEMUL_VT_ID2,
> -   sizeof(WSEMUL_VT_ID2));
> +   sizeof(WSEMUL_VT_ID2)-1);
> break;
>
> case A3('\0', '\0', 'J'): /* ED selective erase in display */
> @@ -461,6 +461,9 @@
> edp->ccol -= min(DEF1_ARG(0), edp->ccol);
> edp->flags &= ~VTFL_LASTCHAR;
> break;
> +   case 'G': /* HPA */
> +   edp->ccol = min(DEF1_ARG(0)-1, edp->ncols-1);
> +   break;
> case 'H': /* CUP */
> case 'f': /* HVP */
> if (edp->flags & VTFL_DECOM)
> @@ -502,16 +505,37 @@
> WSEMULOP(rc, edp, >abortstate, erasecols,
> ERASECOLS(NCOLS - n, n, edp->bkgdattr));
> break;
> +   case 'S': /* INDN */
> +   wsemul_vt100_scrollup (edp,DEF1_ARG(0));
> +   break;
> +   case 'T': /* RIN */
> +   wsemul_vt100_scrolldown (edp,DEF1_ARG(0));
> +   break;
> case 'X': /* ECH erase character */
> n = min(DEF1_ARG(0), COLS_LEFT + 1);
> WSEMULOP(rc, edp, >abortstate, erasecols,
> ERASECOLS(edp->ccol, n, edp->bkgdattr));
> break;
> +   case 'Z': /* KCBT */
> +   if (!edp->ccol)
> +   break;
> +   if (edp->tabs) {
> +   for (n=edp->ccol-1 ; n>0; n--)
> +   if (edp->tabs[n])
> +   break;
> +   } else {
> +   n=((edp->ccol - 1) & 8);
> +   }
> +   edp->ccol=n;
> +   break;
> case 'c': /* DA primary */
> if (ARG(0) == 0)
> wsdisplay_emulinput(edp->cbcookie, WSEMUL_VT_ID1,
> -   sizeof(WSEMUL_VT_ID1));
> +   sizeof(WSEMUL_VT_ID1)-1);
> break;
> +   case 'd': /* VPA */
> +   edp->crow = min(DEF1_ARG(0)-1, edp->nrows-1);
> +   break;
> case 'g': /* TBC */
> if (edp->tabs != NULL)
> switch (ARG(0)) {
> @@ -549,6 +573,12 @@
> case 1: /* bold */
> flags |= WSATTR_HILIT;
> break;
> +   case 2: /* dim */
> +   flags |= WSATTR_DIM;
> +   break;
> +   case 3: /* italic */
> +   flags |= WSATTR_ITALIC;
> +   break;
> case 4: /* underline */
> flags |= WSATTR_UNDERLINE;
> break;
> @@ -558,11 +588,31 @@
> case 7: /* reverse */
> flags |= WSATTR_REVERSE;
> break;
> -   case 22: /* ~bold VT300 only */
> +   /*
> +* Invisible text only makes the _glyph_ invisible.
> +*
> +* Other active attributes such as underlining and
> +* strikeout are still displayed in the character
> cell.
> +*/
> +   case 8: /* invisible */
> +   flags |= WSATTR_INVISIBLE;
> +   break;
> +   case 9: /* strike */
> +   flags |= WSATTR_STRIKE;
> +   break;
> +   case 21: /* double underline */
> +   flags |= WSATTR_DOUBLE_UNDERLINE;
> +   break;
> +   case 22: /* ~bold ~dim VT300 only */
> flags &= ~WSATTR_HILIT;
> +   flags &= ~WSATTR_DIM;
> 

Re: Making wscons a bit more like xterm

[Nicholas Marriott]

Some programs will always ignore terminfo and make their own guesses,
that's just the nature of terminfo, not much to be done about it.

I meant ignore bold in combination with 256 colours, not ignore it
entirely. But if you can come up with some scheme to display it then go for
it.

If possible can you now give me your latest patches without 256 colours and
I will test them and see if we can get them in. Probably best to split them
into three: new escape sequences, additional key sequences, and bug fixes
to existing code.



On Sat, 7 Jan 2023, 12:05 Crystal Kolipe, 
wrote:

> On Sat, Jan 07, 2023 at 11:09:00AM +0000, Nicholas Marriott wrote:
> > As far as SGR 38/48 with anything except 2 and 5 goes - the only
> terminals
> > I have at hand do not discard but I think either of discarding or not
> > discarding would be reasonable. What does xterm do here? It would
> probably
> > be best to do the same.
>
> Xterm only ignores the unknown argument immediately following the 38/48,
> but
> interprets the rest.  So SGR 38;9;1m sets bold.
>
> I've changed my code to do the same.
>
> > On terminals with fewer than 256 colours, I think the sequence should
> > probably do nothing.
>
> I've seen one program that uses it when colours <= 8, but that seems to
> have
> been an oversight.
>
> > I would be inclined to just ignore bold together with 256 colours (that
> is,
> > don't make bold change anything) as the terminal only supports bright and
> > not true bold.
>
> Hummm.  That would break existing behaviour in a few cases.
>
> Although most users are not seeing colour at all on the console, anyone who
> has set TERM=pccon currently has working bold in programs that parse
> terminfo.
>
> Also there are obviously programs that just assume ansi sequences are
> recognised.  Bold is such an old and basic attribute that it's pretty
> reasonable to assume that it is implemented somehow on any modern terminal
> that implements any ansi sequences.
>
> > Making an alternative colour map which is brighter would be
> > possible, although I haven't seen any terminals that do that.
>
> That's interesting.  Although it would be nicer to have a real bold font.
> Maybe just printing the exising font twice, offset one pixel to the right
> would work for that?
>
> Regarding 256 colour support in general, this is very much a late addition
> to
> the whole project of making the console usable with the xterm terminfo
> entry.
>
> I'm quite enthusiastic about adding it now, but if it's going to make the
> original goal of s/vt220/xterm in /etc/ttys less attractive because it
> causes
> breakage in obscure corner cases, then I can always drop the 256 colour
> part.
>
> Although to be honest, I think that the rest of the patchset without the
> 256 colour support is already very close to being usable.  Even as it is,
> it's
> an overall improvement because it fixes several exising bugs.
>
> In any case, I'll post an updated version with 256 in an hour or so.
>

Re: Making wscons a bit more like xterm

[Nicholas Marriott]

On terminals with fewer than 256 colours, I think the sequence should
probably do nothing.

I would be inclined to just ignore bold together with 256 colours (that is,
don't make bold change anything) as the terminal only supports bright and
not true bold. Making an alternative colour map which is brighter would be
possible, although I haven't seen any terminals that do that.


On Fri, 6 Jan 2023, 18:42 Crystal Kolipe, 
wrote:

> On Fri, Jan 06, 2023 at 06:21:41PM +0000, Nicholas Marriott wrote:
> > You should strictly only discard the following two arguments or  stuff
> like
> > SGR 38;5;100;1 won't work.
>
> Well I was initially in two minds about that, because I thought there
> might be
> other un-official extensions that used 38 with a varying number of
> parameters.
>
> But I haven't found any apart from 5 followed by one argument, and 2
> followed
> by three RGB values, so yes I agree with you on this.
>
> However, since this morning I've been working on actually implementing 256
> colour support so the next version of the patch will parse SGR 38;5
> sequences
> correctly.
>
> Presumably though, if we see SGR 38;X where X is neither 2 nor 5, we still
> have to discard everything, right?  Because that would be an invalid
> sequence
> and we don't know for sure whether anything that comes after it is actually
> intended as regular control codes or parameters for something else.
>
> I.E.
>
> SGR 38;5;X - we set colour X.
> SGR 38;2;X;X;X - we discard three parameters, then continue parsing.
> SGR 38;Y;  - where Y is 'something else', we ignore everything until the
> 'm'.
>
> Or is there a better way?
>
> Regarding 256 colour support, I've got it working, (on 32bpp displays
> only),
> but there are two open questions:
>
> 1. On displays that don't support 256 colours, (e.g. VGA console), what
> should
>SGR 38;5;X do?  I currently have it set so that it selects colours 0-7,
> and
>for 8-255 just does nothing, (leaves the current colour).
>
> 2. We implement bold as brighter text rather than a bold font.  This
> doesn't
>work very well with a 256 colour palette.  We can't just +8 to get a
> bright
>version of the colour.  How should bold work with arbitrary colours?
>
> > This is why some people prefer the : form but the ship has rather sailed
> on
> > that.
>
> I'm assuming that we don't really need to implement the : form?
>
> Thanks again for the feedback.
>

Re: Making wscons a bit more like xterm

[Nicholas Marriott]

IIRC xterm-direct uses the : form for RGB colour but xterm-256color does
not use it for 256 colours, so for 256 colours you can no doubt get away
without it.

As far as SGR 38/48 with anything except 2 and 5 goes - the only terminals
I have at hand do not discard but I think either of discarding or not
discarding would be reasonable. What does xterm do here? It would probably
be best to do the same.



On Fri, 6 Jan 2023, 18:42 Crystal Kolipe, 
wrote:

> On Fri, Jan 06, 2023 at 06:21:41PM +0000, Nicholas Marriott wrote:
> > You should strictly only discard the following two arguments or  stuff
> like
> > SGR 38;5;100;1 won't work.
>
> Well I was initially in two minds about that, because I thought there
> might be
> other un-official extensions that used 38 with a varying number of
> parameters.
>
> But I haven't found any apart from 5 followed by one argument, and 2
> followed
> by three RGB values, so yes I agree with you on this.
>
> However, since this morning I've been working on actually implementing 256
> colour support so the next version of the patch will parse SGR 38;5
> sequences
> correctly.
>
> Presumably though, if we see SGR 38;X where X is neither 2 nor 5, we still
> have to discard everything, right?  Because that would be an invalid
> sequence
> and we don't know for sure whether anything that comes after it is actually
> intended as regular control codes or parameters for something else.
>
> I.E.
>
> SGR 38;5;X - we set colour X.
> SGR 38;2;X;X;X - we discard three parameters, then continue parsing.
> SGR 38;Y;  - where Y is 'something else', we ignore everything until the
> 'm'.
>
> Or is there a better way?
>
> Regarding 256 colour support, I've got it working, (on 32bpp displays
> only),
> but there are two open questions:
>
> 1. On displays that don't support 256 colours, (e.g. VGA console), what
> should
>SGR 38;5;X do?  I currently have it set so that it selects colours 0-7,
> and
>for 8-255 just does nothing, (leaves the current colour).
>
> 2. We implement bold as brighter text rather than a bold font.  This
> doesn't
>work very well with a 256 colour palette.  We can't just +8 to get a
> bright
>version of the colour.  How should bold work with arbitrary colours?
>
> > This is why some people prefer the : form but the ship has rather sailed
> on
> > that.
>
> I'm assuming that we don't really need to implement the : form?
>
> Thanks again for the feedback.
>

Re: Making wscons a bit more like xterm

[Nicholas Marriott]

You should strictly only discard the following two arguments or  stuff like
SGR 38;5;100;1 won't work.

This is why some people prefer the : form but the ship has rather sailed on
that.


On Fri, 6 Jan 2023, 11:52 Crystal Kolipe, 
wrote:

> This version of the patchset updates a few things since the one posted
> early
> on Wednesday:
>
> NEW - Fix two off-by-ones that caused spurious trailing null characters
> when
>   requesting the terminal IDs.  (Noticed by Paul running tmux.)
>
> NEW - Discard any parameters passed to a CSI 38 m or CSI 48 m sequence.
>   See below for more discussion on this.
>
> NEW - Tweaked dim attribute to work correctly on byte-swapped framebuffers.
>
> * Control sequences for dim text, invisible text, strike through, italic,
> and
>   double underline attributes are now recognised and set flags in wscons.
> * Rendering of dim, invisible, struck, and double underlined text is now
>   supported in rasops32.c, so users of 32bpp displays will see these text
>   effects on the console.
> * The default number of scrollback screens has been increased from five to
>   twenty.
> * F1 - F4 and F13 - F24 now send different sequences.
> * With numlock OFF, keypad 5 is now 'begin' rather than unused.
> * Home, End, keypad home, and keypad end now send different sequences.
> * A new keysym has been added - KS_Backtab.
> * Shift-TAB is now defined as KS_Backtab and sends ESC [ Z.
> * Shift-F1 - Shift-F12 now send F13 - F24.
> * Five new escape sequences added for cursor movement.
>
> Off by ones:
>
> These can actually be triggered with the existing console code, without
> even
> applying any of my patches, and stem from using sizeof with a string
> constant,
> which returns the length of the string _including_ the terminating null.
>
> CSI 38 m and CSI 48 m sequences:
>
> The 30-37 codes set one of eight fixed foreground colours, the 40-47 codes
> set
> the background in the same way.  Codes 39 and 49 reset to the default
> colours.
>
> Codes 38 and 48 have become used as an escape to a larger colour space,
> typically 256 colours or 24-bit colour.
>
> A typical invocation to select colour 100 from a 256 colour palette might
> be:
>
> CSI 38;5;100m
>
> Such sequences are not defined in the terminfo entry for xterm.  However,
> some
> programs use them, assuming that they are supported.
>
> The current wscons code ignores the unimplemented 38, but then proceeds to
> interpret 5 and 100 as further SGR codes.
>
> In many cases, this doesn't much of a problem, because although 5 enables
> blink, none of the rasops code supports blink anyway.  However, the last
> value
> can potentially set any SGR attribute.  So, for example:
>
> CSI 38;5;4m
>
> ...will not select colour 4 as the program is expecting, but instead turn
> on
> underlining.
>
> To avoid this, the lastest version of my patch simply discards all values
> after a 38 or 48, until the trailing 'm'.
>
> This works to avoid setting unwanted attributes, until such time as 256
> colour
> support is added, (if this is even desired).
>
> --- dev/wscons/wsemul_vt100_keys.c.dist Sat Mar 14 00:38:50 2015
> +++ dev/wscons/wsemul_vt100_keys.c  Mon Jan  2 16:01:42 2023
> @@ -37,11 +37,9 @@
>  #include 
>  #include 
>
> +#define vt100_fkeys_len(x) (5+(x>=8)+(x>=12))
> +
>  static const u_char *vt100_fkeys[] = {
> -   "\033[11~", /* F1 */
> -   "\033[12~",
> -   "\033[13~", /* F1-F5 normally don't send codes */
> -   "\033[14~",
> "\033[15~", /* F5 */
> "\033[17~", /* F6 */
> "\033[18~",
> @@ -50,18 +48,18 @@
> "\033[21~",
> "\033[23~", /* VT100: ESC */
> "\033[24~", /* VT100: BS */
> -   "\033[25~", /* VT100: LF */
> -   "\033[26~",
> -   "\033[28~", /* help */
> -   "\033[29~", /* do */
> -   "\033[31~",
> -   "\033[32~",
> -   "\033[33~",
> -   "\033[34~", /* F20 */
> -   "\033[35~",
> -   "\033[36~",
> -   "\033[37~",
> -   "\033[38~"
> +   "\033[1;2P",/* VT100: LF */
> +   "\033[1;2Q",
> +   "\033[1;2R",/* help */
> +   "\033[1;2S",/* do */
> +   "\033[15;2~",
> +   "\033[17;2~",
> +   "\033[18;2~",
> +   "\033[19;2~",   /* F20 */
> +   "\033[20;2~",
> +   "\033[21;2~",
> +   "\033[23;2~",
> +   "\033[24;2~"
>  };
>
>  static const u_char *vt100_pfkeys[] = {
> @@ -96,14 +94,22 @@
> edp->translatebuf, edp->flags & VTFL_UTF8));
> }
>
> -   if (in >= KS_f1 && in <= KS_f24) {
> -   *out = vt100_fkeys[in - KS_f1];
> -   return (5);
> +   if (in >= KS_f1 && in <= KS_f4) {
> +   *out = vt100_pfkeys[in - KS_f1];
> +   return (3);
> }
> -   if (in >= KS_F1 && in <= KS_F24) {
> -   *out = vt100_fkeys[in - KS_F1];
> -   return (5);
> +   if (in >= KS_F1 && in <= KS_F4) {
> +   *out = 

Re: Making wscons a bit more like xterm

[Nicholas Marriott]

Hi

It is a good idea to make wscons more like xterm, at least so far as
setting TERM to xterm will not cause major problems for most programs.

Hardly anything uses blinking and the lack of it will bother nobody so TBH
I wouldn't worry too much about it.

Your diff looks good to me, although it would be nice to label the cases
with the escape sequence names like most of the existing ones, eg VPA, HPA,
SD, SU, CBT. I am not able to test it however.

It looks like SD and SU are VT420 sequences, but we should definitely be
careful about ever promising to be VT420 because it has a large variety of
weird and wonderful stuff, much of it useless.

https://invisible-island.net/xterm/ctlseqs/ctlseqs.html and
https://vt100.net/ are useful pages here, although you have probably seen
them already.


On Sat, 31 Dec 2022, 23:37 Crystal Kolipe, 
wrote:

> The following patch adds five escape sequences to the wscons vt100
> emulation.
>
> It's one part of a larger set of patches I am working on to make the
> console
> more like xterm.
>
>Note: Casual readers of -tech and those not familar with terminfo might
>prefer to read my write-up at
> https://research.exoticsilicon.com/articles/terminal_emulations
>
> Why do we want this?
>
> Currently the console emulates a superset of a subset of vt220 features.
>
> In other words, it supports most common vt220 sequences and a few
> additional
> sequences to do things like set colours, (which a real vt220 doesn't
> support).
>
> The problem is that since the terminal emulation has to default to
> something
> that is universally recognised, it's set by default to vt220, (in
> /etc/ttys).
>
> This means that none of these 'extra' features are available to
> curses-based
> programs running on the console, unless the user manually fiddles with the
> terminal type.
>
> The terminfo file has long included 'pccon', which claims to support the
> OpenBSD console, but this entry has numerous issues such as:
>
> * Simultaneous use of colour and underline is disabled, even though the
>   framebuffer console supports it, (presumably because the VGA console does
>   not).
>
> * The op entry is hard coded to set black text on a white background
> instead
>   of resetting the terminal to it's default colours. This causes, for
> example,
>   the mutt email client to exit with the terminal inverted, (black text on
> a
>   white background).
>
> * It doesn't include support for civis or cnorm, (to hide/restore the
> cursor),
>   even though both the framebuffer console and the VGA console support it.
>
> Even using the vt220 entry as is currently done causes some issues beyond
> lack
> of colour:
>
> * F1-F4 are not recognised as they send different sequences to what the
> vt220
>   entry expects.
>
> * Also doesn't include civis or cnorm.
>
> Fixing up the pccon entry is not feasible, as it would take many years for
> an
> updated version to appear on a sufficient number of systems for it to be
> suitable as a default in /etc/ttys.
>
> The solution:
>
> If wscons can be made to work adequately with the existing xterm entry in
> the
> terminfo file, then xterm could eventually become the default terminal
> emulation.
>
> In 2023, the xterm emulation should be almost as universally accepted as
> vt220, and allows for many more features such as colour.
>
> Currently, setting TERM=xterm in the console and starting any curses-based
> program will quickly show a lot of breakage.
>
> Comparing the terminfo entries for vt220 and xterm, the most obvious
> functions
> that are missing from the vt220 entry and which are not currently
> supported by
> wscons are:
>
> Sequence  Equivalent to termcap  Description
>
> CSI x T   rinscroll back x lines
> CSI x S   indn   scroll forward x lines
> CSI x d   vpago to line x
> CSI x G   hpago to column x
> CSI Z cbtreverse horizontal tab
>
> (Note: indices are 1-based and not 0-based.)
>
> These are the five functions implemented by the attached patch.
>
> Already, just enabling these functions makes a lot of curses programs run
> fine
> on the console - in colour - with TERM set to xterm.
>
> This patch is very unintrusive, and it's hard to see how it could break
> anything.  It also allows for more widespread testing with TERM=xterm to
> find
> out what works and what doesn't, and any odd corner cases.
>
> Beyond this, things become slightly more complicated for several reasons:
>
> 1. Xterm implements some features such as dim text, strikethrough, double
>underline, and italics, which wscons doesn't yet support.  I have code
> to
>implement the first three in rasops32, but didn't make that part of this
>diff to keep things simple.
>
> 2. Xterm implements blinking text, which would require a lot of changes to
>implement on the framebuffer console.  Should blinking text be
> represented
>

Re: Patch: ksh: fix input handling for 4 byte UTF-8 sequences

[Nicholas Marriott]

Looks good to me, ok nicm



On Wed, Jun 02, 2021 at 09:00:16PM +0200, Ingo Schwarze wrote:
> Hi,
> 
> feeling hesitant to commit into ksh without at least one proper OK,
> i'm resending this patch here, sorry if i missed private feedback.
> 
> What the existing code does:
> It tries to make sure that multi-byte UTF-8 characters get passed on by
> the shell without fragmentation, not one byte at time.  I heard people
> say that some software, for example tmux(1), may sometimes get confused
> when receiving a UTF-8 character in a piecemeal manner.
> 
> Which problem needs fixing:
> Of the four-byte UTF-8 sequences, only a subset is identified by the
> existing code.  The other four-byte UTF-8 sequences still get chopped
> up resulting in individual bytes being passed on.
> 
> 
> I'm also adding a few comments as suggested by jca@.  Parsing of UTF-8
> is less trivial than one might think, witnessed once again by the fact
> that i got this code wrong in the first place.
> 
> I also changed "cc & 0x20" to "cc > 0x9f" and "cc & 0x30" to "cc > 0x8f"
> for uniformity and readabilty - UTF-8-parsing is bad enough without
> needless micro-optimization, right?
> 
> 
> Note that even with the patch below, moving backward and forward
> over a blowfish icon on the command line still does not work because
> the character is width 2 and the ksh code intentionally does not
> use wcwidth(3).  But maybe it improves something in tmux?  Not sure.
> 
> Either way, unless it causes regressions, this (or a further improved
> version) should go in because what is there is clearly wrong.
> 
> OK?
>   Ingo
> 
> 
> Index: emacs.c
> ===
> RCS file: /cvs/src/bin/ksh/emacs.c,v
> retrieving revision 1.87
> diff -u -p -r1.87 emacs.c
> --- emacs.c   8 May 2020 14:30:42 -   1.87
> +++ emacs.c   13 May 2021 18:16:13 -
> @@ -1851,11 +1851,17 @@ x_e_getu8(char *buf, int off)
>   return -1;
>   buf[off++] = c;
>  
> - if (c == 0xf4)
> + /*
> +  * In the following, comments refer to violations of
> +  * the inequality tests at the ends of the lines.
> +  * See the utf8(7) manual page for details.
> +  */
> +
> + if ((c & 0xf8) == 0xf0 && c < 0xf5)  /* beyond Unicode */
>   len = 4;
>   else if ((c & 0xf0) == 0xe0)
>   len = 3;
> - else if ((c & 0xe0) == 0xc0 && c > 0xc1)
> + else if ((c & 0xe0) == 0xc0 && c > 0xc1)  /* use single byte */
>   len = 2;
>   else
>   len = 1;
> @@ -1865,9 +1871,10 @@ x_e_getu8(char *buf, int off)
>   if (cc == -1)
>   break;
>   if (isu8cont(cc) == 0 ||
> - (c == 0xe0 && len == 3 && cc < 0xa0) ||
> - (c == 0xed && len == 3 && cc & 0x20) ||
> - (c == 0xf4 && len == 4 && cc & 0x30)) {
> + (c == 0xe0 && len == 3 && cc < 0xa0) ||  /* use 2 bytes */
> + (c == 0xed && len == 3 && cc > 0x9f) ||  /* surrogates  */
> + (c == 0xf0 && len == 4 && cc < 0x90) ||  /* use 3 bytes */
> + (c == 0xf4 && len == 4 && cc > 0x8f)) {  /* beyond Uni. */
>   x_e_ungetc(cc);
>   break;
>   }

Re: adjfreq(2): limit adjustment to prevent overflow during tc_windup()

[Nicholas Marriott]

10^12 was the old definition of billion in the UK also, although in the
last few decades it has become rare and 10^9 is now the norm.

https://en.wikipedia.org/wiki/Long_and_short_scales has quite a bit about
it.


On Tue, 7 Jul 2020 at 00:27, Scott Cheloha  wrote:

> On Mon, Jul 06, 2020 at 11:57:32PM +0200, Mark Kettenis wrote:
> > > Date: Mon, 6 Jul 2020 16:40:35 -0500
> > > From: Scott Cheloha 
> > >
> > > On Fri, Jul 03, 2020 at 10:52:15AM +0200, Otto Moerbeek wrote:
> > > > On Thu, Jul 02, 2020 at 08:27:58PM -0500, Scott Cheloha wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > When we recompute the scaling factor during tc_windup() there is an
> > > > > opportunity for arithmetic overflow/underflow when we add the NTP
> > > > > adjustment into the scale:
> > > > >
> > > > >649  scale = (u_int64_t)1 << 63;
> > > > >650  scale += \
> > > > >651  ((th->th_adjustment +
> th->th_counter->tc_freq_adj) / 1024) * 2199;
> > > > >652  scale /= th->th_counter->tc_frequency;
> > > > >653  th->th_scale = scale * 2;
> > > > >
> > > > > At lines 650 and 651, you will overflow/underflow if
> > > > > th->th_counter->tc_freq_adj is sufficiently positive/negative.
> > > > >
> > > > > I don't like the idea of checking for that overflow during
> > > > > tc_windup().  We can pick a reasonable adjustment range and check
> for
> > > > > it during adjfreq(2) and that should be good enough.
> > > > >
> > > > > My strawman proposal is a range of -5 to 5 parts
> per
> > > > > billion.  We could push the limits a bit, but half a billion seems
> > > > > like a nice round number to me.
> > > > >
> > > > > On a perfect clock, this means you can effect a 0.5x slowdown or a
> > > > > 1.5x speedup via adjfreq(2), but no slower/faster.
> > > > >
> > > > > I don't *think* ntpd(8) would ever reach such extreme adjustments
> > > > > through its algorithm.  I don't think this will break anyone's
> working
> > > > > setup.
> > > > >
> > > > > (Maybe I'm wrong, though.  otto@?)
> > > >
> > > > Right, ntpd is pretty conversative and won't do big adjustments.
> > >
> > > So, what is the right way to describe these limits?
> > >
> > > "Parts per billion"?  Something else?
> >
> > The traditional way to express clock drift in the context of NTP is
> > "parts per million" or "ppm" for short.  There are good reasons to
> > avoid using "billion" in documentation as a very similar word is used
> > in Germanic languages for 10^12 where in English you mean 10^9.
>
> Huh.  So from what I've gathered:
>
> German  English ISO
>
> Million Million 10^6
> Milliarde   Billion 10^9
> Billion Trillion10^12
> Billiarde   Quadrillion 10^15
> TrillionQuintillion 10^18
>
> Potentially confusing.
>
> The "German Connection" to NTP in particular eludes me, though.
>
> > Stripping three zeroes also makes it easier to read.  [...]
>
> Yes, it always does.  It looks nicer as "N ppm" in the mandoc(1)
> output, too.
>
> --
>
> otto@, from what you said I take it you're OK with the new limits so
> I'm going commit it as follows in a day or two.
>
> Index: lib/libc/sys/adjfreq.2
> ===
> RCS file: /cvs/src/lib/libc/sys/adjfreq.2,v
> retrieving revision 1.7
> diff -u -p -r1.7 adjfreq.2
> --- lib/libc/sys/adjfreq.2  10 Sep 2015 17:55:21 -  1.7
> +++ lib/libc/sys/adjfreq.2  6 Jul 2020 23:00:24 -
> @@ -60,6 +60,9 @@ The
>  .Fa freq
>  argument is non-null and the process's effective user ID is not that
>  of the superuser.
> +.It Bq Er EINVAL
> +.Fa freq
> +is less than -50 ppm or greater than 50 ppm.
>  .El
>  .Sh SEE ALSO
>  .Xr date 1 ,
> Index: sys/kern/kern_time.c
> ===
> RCS file: /cvs/src/sys/kern/kern_time.c,v
> retrieving revision 1.131
> diff -u -p -r1.131 kern_time.c
> --- sys/kern/kern_time.c22 Jun 2020 18:25:57 -  1.131
> +++ sys/kern/kern_time.c6 Jul 2020 23:00:24 -
> @@ -391,6 +391,9 @@ sys_settimeofday(struct proc *p, void *v
> return (0);
>  }
>
> +#define ADJFREQ_MAX (5LL << 32)
> +#define ADJFREQ_MIN (-5LL << 32)
> +
>  int
>  sys_adjfreq(struct proc *p, void *v, register_t *retval)
>  {
> @@ -408,6 +411,8 @@ sys_adjfreq(struct proc *p, void *v, reg
> return (error);
> if ((error = copyin(freq, , sizeof(f
> return (error);
> +   if (f < ADJFREQ_MIN || f > ADJFREQ_MAX)
> +   return (EINVAL);
> }
>
> rw_enter(_lock, (freq == NULL) ? RW_READ : RW_WRITE);
>
>

Re: top: Fill last character in process line

[Nicholas Marriott]

Actually I've got them the wrong way round here, but others have already
explained them anyway :-).



On Wed, Jun 03, 2020 at 05:04:43PM +0100, Nicholas Marriott wrote:
> xenl (xm) only matters for the last line - you can't write into the very
> bottom right position without causing the terminal to scroll. No xenl is
> a pain and most applications solve it by just leaving that position
> always empty. The only terminal still about without xenl that I am aware
> of is the NetBSD console.
> 
> am only matters if you care where the cursor is after you write to the
> last character - that is, if you want to rely on it moving to the next
> line automatically. You can still write at the rightmost cursor
> position, just don't rely on the cursor moving to the next line or the
> terminal scrolling when you do. I expect top doesn't need to worry about
> this.
> 
> You could probably just use the full width except for the last line and
> trim the last line by 1 if tgetflag("xm") is true.
> 
> Or doesn't ncurses take care of all this anyway? Maybe it doesn't...
> 
> 
> On Wed, Jun 03, 2020 at 02:49:48PM +0200, Klemens Nanni wrote:
> > On Wed, Jun 03, 2020 at 12:45:35PM +0100, Stuart Henderson wrote:
> > > It should check terminal capabilities for this, see termcap(5).
> > > If 'am' (auto-margin) is set then it shouldn't write to the final column.
> > > If 'xn' is set then it's OK in some circumstances (it's probably easier to
> > > skip writing to the final column if this is set too).
> > Thanks mark and Stuart, I did not know about auto-margin (or autoWrap as
> > xterm(1) seems to call it).
> > 
> > What I understand is that writing to the screen's last terminal should
> > be avoided in terminal without the "am" capability, presumably because
> > it would cause a line wrap - is that correct?
> > 
> > Preliminary testing however indicates to me that at least xterm(1)
> > behaves the same in top's interactive screen with my patch, regardless
> > of the auto-margin capablility.
> > 
> > According to termcap(5) I did the following to disable "am", with
> > tput(1) I verify that it gets indeed disabled:
> > 
> > $  echo $TERM ; tput am ; echo $?
> > xterm
> > 0
> > $ TERM=vt100-nam ; tput am ; echo $?
> > 1
> > 
> > But in both cases, starting ./obj/top in the very same terminal/shell
> > behaves the same, that is to say the last column is written properly and
> > I see no line wrap or any change of behaviour.
> > 

Re: top: Fill last character in process line

[Nicholas Marriott]

xenl (xm) only matters for the last line - you can't write into the very
bottom right position without causing the terminal to scroll. No xenl is
a pain and most applications solve it by just leaving that position
always empty. The only terminal still about without xenl that I am aware
of is the NetBSD console.

am only matters if you care where the cursor is after you write to the
last character - that is, if you want to rely on it moving to the next
line automatically. You can still write at the rightmost cursor
position, just don't rely on the cursor moving to the next line or the
terminal scrolling when you do. I expect top doesn't need to worry about
this.

You could probably just use the full width except for the last line and
trim the last line by 1 if tgetflag("xm") is true.

Or doesn't ncurses take care of all this anyway? Maybe it doesn't...


On Wed, Jun 03, 2020 at 02:49:48PM +0200, Klemens Nanni wrote:
> On Wed, Jun 03, 2020 at 12:45:35PM +0100, Stuart Henderson wrote:
> > It should check terminal capabilities for this, see termcap(5).
> > If 'am' (auto-margin) is set then it shouldn't write to the final column.
> > If 'xn' is set then it's OK in some circumstances (it's probably easier to
> > skip writing to the final column if this is set too).
> Thanks mark and Stuart, I did not know about auto-margin (or autoWrap as
> xterm(1) seems to call it).
> 
> What I understand is that writing to the screen's last terminal should
> be avoided in terminal without the "am" capability, presumably because
> it would cause a line wrap - is that correct?
> 
> Preliminary testing however indicates to me that at least xterm(1)
> behaves the same in top's interactive screen with my patch, regardless
> of the auto-margin capablility.
> 
> According to termcap(5) I did the following to disable "am", with
> tput(1) I verify that it gets indeed disabled:
> 
>   $  echo $TERM ; tput am ; echo $?
>   xterm
>   0
>   $ TERM=vt100-nam ; tput am ; echo $?
>   1
> 
> But in both cases, starting ./obj/top in the very same terminal/shell
> behaves the same, that is to say the last column is written properly and
> I see no line wrap or any change of behaviour.
> 

Re: Ncurses: issue with "rep" capability and --enable-bsdpad (BSD_TPUTS)

[Nicholas Marriott]

Hi

I have applied this now, thanks!

> Is there a specific reason why OpenBSD still uses an old ncurses-based
> version?

Updating ncurses in OpenBSD takes quite a lot of time because it is big
and quite integrated - so for example the file layout is where OpenBSD
would put stuff not where upstream does, there are bits in various
places (libcurses, libform, libpanel, libmenu, tic, infocmp, tset,
terminfo.src), quite a lot of local changes (many of them cosmetic), the
build process has a lot of scripts and fiddly stuff. Plus it is used by
a lot of programs both in base and ports, and kind of has to be done all
in one go.

But we could definitely do with an upgrade now... I've started it a
couple of times but never had time to finish. So if you or anyone wants
to try to do it, let me know :-).

Next time it would be nice to see if we could streamline and document it
the upgrade process so we don't end up so far behind so often.



On Mon, May 25, 2020 at 07:02:00PM +0200, Hiltjo Posthuma wrote:
> Hi,
> 
> I've noticed an issue when testing the "rep" capability to st (terminal
> emulator) on OpenBSD.
> 
> After some digging I found the issue is when BSD_TPUTS is defined
> (ncurses_cfg.h).  This enables code for BSD prefix padding and a workaround 
> for
> nethack and ancient BSD terminals as the C comment says in the file
> tinfo/lib_tputs.c.
> 
> I noticed while using the lynx browser, it will incorrectly print repeated
> digit characters. For example:
> 
>   echo "Z000" | lynx -stdin
> 
> will print: ""
> 
> 
> To reproduce it:
> 
> * Use a terminal with the "rep" capability supported and exposed:
>   rep=%p1%c\E[%p2%{1}%-%db
> 
>   infocmp | grep rep=
> 
>   The sequence is documented in terminfo(5).
> 
> * Compile ncurses with BSD_TPUTS defined (or ./configure --enable-bsdpad on 
> Linux).
>   BSD_TPUTS is set by default on OpenBSD in lib/libcurses/ncurses_cfg.h
> 
> I could reproduce the issue on Void Linux also on the latest ncurses snapshot,
> but a fix has been sent upstream and added in 20200523:
> 
> https://github.com/ThomasDickey/ncurses-snapshots/blob/master/NEWS
> 
> "+ add a check in EmitRange to guard against repeat_char emitting digits which
>   could be interpreted as BSD-style padding when --enable-bsdpad is configured
>   (report/patch by Hiltjo Posthuma)."
> 
> 
> A small program to reproduce it:
> 
> #include 
> #include 
> #include 
> 
> int
> main(void)
> {
>   WINDOW *win;
>   
>   win = initscr();
>   printw("Z000");
> 
>   refresh();
>   
>   sleep(5);
> 
>   return 0;
> }
> 
> This program will incorrectly print "" also.
> 
> The backported below patch from ncurses snapshot 20200523 to -current fixes 
> it:
> 
> 
> diff --git a/lib/libcurses/tty/tty_update.c b/lib/libcurses/tty/tty_update.c
> index bd2f088adf6..3d10b76cce2 100644
> --- a/lib/libcurses/tty/tty_update.c
> +++ b/lib/libcurses/tty/tty_update.c
> @@ -540,7 +540,11 @@ EmitRange(const NCURSES_CH_T * ntext, int num)
>   } else {
>   return 1;   /* cursor stays in the middle */
>   }
> - } else if (repeat_char && runcount > SP->_rep_cost) {
> + } else if (repeat_char &&
> +#if BSD_TPUTS
> +   !isdigit(UChar(CharOf(ntext0))) &&
> +#endif
> +   runcount > SP->_rep_cost) {
>   bool wrap_possible = (SP->_curscol + runcount >= 
> screen_columns);
>   int rep_count = runcount;
>  
> 
> Is there a specific reason why OpenBSD still uses an old ncurses-based 
> version?
> 
> -- 
> Kind regards,
> Hiltjo
> 

Re: new option -S for uptime(1)

[Nicholas Marriott]

You could do:

expr `date +%s` - `sysctl -n kern.boottime`




On Fri, 20 Mar 2020 at 12:54, Alex Naumov  wrote:
>
> Hi,
>
> this patch adds a new option to uptime(1) that allows to see the length of
> time the system has been up in seconds.
> Patch includes documentation (man-page) update.
>
> Example:
> $ uptime
>  1:26PM  up  1:28, 1 user, load averages: 0.00, 0.00, 0.00
> $ uptime -S
>  1:26PM  5308 secs, 1 user, load averages: 0.00, 0.00, 0.00
>
> It's tested in current on amd64 and aarch64.
>
> Why I need it? Well, it makes it easy to check the time the system has been
> up, for example, via scripts. It doesn't need to parse the output and
> recalculate it.
>
> Cheers,
> Alexander

Re: sys/cdefs.h: fix __predict_false fallback implementation

[Nicholas Marriott]

Looks like the existing code is OK, you still want to test the original
expression even if you are predicting it is false, no?



On Tue, Feb 11, 2020 at 07:33:19PM +0100, Jeremie Courreges-Anglas wrote:
> 
> Found while looking at __ISO_C_VISIBLE.  I'm not sure which compilers
> would be affected.  The fallback could simply be
>  #define __predict_true(exp)  (exp)
>  #define __predict_false(exp) (exp)
> but I settled for a minimal change.
> 
> ok?
> 
> 
> Index: cdefs.h
> ===
> RCS file: /d/cvs/src/sys/sys/cdefs.h,v
> retrieving revision 1.43
> diff -u -p -p -u -r1.43 cdefs.h
> --- cdefs.h   29 Oct 2018 17:10:40 -  1.43
> +++ cdefs.h   11 Feb 2020 18:27:18 -
> @@ -193,7 +193,7 @@
>  #define __predict_false(exp) __builtin_expect(((exp) != 0), 0)
>  #else
>  #define __predict_true(exp)  ((exp) != 0)
> -#define __predict_false(exp) ((exp) != 0)
> +#define __predict_false(exp) ((exp) == 0)
>  #endif
>  
>  /* Delete pseudo-keywords wherever they are not available or needed. */
> 
> 
> -- 
> jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE
> 

Re: pretty borders for slitherins

[Nicholas Marriott]

OK seems like it is always using ACS which is correct.

I just remembered jmc asked me about this before and it turned out that
ACS doesn't work with the DRM console - I don't think the font have the
right symbol for either ACS or UTF-8 line drawing, but there may be
other problems in there as well.

If you are using DRM you can set TERM to pccon0 which is a variant with
ASCII line drawing and ncurses will use ASCII for line drawing.




On Wed, Sep 25, 2019 at 05:02:17PM -0500, Scott Cheloha wrote:
> On Wed, Sep 25, 2019 at 08:29:47PM +0100, Nicholas Marriott wrote:
> > It will use either UTF-8 characters or the ACS characters from TERM.
> > 
> > The way it decides which is somewhat confusing and it looks like it is
> > choosing to use UTF-8 even when I wouldn't expect it to so I'm not sure
> > it is actually working correctly in our ncurses version.
> > 
> > I don't think UTF-8 line drawing works properly, at least with the DRM
> > wscons.
> > 
> > What do you have in TERM now? Can you see if the borders are OK with
> > "TERM=xterm snake"? This seems to use ACS rather than UTF-8. If it does
> > I'll see if I can figure out why it choosing UTF-8 borders for the
> > others.
> 
> In wscons I have vt220.  The borders are even more screwy with
> `TERM=xterm snake` in the wscons.  Now you have pairs of question
> marks instead of just a single '?'.
> 
> The (maybe not so) odd thing is that in a real xterm if I do
> `TERM=vt220 snake` the borders are fine.  So yeah, I don't know how
> our ncurses is deciding what to do.
> 
> > You can run it in script(1) to see what it is using for the border (lots
> > of 'q' or 'x' means ACS).
> 
> I see a string of 'q' in the typescript when in the xterm and TERM=xterm:
> 
> ESC[?1049hESC[1;24rESC(BESC[mESC[4lESC[?7hESC[?1hESC=ESC[HESC[2J Worm
> ESC(0ESC[0mlqkESC(B
> ESC(0ESC[0mxESC(BESC[79GESC(0ESC[0mxESC(B
> ESC(0ESC[0mxESC(BESC[79GESC(0ESC[0mxESC(B
> 
> in xterm when TERM=vt220:
> 
> ESC)0ESC[1;24rESC[mESC(BESC[4lESC[?7hESC[HESC[J Worm
> ESC[0mESC(0lqkESC(B
> ESC[0mESC(0xESC(BESC[77CESC[0mESC(0xESC(B
> ESC[0mESC(0xESC(BESC[77CESC[0mESC(0xESC(B
> 
> in wscons when TERM=vt220:
> 
> ESC)0ESC[1;45rESC[mESC(BESC[4lESC[?7hESC[HESC[J Worm
> ESC[0mESC(0lqkESC(B
> ESC[0mESC(0xESC(BESC[157CESC[0mESC(0xESC(B
> ESC[0mESC(0xESC(BESC[157CESC[0mESC(0xESC(B
> 
> in wscons when TERM=xterm:
> 
> ESC[?1049hESC[1;45rESC(BESC[mESC[4lESC[?7hESC[?1hESC=ESC[HESC[2J Worm
> ESC(0ESC[0mlqkESC(B
> ESC(0ESC[0mxESC(BESC[159GESC(0ESC[0mxESC(B
> ESC(0ESC[0mxESC(BESC[159GESC(0ESC[0mxESC(B
> 
> ... so it's doing something based on TERM.

Re: pretty borders for slitherins

[Nicholas Marriott]

It will use either UTF-8 characters or the ACS characters from TERM.

The way it decides which is somewhat confusing and it looks like it is
choosing to use UTF-8 even when I wouldn't expect it to so I'm not sure
it is actually working correctly in our ncurses version.

I don't think UTF-8 line drawing works properly, at least with the DRM
wscons.

What do you have in TERM now? Can you see if the borders are OK with
"TERM=xterm snake"? This seems to use ACS rather than UTF-8. If it does
I'll see if I can figure out why it choosing UTF-8 borders for the
others.

You can run it in script(1) to see what it is using for the border (lots
of 'q' or 'x' means ACS).


On Wed, Sep 25, 2019 at 11:15:01AM -0500, Scott Cheloha wrote:
> On Mon, Sep 23, 2019 at 06:23:32PM -0400, Ted Unangst wrote:
> > snake and worm draw boxes, but they can be prettier by using the default
> > style, which will use line drawing instead of ugly -*| characters.
> > 
> > should do the right thing on a vt100, but only tested in xterm.
> 
> It looks much prettier in an xterm but in the system console I'm
> getting question marks.  Is that a limitation of the terminal or
> am I missing the glyphs?  Dunno if this is the expected behavior
> but it is uglier than what we had before.
> 
> Also, you forgot to delete the prototype for drawbox().
> 
> > Index: snake/snake.c
> > ===
> > RCS file: /home/cvs/src/games/snake/snake.c,v
> > retrieving revision 1.34
> > diff -u -p -r1.34 snake.c
> > --- snake/snake.c   28 Jun 2019 13:32:52 -  1.34
> > +++ snake/snake.c   13 Sep 2019 17:05:19 -
> > @@ -450,23 +450,8 @@ setup(void)
> > pchar([i], SNAKETAIL);
> > }
> > pchar([0], SNAKEHEAD);
> > -   drawbox();
> > +   border(0, 0, 0, 0, 0, 0, 0, 0);
> > refresh();
> > -}
> > -
> > -void
> > -drawbox(void)
> > -{
> > -   int i;
> > -
> > -   for (i = 1; i <= ccnt; i++) {
> > -   mvaddch(0, i, '-');
> > -   mvaddch(lcnt + 1, i, '-');
> > -   }
> > -   for (i = 0; i <= lcnt + 1; i++) {
> > -   mvaddch(i, 0, '|');
> > -   mvaddch(i, ccnt + 1, '|');
> > -   }
> >  }
> >  
> >  void
> > Index: worm/worm.c
> > ===
> > RCS file: /home/cvs/src/games/worm/worm.c,v
> > retrieving revision 1.39
> > diff -u -p -r1.39 worm.c
> > --- worm/worm.c 24 Aug 2018 11:14:49 -  1.39
> > +++ worm/worm.c 13 Sep 2019 16:51:39 -
> > @@ -122,7 +122,7 @@ main(int argc, char **argv)
> > }
> > stw = newwin(1, COLS-1, 0, 0);
> > tv = newwin(LINES-1, COLS-1, 1, 0);
> > -   box(tv, '*', '*');
> > +   box(tv, 0, 0);
> > scrollok(tv, FALSE);
> > scrollok(stw, FALSE);
> > wmove(stw, 0, 0);
> > 
> 

Re: tmux: cannot select pane after prefix-b q

[Nicholas Marriott]

Hi

You have the right idea general idea of the problem. display-panes
blocks the queue until it is finished, so the key press isn't processed
until then, which is too late.

But your change defeats the purpose, the idea is that new key presses
should be queued after the commands inserted by previous key presses,
not before them.

Identify mode (display-panes) can just be treated specially I think.

Please try the diff below.

Thanks!


Index: server-client.c
===
RCS file: /cvs/src/usr.bin/tmux/server-client.c,v
retrieving revision 1.279
diff -u -p -r1.279 server-client.c
--- server-client.c 3 May 2019 20:44:24 -   1.279
+++ server-client.c 7 May 2019 10:50:07 -
@@ -986,7 +986,7 @@ server_client_assume_paste(struct sessio
  * Handle data key input from client. This owns and can modify the key event it
  * is given and is responsible for freeing it.
  */
-enum cmd_retval
+static enum cmd_retval
 server_client_key_callback(struct cmdq_item *item, void *data)
 {
struct client   *c = item->client;
@@ -1204,6 +1204,44 @@ forward_key:
 out:
free(event);
return (CMD_RETURN_NORMAL);
+}
+
+/* Handle a key event. */
+int
+server_client_handle_key(struct client *c, struct key_event *event)
+{
+   struct session  *s = c->session;
+   struct window   *w;
+   struct window_pane  *wp = NULL;
+   struct cmdq_item*item;
+
+   /* Check the client is good to accept input. */
+   if (s == NULL || (c->flags & (CLIENT_DEAD|CLIENT_SUSPENDED)) != 0)
+   return (0);
+   w = s->curw->window;
+
+   /*
+* Key presses in identify mode are a special case. The queue might be
+* blocked so they need to processed immediately rather than queued.
+*/
+   if (c->flags & CLIENT_IDENTIFY) {
+   if (c->flags & CLIENT_READONLY)
+   return (0);
+   if (event->key >= '0' && event->key <= '9') {
+   window_unzoom(w);
+   wp = window_pane_at_index(w, event->key - '0');
+   }
+   server_client_clear_identify(c, wp);
+   return (0);
+   }
+
+   /*
+* Add the key to the queue so it happens after any commands queued by
+* previous keys.
+*/
+   item = cmdq_get_callback(server_client_key_callback, event);
+   cmdq_append(c, item);
+   return (1);
 }
 
 /* Client functions that need to happen every loop. */
Index: tmux.h
===
RCS file: /cvs/src/usr.bin/tmux/tmux.h,v
retrieving revision 1.888
diff -u -p -r1.888 tmux.h
--- tmux.h  7 May 2019 10:25:15 -   1.888
+++ tmux.h  7 May 2019 10:50:09 -
@@ -2012,7 +2012,7 @@ void   server_client_set_identify(struct 
 voidserver_client_set_key_table(struct client *, const char *);
 const char *server_client_get_key_table(struct client *);
 int server_client_check_nested(struct client *);
-enum cmd_retval server_client_key_callback(struct cmdq_item *, void *);
+int server_client_handle_key(struct client *, struct key_event *);
 struct client *server_client_create(int);
 int server_client_open(struct client *, char **);
 voidserver_client_unref(struct client *);
Index: tty-keys.c
===
RCS file: /cvs/src/usr.bin/tmux/tty-keys.c,v
retrieving revision 1.112
diff -u -p -r1.112 tty-keys.c
--- tty-keys.c  3 May 2019 18:00:19 -   1.112
+++ tty-keys.c  7 May 2019 10:50:09 -
@@ -573,7 +573,6 @@ tty_keys_next(struct tty *tty)
cc_t bspace;
int  delay, expired = 0, n;
key_code key;
-   struct cmdq_item*item;
struct mouse_event   m = { 0 };
struct key_event*event;
 
@@ -732,9 +731,8 @@ complete_key:
event = xmalloc(sizeof *event);
event->key = key;
memcpy(>m, , sizeof event->m);
-
-   item = cmdq_get_callback(server_client_key_callback, event);
-   cmdq_append(c, item);
+   if (!server_client_handle_key(c, event))
+   free(event);
}
 
return (1);

Re: libevent: endless loop on excessively large buffers

[Nicholas Marriott]

ok nicm



On Thu, May 02, 2019 at 06:59:33PM +0200, Tobias Stöckmann wrote:
> It is possible to trigger an endless loop or out of boundary write
> on 64 bit systems with evbuffer_readline calls for buffers which
> exceed 4 GB (i.e. overflow uint).
> 
>   for (i = 0; i < len; i++)
> 
> Variable i is unsigned int and len size_t. This leads to an endless
> loop if len is larger than UINT_MAX.
> 
> If the loop ends exactly at UINT_MAX, then a malloc with additional
> space for an ending '\0' leads to an overflow, effectively allocating
> 0 bytes and therefore an out of boundary write occurs.
> 
> This is a proof of concept for an endless loop:
> 
> #include 
> #include 
> #include 
> #include 
> #include 
> 
> int
> main(void)
> {
>   char *buf;
>   const size_t len = 0x1UL;
>   struct evbuffer *buffer;
>   char *line;
> 
>   if ((buf = malloc(len)) == NULL)
>   err(1, "malloc");
>   memset(buf, 'A', len);
> 
>   if ((buffer = evbuffer_new()) == NULL)
>   errx(1, "evbuffer_new");
>   if (evbuffer_expand(buffer, len + 1))
>   errx(1, "evbuffer_expand");
>   evbuffer_add(buffer, buf, len);
>   evbuffer_add(buffer, "", 1);
> 
>   printf("%p\n", evbuffer_readline(buffer));
>   return 0;
> }
> 
> Generally this is a rather theoretical case. Normal users are not
> allowed to allocate so much memory. But better be safe than sorry,
> especially if login.conf values were adjusted (or the process runs
> as root).
> 
> This patch completely removes "unsigned int" from buffer.c.
> 
> 
> Index: buffer.c
> ===
> RCS file: /cvs/src/lib/libevent/buffer.c,v
> retrieving revision 1.31
> diff -u -p -u -p -r1.31 buffer.c
> --- buffer.c  18 Mar 2017 01:48:43 -  1.31
> +++ buffer.c  1 May 2019 11:00:29 -
> @@ -188,7 +188,7 @@ evbuffer_readline(struct evbuffer *buffe
>   u_char *data = EVBUFFER_DATA(buffer);
>   size_t len = EVBUFFER_LENGTH(buffer);
>   char *line;
> - unsigned int i;
> + size_t i;
> 
>   for (i = 0; i < len; i++) {
>   if (data[i] == '\r' || data[i] == '\n')
> @@ -232,7 +232,7 @@ evbuffer_readln(struct evbuffer *buffer,
>   u_char *start_of_eol, *end_of_eol;
>   size_t len = EVBUFFER_LENGTH(buffer);
>   char *line;
> - unsigned int i, n_to_copy, n_to_drain;
> + size_t i, n_to_copy, n_to_drain;
> 
>   if (n_read_out)
>   *n_read_out = 0;
> 

Re: libevent: remove non-monotonic compat code

[Nicholas Marriott]

Looks good to me.



On Tue, 30 Apr 2019, 19:32 Jeremie Courreges-Anglas,  wrote:

> On Tue, Apr 30 2019, Nicholas Marriott 
> wrote:
> > Oh, event_err typically uses __func__ so I would do that here too.
>
> Indeed my proposal doesn't match the pattern used here.
> Does this look better?
>
> if (clock_gettime(CLOCK_MONOTONIC, ) == -1)
> event_err(1, "%s: clock_gettime", __func__);
>
> --
> jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE
>

Re: libevent: remove non-monotonic compat code

[Nicholas Marriott]

Oh, event_err typically uses __func__ so I would do that here too.


On Tue, Apr 30, 2019 at 06:48:13PM +0100, Nicholas Marriott wrote:
> ok for both
> 
> 
> On Tue, Apr 30, 2019 at 07:13:55PM +0200, Jeremie Courreges-Anglas wrote:
> > On Tue, Apr 30 2019, Jeremie Courreges-Anglas  wrote:
> > > libevent contains a fallback path in case clock_gettime(CLOCK_MONOTONIC)
> > > fails.  The fallback path tries to cope with time going backwards and
> > > reaches into the timeheap internals, as noticed by Tobias in
> > >
> > >   https://marc.info/?l=openbsd-tech=155595247719664=2
> > >
> > > I doubt that we care about this kind of seldom-tested compat code, which
> > > was probably useful to get portable libevent to run on various
> > > (ancient?) systems.
> > >
> > > Our clock_gettime can only fail for two reasons:
> > > - invalid clock id.  Lots of our userland uses CLOCK_MONOTONIC now,
> > >   I think it can be considered a requirement.
> > > - invalid timespec pointer. Here, since we pass the address of
> > >   a timespec structure on the stack something must be really wrong if we
> > >   get EFAULT.
> > >
> > > So the diff below removes the fallback path and all associated code and
> > > variables.
> > 
> > > I have left out some minor cleanups for now to ease reviews.
> > 
> > Here's a diff that amends the signature of gettime() and makes use of
> > TIMESPEC_TO_TIMEVAL().
> > 
> > 
> > Index: event.c
> > ===
> > --- event.c.orig
> > +++ event.c
> > @@ -77,23 +77,20 @@ static void event_process_active(struct
> >  static int timeout_next(struct event_base *, struct timeval **);
> >  static voidtimeout_process(struct event_base *);
> >  
> > -static int
> > +static void
> >  gettime(struct event_base *base, struct timeval *tp)
> >  {
> > struct timespec ts;
> >  
> > if (base->tv_cache.tv_sec) {
> > *tp = base->tv_cache;
> > -   return (0);
> > +   return;
> > }
> >  
> > if (clock_gettime(CLOCK_MONOTONIC, ) == -1)
> > event_err(1, "libevent: clock_gettime failed");
> >  
> > -   tp->tv_sec = ts.tv_sec;
> > -   tp->tv_usec = ts.tv_nsec / 1000;
> > -
> > -   return (0);
> > +   TIMESPEC_TO_TIMEVAL(tp, );
> >  }
> >  
> >  struct event_base *
> > @@ -803,8 +800,7 @@ timeout_next(struct event_base *base, st
> > return (0);
> > }
> >  
> > -   if (gettime(base, ) == -1)
> > -   return (-1);
> > +   gettime(base, );
> >  
> > if (timercmp(>ev_timeout, , <=)) {
> > timerclear(tv);
> > 
> > 
> > -- 
> > jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE
> > 

Re: libevent: remove non-monotonic compat code

[Nicholas Marriott]

ok for both


On Tue, Apr 30, 2019 at 07:13:55PM +0200, Jeremie Courreges-Anglas wrote:
> On Tue, Apr 30 2019, Jeremie Courreges-Anglas  wrote:
> > libevent contains a fallback path in case clock_gettime(CLOCK_MONOTONIC)
> > fails.  The fallback path tries to cope with time going backwards and
> > reaches into the timeheap internals, as noticed by Tobias in
> >
> >   https://marc.info/?l=openbsd-tech=155595247719664=2
> >
> > I doubt that we care about this kind of seldom-tested compat code, which
> > was probably useful to get portable libevent to run on various
> > (ancient?) systems.
> >
> > Our clock_gettime can only fail for two reasons:
> > - invalid clock id.  Lots of our userland uses CLOCK_MONOTONIC now,
> >   I think it can be considered a requirement.
> > - invalid timespec pointer. Here, since we pass the address of
> >   a timespec structure on the stack something must be really wrong if we
> >   get EFAULT.
> >
> > So the diff below removes the fallback path and all associated code and
> > variables.
> 
> > I have left out some minor cleanups for now to ease reviews.
> 
> Here's a diff that amends the signature of gettime() and makes use of
> TIMESPEC_TO_TIMEVAL().
> 
> 
> Index: event.c
> ===
> --- event.c.orig
> +++ event.c
> @@ -77,23 +77,20 @@ static void   event_process_active(struct
>  static int   timeout_next(struct event_base *, struct timeval **);
>  static void  timeout_process(struct event_base *);
>  
> -static int
> +static void
>  gettime(struct event_base *base, struct timeval *tp)
>  {
>   struct timespec ts;
>  
>   if (base->tv_cache.tv_sec) {
>   *tp = base->tv_cache;
> - return (0);
> + return;
>   }
>  
>   if (clock_gettime(CLOCK_MONOTONIC, ) == -1)
>   event_err(1, "libevent: clock_gettime failed");
>  
> - tp->tv_sec = ts.tv_sec;
> - tp->tv_usec = ts.tv_nsec / 1000;
> -
> - return (0);
> + TIMESPEC_TO_TIMEVAL(tp, );
>  }
>  
>  struct event_base *
> @@ -803,8 +800,7 @@ timeout_next(struct event_base *base, st
>   return (0);
>   }
>  
> - if (gettime(base, ) == -1)
> - return (-1);
> + gettime(base, );
>  
>   if (timercmp(>ev_timeout, , <=)) {
>   timerclear(tv);
> 
> 
> -- 
> jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE
> 

Re: cu(1)/remote(5): add and document support for escape character capability

[Nicholas Marriott]

I personally think -E is enough, I don't know why you would need to
configure this per-host.


On Sun, Mar 24, 2019 at 04:18:58AM +0200, Artturi Alm wrote:
> Hi,
> 
> for completeness, would anything like the diff below be acceptable?
> 
> I might begin using /etc/remote soon, and I found the old remote.5[0]
> to have had this, and also described as such in remote(5) on
> docs.oracle.com, so i chose to reuse "es".
> 
> -Artturi
> 
> [0]: http://man.openbsd.org/OpenBSD-5.7/remote.5
> 
> 
> diff --git a/share/man/man5/remote.5 b/share/man/man5/remote.5
> index 2a50ca97d8a..b0d046a266a 100644
> --- a/share/man/man5/remote.5
> +++ b/share/man/man5/remote.5
> @@ -88,6 +88,15 @@ If this file refers to a terminal line,
>  .Xr cu 1
>  attempts to perform an exclusive open on the device to ensure only
>  one user at a time has access to the port.
> +.It Sy \
> +(str)
> +The escape character for
> +.Xr cu 1 .
> +This should be a single character, or
> +.Ql ^
> +followed by a letter.
> +The default is
> +.Ql ~ .
>  .El
>  .Sh FILES
>  .Bl -tag -width /etc/remote -compact
> diff --git a/usr.bin/cu/cu.1 b/usr.bin/cu/cu.1
> index 5daab4bc074..61c723a45e4 100644
> --- a/usr.bin/cu/cu.1
> +++ b/usr.bin/cu/cu.1
> @@ -105,6 +105,7 @@ uses the
>  .Xr remote 5
>  database to retrieve the
>  .Sy dc Pq directly connected ,
> +.Sy es Pq escape character ,
>  .Sy dv Pq device
>  and
>  .Sy br Pq baud rate
> diff --git a/usr.bin/cu/cu.c b/usr.bin/cu/cu.c
> index 697f72d3418..cba4ff1601d 100644
> --- a/usr.bin/cu/cu.c
> +++ b/usr.bin/cu/cu.c
> @@ -41,7 +41,7 @@ FILE*record_file;
>  struct termiossaved_tio;
>  struct bufferevent   *input_ev;
>  struct bufferevent   *output_ev;
> -int   escape_char = '~';
> +int   escape_char = -1;
>  int   is_direct = -1;
>  int   restricted = 0;
>  const char   *line_path = NULL;
> @@ -159,6 +159,8 @@ main(int argc, char **argv)
>   }
>   }
>  
> + if (escape_char == -1)
> + escape_char = '~';
>   if (line_path == NULL)
>   line_path = "/dev/cua00";
>   if (line_speed == -1)
> @@ -402,6 +404,25 @@ try_remote(const char *host, const char *path, const 
> char *entry)
>   if (line_path == NULL && cgetstr(cp, "dv", ) >= 0)
>   line_path = s;
>  
> + s = NULL;
> + if (escape_char == -1 && cgetustr(cp, "es", ) >= 0) {
> + switch (strnlen(s, 3)) {
> + case 1:
> + escape_char = (u_char)s[0];
> + break;
> + case 2:
> + if (s[0] == '^' &&
> + (u_char)s[1] >= 64 && (u_char)s[1] < 128) {
> + escape_char = (u_char)s[1] & 31;
> + break;
> + }
> + /* FALLTHROUGH */
> + default:
> + cu_warnx("invalid escape character: %s", s);
> + }
> + free(s);
> + }
> +
>   if (line_speed == -1 && cgetnum(cp, "br", ) >= 0) {
>   if (l < 0 || l > INT_MAX)
>   cu_errx(1, "speed out of range");

Re: Option for alternative escape character with cu(1)

[Nicholas Marriott]

I will commit this in the next few days unless I hear objections.



On Mon, Mar 18, 2019 at 07:39:45AM +, Nicholas Marriott wrote:
> 
> This is OK with me, anyone else?
> 
> 
> On Fri, Mar 15, 2019 at 06:56:31PM +0200, Artturi Alm wrote:
> > On Fri, Mar 15, 2019 at 02:43:04PM +0000, Nicholas Marriott wrote:
> > > .
> > > Another couple of minor changes below, with those it looks good to
> > > me. Any OK for this?
> > > 
> > 
> > With joined lines, the cast, and some runtime testing with ~, % and ^[.
> > 
> > -Artturi
> > 
> > diff --git a/usr.bin/cu/command.c b/usr.bin/cu/command.c
> > index c07fe73aeca..27d80f16dd7 100644
> > --- a/usr.bin/cu/command.c
> > +++ b/usr.bin/cu/command.c
> > @@ -30,6 +30,7 @@
> >  #include 
> >  #include 
> >  #include 
> > +#include 
> >  
> >  #include "cu.h"
> >  
> > @@ -223,6 +224,8 @@ start_record(void)
> >  void
> >  do_command(char c)
> >  {
> > +   char esc[4 + 1];
> > +
> > if (restricted && strchr("CRX$>", c) != NULL) {
> > cu_warnx("~%c command is not allowed in restricted mode", c);
> > return;
> > @@ -266,20 +269,23 @@ do_command(char c)
> > sleep(1);
> > ioctl(line_fd, TIOCCBRK, NULL);
> > break;
> > -   case '~':
> > -   bufferevent_write(line_ev, "~", 1);
> > +   default:
> > +   if (c == escape_char)
> > +   bufferevent_write(line_ev, , 1);
> > break;
> > case '?':
> > +   vis(esc, escape_char, VIS_WHITE | VIS_NOSLASH, 0);
> > printf("\r\n"
> > -   "~#  send break\r\n"
> > -   "~$  pipe local command to remote host\r\n"
> > -   "~>  send file to remote host\r\n"
> > -   "~C  connect program to remote host\r\n"
> > -   "~D  de-assert DTR line briefly\r\n"
> > -   "~R  start recording to file\r\n"
> > -   "~S  set speed\r\n"
> > -   "~X  send file with XMODEM\r\n"
> > -   "~?  get this summary\r\n"
> > +   "%s#  send break\r\n"
> > +   "%s$  pipe local command to remote host\r\n"
> > +   "%s>  send file to remote host\r\n"
> > +   "%sC  connect program to remote host\r\n"
> > +   "%sD  de-assert DTR line briefly\r\n"
> > +   "%sR  start recording to file\r\n"
> > +   "%sS  set speed\r\n"
> > +   "%sX  send file with XMODEM\r\n"
> > +   "%s?  get this summary\r\n",
> > +   esc, esc, esc, esc, esc, esc, esc, esc, esc
> > );
> > break;
> > }
> > diff --git a/usr.bin/cu/cu.1 b/usr.bin/cu/cu.1
> > index 104a6ea7893..3e85488e87c 100644
> > --- a/usr.bin/cu/cu.1
> > +++ b/usr.bin/cu/cu.1
> > @@ -36,6 +36,7 @@
> >  .Sh SYNOPSIS
> >  .Nm
> >  .Op Fl dr
> > +.Op Fl E Ar escape_char
> >  .Op Fl l Ar line
> >  .Op Fl s Ar speed | Fl Ar speed
> >  .Nm
> > @@ -55,6 +56,8 @@ The options are as follows:
> >  Specify that the line is directly connected and
> >  .Nm
> >  should not allow the driver to block waiting for a carrier to be detected.
> > +.It Fl E Ar escape_char
> > +Specify an escape character to use instead of the default tilde.
> >  .It Fl l Ar line
> >  Specify the line to use.
> >  Either of the forms like
> > diff --git a/usr.bin/cu/cu.c b/usr.bin/cu/cu.c
> > index 03a2df4181f..d01c3327042 100644
> > --- a/usr.bin/cu/cu.c
> > +++ b/usr.bin/cu/cu.c
> > @@ -41,6 +41,7 @@ FILE  *record_file;
> >  struct termios  saved_tio;
> >  struct bufferevent *input_ev;
> >  struct bufferevent *output_ev;
> > +int escape_char = '~';
> >  int is_direct = -1;
> >  int restricted = 0;
> >  const char *line_path = NULL;
> > @@ -53,7 +54,7 @@ struct event   sighup_ev;
> >  enum {
> > STATE_NONE,
> > STATE_NEWLINE,
> > -   STATE_TILDE
> > +   STATE_ESCAPE
> >  } last_state = STA

Re: Option for alternative escape character with cu(1)

[Nicholas Marriott]

This is OK with me, anyone else?


On Fri, Mar 15, 2019 at 06:56:31PM +0200, Artturi Alm wrote:
> On Fri, Mar 15, 2019 at 02:43:04PM +0000, Nicholas Marriott wrote:
> > .
> > Another couple of minor changes below, with those it looks good to
> > me. Any OK for this?
> > 
> 
> With joined lines, the cast, and some runtime testing with ~, % and ^[.
> 
> -Artturi
> 
> diff --git a/usr.bin/cu/command.c b/usr.bin/cu/command.c
> index c07fe73aeca..27d80f16dd7 100644
> --- a/usr.bin/cu/command.c
> +++ b/usr.bin/cu/command.c
> @@ -30,6 +30,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #include "cu.h"
>  
> @@ -223,6 +224,8 @@ start_record(void)
>  void
>  do_command(char c)
>  {
> + char esc[4 + 1];
> +
>   if (restricted && strchr("CRX$>", c) != NULL) {
>   cu_warnx("~%c command is not allowed in restricted mode", c);
>   return;
> @@ -266,20 +269,23 @@ do_command(char c)
>   sleep(1);
>   ioctl(line_fd, TIOCCBRK, NULL);
>   break;
> - case '~':
> - bufferevent_write(line_ev, "~", 1);
> + default:
> + if (c == escape_char)
> + bufferevent_write(line_ev, , 1);
>   break;
>   case '?':
> + vis(esc, escape_char, VIS_WHITE | VIS_NOSLASH, 0);
>   printf("\r\n"
> - "~#  send break\r\n"
> - "~$  pipe local command to remote host\r\n"
> - "~>  send file to remote host\r\n"
> - "~C  connect program to remote host\r\n"
> - "~D  de-assert DTR line briefly\r\n"
> - "~R  start recording to file\r\n"
> - "~S  set speed\r\n"
> - "~X  send file with XMODEM\r\n"
> - "~?  get this summary\r\n"
> + "%s#  send break\r\n"
> + "%s$  pipe local command to remote host\r\n"
> + "%s>  send file to remote host\r\n"
> + "%sC  connect program to remote host\r\n"
> + "%sD  de-assert DTR line briefly\r\n"
> + "%sR  start recording to file\r\n"
> + "%sS  set speed\r\n"
> + "%sX  send file with XMODEM\r\n"
> + "%s?  get this summary\r\n",
> + esc, esc, esc, esc, esc, esc, esc, esc, esc
>   );
>   break;
>   }
> diff --git a/usr.bin/cu/cu.1 b/usr.bin/cu/cu.1
> index 104a6ea7893..3e85488e87c 100644
> --- a/usr.bin/cu/cu.1
> +++ b/usr.bin/cu/cu.1
> @@ -36,6 +36,7 @@
>  .Sh SYNOPSIS
>  .Nm
>  .Op Fl dr
> +.Op Fl E Ar escape_char
>  .Op Fl l Ar line
>  .Op Fl s Ar speed | Fl Ar speed
>  .Nm
> @@ -55,6 +56,8 @@ The options are as follows:
>  Specify that the line is directly connected and
>  .Nm
>  should not allow the driver to block waiting for a carrier to be detected.
> +.It Fl E Ar escape_char
> +Specify an escape character to use instead of the default tilde.
>  .It Fl l Ar line
>  Specify the line to use.
>  Either of the forms like
> diff --git a/usr.bin/cu/cu.c b/usr.bin/cu/cu.c
> index 03a2df4181f..d01c3327042 100644
> --- a/usr.bin/cu/cu.c
> +++ b/usr.bin/cu/cu.c
> @@ -41,6 +41,7 @@ FILE*record_file;
>  struct termiossaved_tio;
>  struct bufferevent   *input_ev;
>  struct bufferevent   *output_ev;
> +int   escape_char = '~';
>  int   is_direct = -1;
>  int   restricted = 0;
>  const char   *line_path = NULL;
> @@ -53,7 +54,7 @@ struct event sighup_ev;
>  enum {
>   STATE_NONE,
>   STATE_NEWLINE,
> - STATE_TILDE
> + STATE_ESCAPE
>  } last_state = STATE_NEWLINE;
>  
>  __dead void  usage(void);
> @@ -67,8 +68,8 @@ voidtry_remote(const char *, const char *, 
> const char *);
>  __dead void
>  usage(void)
>  {
> - fprintf(stderr, "usage: %s [-dr] [-l line] [-s speed | -speed]\n",
> - __progname);
> + fprintf(stderr, "usage: %s [-dr] [-E escape_char] [-l line] "
> + "[-s speed | -speed]\n", __progname);
>   fprintf(stderr, "   %s [host]\n", __progname);
>   exit(1);
>  }
> @@ -94,14 +95,14 @@ main(int argc, char **argv)
>   for (i = 1; i 

Re: Option for alternative escape character with cu(1)

[Nicholas Marriott]

On Fri, Mar 15, 2019 at 09:43:56AM -0600, Todd C. Miller wrote:
> Wouldn't it be less error-prone to make escape_char u_char instead
> of int?

Maybe, I don't mind either way.

However this in stream_read would still need a cast as ptr is signed:

if (state_change && (u_char)*ptr == escape_char) {

Although there is no reason ptr couldn't be u_char * too (in that case
it would be nice to change line_read as well).

Re: Option for alternative escape character with cu(1)

[Nicholas Marriott]

.
Another couple of minor changes below, with those it looks good to
me. Any OK for this?


On Fri, Mar 15, 2019 at 01:49:52PM +0200, Artturi Alm wrote:
> On Thu, Mar 14, 2019 at 10:18:57AM +0000, Nicholas Marriott wrote:
> > Thanks, comments inline.
> > 
> 
> The diff looks much better to me, now with those things fixed based
> on your feedback, thanks :]
> 
> -Artturi
> 
> 
> diff --git a/usr.bin/cu/command.c b/usr.bin/cu/command.c
> index c07fe73aeca..27d80f16dd7 100644
> --- a/usr.bin/cu/command.c
> +++ b/usr.bin/cu/command.c
> @@ -30,6 +30,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #include "cu.h"
>  
> @@ -223,6 +224,8 @@ start_record(void)
>  void
>  do_command(char c)
>  {
> + char esc[4 + 1];
> +
>   if (restricted && strchr("CRX$>", c) != NULL) {
>   cu_warnx("~%c command is not allowed in restricted mode", c);
>   return;
> @@ -266,20 +269,23 @@ do_command(char c)
>   sleep(1);
>   ioctl(line_fd, TIOCCBRK, NULL);
>   break;
> - case '~':
> - bufferevent_write(line_ev, "~", 1);
> + default:
> + if (c == escape_char)
> + bufferevent_write(line_ev, , 1);
>   break;
>   case '?':
> + vis(esc, escape_char, VIS_WHITE | VIS_NOSLASH, 0);
>   printf("\r\n"
> - "~#  send break\r\n"
> - "~$  pipe local command to remote host\r\n"
> - "~>  send file to remote host\r\n"
> - "~C  connect program to remote host\r\n"
> - "~D  de-assert DTR line briefly\r\n"
> - "~R  start recording to file\r\n"
> - "~S  set speed\r\n"
> - "~X  send file with XMODEM\r\n"
> - "~?  get this summary\r\n"
> + "%s#  send break\r\n"
> + "%s$  pipe local command to remote host\r\n"
> + "%s>  send file to remote host\r\n"
> + "%sC  connect program to remote host\r\n"
> + "%sD  de-assert DTR line briefly\r\n"
> + "%sR  start recording to file\r\n"
> + "%sS  set speed\r\n"
> + "%sX  send file with XMODEM\r\n"
> + "%s?  get this summary\r\n",
> + esc, esc, esc, esc, esc, esc, esc, esc, esc
>   );
>   break;
>   }
> diff --git a/usr.bin/cu/cu.1 b/usr.bin/cu/cu.1
> index 104a6ea7893..3e85488e87c 100644
> --- a/usr.bin/cu/cu.1
> +++ b/usr.bin/cu/cu.1
> @@ -36,6 +36,7 @@
>  .Sh SYNOPSIS
>  .Nm
>  .Op Fl dr
> +.Op Fl E Ar escape_char
>  .Op Fl l Ar line
>  .Op Fl s Ar speed | Fl Ar speed
>  .Nm
> @@ -55,6 +56,8 @@ The options are as follows:
>  Specify that the line is directly connected and
>  .Nm
>  should not allow the driver to block waiting for a carrier to be detected.
> +.It Fl E Ar escape_char
> +Specify an escape character to use instead of the default tilde.
>  .It Fl l Ar line
>  Specify the line to use.
>  Either of the forms like
> diff --git a/usr.bin/cu/cu.c b/usr.bin/cu/cu.c
> index 03a2df4181f..8e3ce0b0c0e 100644
> --- a/usr.bin/cu/cu.c
> +++ b/usr.bin/cu/cu.c
> @@ -41,6 +41,7 @@ FILE*record_file;
>  struct termiossaved_tio;
>  struct bufferevent   *input_ev;
>  struct bufferevent   *output_ev;
> +int   escape_char = '~';
>  int   is_direct = -1;
>  int   restricted = 0;
>  const char   *line_path = NULL;
> @@ -53,7 +54,7 @@ struct event sighup_ev;
>  enum {
>   STATE_NONE,
>   STATE_NEWLINE,
> - STATE_TILDE
> + STATE_ESCAPE
>  } last_state = STATE_NEWLINE;
>  
>  __dead void  usage(void);
> @@ -67,8 +68,8 @@ voidtry_remote(const char *, const char *, 
> const char *);
>  __dead void
>  usage(void)
>  {
> - fprintf(stderr, "usage: %s [-dr] [-l line] [-s speed | -speed]\n",
> - __progname);
> + fprintf(stderr, "usage: %s [-dr] [-E escape_char] [-l line] "
> + "[-s speed | -speed]\n", __progname);
>   fprintf(stderr, "   %s [host]\n", __progname);
>   exit(1);
>  }
> @@ -94,14 +95,14 @@ main(int argc, char **argv)
>   for (i = 

Re: Option for alternative escape character with cu(1)

[Nicholas Marriott]

Thanks, comments inline.

> diff --git a/usr.bin/cu/command.c b/usr.bin/cu/command.c
> index c07fe73aeca..e225fb544be 100644
> --- a/usr.bin/cu/command.c
> +++ b/usr.bin/cu/command.c
> @@ -30,6 +30,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #include "cu.h"
>  
> @@ -223,6 +224,8 @@ start_record(void)
>  void
>  do_command(char c)
>  {
> + char esc[4 + 1];
> +
>   if (restricted && strchr("CRX$>", c) != NULL) {
>   cu_warnx("~%c command is not allowed in restricted mode", c);
>   return;
> @@ -266,20 +269,26 @@ do_command(char c)
>   sleep(1);
>   ioctl(line_fd, TIOCCBRK, NULL);
>   break;
> - case '~':
> - bufferevent_write(line_ev, "~", 1);
> + default:
> + if (c != escape_char)
> + break;
> + esc[0] = escape_char;
> + esc[1] = '\0';

No need for \0 since you are only writing one byte, but...

> + bufferevent_write(line_ev, esc, 1);

...why not do this instead:

bufferevent_write(line_ev, , 1);

>   break;
>   case '?':
> + vis(esc, escape_char, VIS_WHITE | VIS_NOSLASH, 0);
>   printf("\r\n"
> - "~#  send break\r\n"
> - "~$  pipe local command to remote host\r\n"
> - "~>  send file to remote host\r\n"
> - "~C  connect program to remote host\r\n"
> - "~D  de-assert DTR line briefly\r\n"
> - "~R  start recording to file\r\n"
> - "~S  set speed\r\n"
> - "~X  send file with XMODEM\r\n"
> - "~?  get this summary\r\n"
> + "%s#  send break\r\n"
> + "%s$  pipe local command to remote host\r\n"
> + "%s>  send file to remote host\r\n"
> + "%sC  connect program to remote host\r\n"
> + "%sD  de-assert DTR line briefly\r\n"
> + "%sR  start recording to file\r\n"
> + "%sS  set speed\r\n"
> + "%sX  send file with XMODEM\r\n"
> + "%s?  get this summary\r\n",
> + esc, esc, esc, esc, esc, esc, esc, esc, esc
>   );
>   break;
>   }
> diff --git a/usr.bin/cu/cu.1 b/usr.bin/cu/cu.1
> index 104a6ea7893..d52e0b94e5f 100644
> --- a/usr.bin/cu/cu.1
> +++ b/usr.bin/cu/cu.1
> @@ -38,6 +38,7 @@
>  .Op Fl dr
>  .Op Fl l Ar line
>  .Op Fl s Ar speed | Fl Ar speed
> +.Op Fl E Ar escape_char

E comes before l and s.

>  .Nm
>  .Op Ar host
>  .Sh DESCRIPTION
> @@ -92,6 +93,8 @@ and
>  .It Fl s Ar speed | Fl Ar speed
>  Set the speed of the connection.
>  The default is 9600.
> +.It Fl E Ar escape_char
> +Specify a escape character to use instead of the default tilde.

"an escape character".

>  .El
>  .Pp
>  If
> diff --git a/usr.bin/cu/cu.c b/usr.bin/cu/cu.c
> index 03a2df4181f..f269f4a74f3 100644
> --- a/usr.bin/cu/cu.c
> +++ b/usr.bin/cu/cu.c
> @@ -41,6 +41,7 @@ FILE*record_file;
>  struct termiossaved_tio;
>  struct bufferevent   *input_ev;
>  struct bufferevent   *output_ev;
> +int   escape_char = -1;

Why not just initialize it to '~'?

>  int   is_direct = -1;
>  int   restricted = 0;
>  const char   *line_path = NULL;
> @@ -53,7 +54,7 @@ struct event sighup_ev;
>  enum {
>   STATE_NONE,
>   STATE_NEWLINE,
> - STATE_TILDE
> + STATE_ESCAPE
>  } last_state = STATE_NEWLINE;
>  
>  __dead void  usage(void);
> @@ -67,8 +68,8 @@ voidtry_remote(const char *, const char *, 
> const char *);
>  __dead void
>  usage(void)
>  {
> - fprintf(stderr, "usage: %s [-dr] [-l line] [-s speed | -speed]\n",
> - __progname);
> + fprintf(stderr, "usage: %s [-dr] [-l line] [-s speed | -speed]"
> + " [-E escape_char]\n", __progname);

-E comes before -l and -s.

>   fprintf(stderr, "   %s [host]\n", __progname);
>   exit(1);
>  }
> @@ -101,7 +102,7 @@ main(int argc, char **argv)
>   errx(1, "speed asprintf");
>   }
>  
> - while ((opt = getopt(argc, argv, "drl:s:")) != -1) {
> + while ((opt = getopt(argc, argv, "drl:s:E:")) != -1) {
>   switch (opt) {
>   case 'd':
>   is_direct = 1;
> @@ -119,6 +120,17 @@ main(int argc, char **argv)
>   if (errstr != NULL)
>   errx(1, "speed is %s: %s", errstr, optarg);
>   break;
> + case 'E':
> + if (optarg[0] == '^' && optarg[2] == 0 &&
> + (unsigned char)optarg[1] >= 64 &&
> + (unsigned char)optarg[1] < 128)
> + escape_char = (unsigned char)optarg[1] & 31;
> + 

Re: Option for alternative escape character with cu(1)

[Nicholas Marriott]

Why only % rather than have -e take an argument like ssh?


On Wed, Mar 13, 2019 at 02:35:06PM +0200, Artturi Alm wrote:
> Hi,
> 
> i don't have issues with tilde when using locally, but i mostly ssh to
> reach cu, and too many times i've forgotten to configure ssh/use -e,
> with this cu(1) becomes safer/easier to use for us with non-english
> keyboard.
> ~tilde is certainly annoying when it's three key presses alone,
> and then you mostly get only one shot at trying..
> 
> is this bloat?
> 
> -Artturi
> 
> 
> diff --git a/usr.bin/cu/command.c b/usr.bin/cu/command.c
> index c07fe73aeca..d97db3b56de 100644
> --- a/usr.bin/cu/command.c
> +++ b/usr.bin/cu/command.c
> @@ -223,6 +223,8 @@ start_record(void)
>  void
>  do_command(char c)
>  {
> + char esc = alt_esc ? '%' : '~';
> +
>   if (restricted && strchr("CRX$>", c) != NULL) {
>   cu_warnx("~%c command is not allowed in restricted mode", c);
>   return;
> @@ -271,15 +273,16 @@ do_command(char c)
>   break;
>   case '?':
>   printf("\r\n"
> - "~#  send break\r\n"
> - "~$  pipe local command to remote host\r\n"
> - "~>  send file to remote host\r\n"
> - "~C  connect program to remote host\r\n"
> - "~D  de-assert DTR line briefly\r\n"
> - "~R  start recording to file\r\n"
> - "~S  set speed\r\n"
> - "~X  send file with XMODEM\r\n"
> - "~?  get this summary\r\n"
> + "%c#  send break\r\n"
> + "%c$  pipe local command to remote host\r\n"
> + "%c>  send file to remote host\r\n"
> + "%cC  connect program to remote host\r\n"
> + "%cD  de-assert DTR line briefly\r\n"
> + "%cR  start recording to file\r\n"
> + "%cS  set speed\r\n"
> + "%cX  send file with XMODEM\r\n"
> + "%c?  get this summary\r\n",
> + esc, esc, esc, esc, esc, esc, esc, esc, esc
>   );
>   break;
>   }
> diff --git a/usr.bin/cu/cu.1 b/usr.bin/cu/cu.1
> index 104a6ea7893..1d609e14947 100644
> --- a/usr.bin/cu/cu.1
> +++ b/usr.bin/cu/cu.1
> @@ -35,7 +35,7 @@
>  .Nd serial terminal emulator
>  .Sh SYNOPSIS
>  .Nm
> -.Op Fl dr
> +.Op Fl der
>  .Op Fl l Ar line
>  .Op Fl s Ar speed | Fl Ar speed
>  .Nm
> @@ -55,6 +55,10 @@ The options are as follows:
>  Specify that the line is directly connected and
>  .Nm
>  should not allow the driver to block waiting for a carrier to be detected.
> +.It Fl e
> +Use a percent sign
> +.Pq Ql %
> +as the escape character instead of tilde.
>  .It Fl l Ar line
>  Specify the line to use.
>  Either of the forms like
> diff --git a/usr.bin/cu/cu.c b/usr.bin/cu/cu.c
> index 03a2df4181f..b66f4698605 100644
> --- a/usr.bin/cu/cu.c
> +++ b/usr.bin/cu/cu.c
> @@ -41,6 +41,7 @@ FILE*record_file;
>  struct termiossaved_tio;
>  struct bufferevent   *input_ev;
>  struct bufferevent   *output_ev;
> +int   alt_esc = 0;
>  int   is_direct = -1;
>  int   restricted = 0;
>  const char   *line_path = NULL;
> @@ -53,7 +54,7 @@ struct event sighup_ev;
>  enum {
>   STATE_NONE,
>   STATE_NEWLINE,
> - STATE_TILDE
> + STATE_ESCAPE
>  } last_state = STATE_NEWLINE;
>  
>  __dead void  usage(void);
> @@ -67,7 +68,7 @@ voidtry_remote(const char *, const char *, 
> const char *);
>  __dead void
>  usage(void)
>  {
> - fprintf(stderr, "usage: %s [-dr] [-l line] [-s speed | -speed]\n",
> + fprintf(stderr, "usage: %s [-der] [-l line] [-s speed | -speed]\n",
>   __progname);
>   fprintf(stderr, "   %s [host]\n", __progname);
>   exit(1);
> @@ -101,11 +102,14 @@ main(int argc, char **argv)
>   errx(1, "speed asprintf");
>   }
>  
> - while ((opt = getopt(argc, argv, "drl:s:")) != -1) {
> + while ((opt = getopt(argc, argv, "derl:s:")) != -1) {
>   switch (opt) {
>   case 'd':
>   is_direct = 1;
>   break;
> + case 'e':
> + alt_esc = 1;
> + break;
>   case 'r':
>   if (pledge("stdio rpath wpath tty", NULL) == -1)
>   err(1, "pledge");
> @@ -308,14 +312,14 @@ stream_read(struct bufferevent *bufev, void *data)
>   last_state = STATE_NEWLINE;
>   break;
>   case STATE_NEWLINE:
> - if (state_change && *ptr == '~') {
> - last_state = STATE_TILDE;
> + if (state_change && *ptr == "~%"[alt_esc]) {
> + last_state = STATE_ESCAPE;
>

Re: start cleaning up UTF-8 processing in less(1)

[Nicholas Marriott]

Looks good, ok nicm



On Tue, Feb 26, 2019 at 02:39:14AM +0100, Ingo Schwarze wrote:
> Hi Todd,
> 
> Todd C. Miller wrote on Mon, Feb 25, 2019 at 01:06:02PM -0700:
> > On Mon, 25 Feb 2019 19:43:36 +0100, Ingo Schwarze wrote:
> >> Todd C. Miller wrote on Mon, Feb 25, 2019 at 09:45:12AM -0700:
> >>> On Mon, 25 Feb 2019 12:39:41 +0100, Ingo Schwarze wrote:
> 
>  Index: line.c
> >> [...]
>  @@ -469,11 +469,10 @@ in_ansi_esc_seq(void)
>    * Search backwards for either an ESC (which means we ARE in a 
>  seq);
>    * or an end char (which means we're NOT in a seq).
>    */
>  -for (p = [curr]; p > linebuf; ) {
>  -LWCHAR ch = step_char(, -1, linebuf);
>  -if (IS_CSI_START(ch))
>  +for (p = linebuf + curr - 1; p >= linebuf; p--) {
> 
> >>> Since curr can be 0, can this lead to be a single byte underflow?
> 
> >> No, in that case (which logically means the line buffer is empty),
> >> the end condition p >= linebuf is false right away, the loop
> >> is never entered, the function returns 0 right away and at the
> >> call site, the first if brach (containing "curr--") isn't entered
> >> either.
> 
> > Strictly speaking, the result of "p = linebuf + curr - 1" is undefined
> > when curr < 1.  There is a special case in the standard when the
> > result is one past the end of an array but no corresponding case
> > for one element before the array.  In practice, it is unlikely to
> > matter.
> 
> Ouch, i keep forgetting that one, thanks for reminding me.
> 
> In that case, let's just use an index rather than a pointer;
> diff otherwise unchanged.
> 
> OK?
>   Ingo
> 
> 
> Index: cvt.c
> ===
> RCS file: /cvs/src/usr.bin/less/cvt.c,v
> retrieving revision 1.8
> diff -u -p -r1.8 cvt.c
> --- cvt.c 17 Sep 2016 15:06:41 -  1.8
> +++ cvt.c 26 Feb 2019 01:31:31 -
> @@ -77,7 +77,7 @@ cvt_text(char *odst, char *osrc, int *ch
>   dst--;
>   } while (dst > odst &&
>   !IS_ASCII_OCTET(*dst) && !IS_UTF8_LEAD(*dst));
> - } else if ((ops & CVT_ANSI) && IS_CSI_START(ch)) {
> + } else if ((ops & CVT_ANSI) && ch == ESC) {
>   /* Skip to end of ANSI escape sequence. */
>   src++;  /* skip the CSI start char */
>   while (src < src_end)
> Index: filename.c
> ===
> RCS file: /cvs/src/usr.bin/less/filename.c,v
> retrieving revision 1.26
> diff -u -p -r1.26 filename.c
> --- filename.c29 Oct 2017 17:10:55 -  1.26
> +++ filename.c26 Feb 2019 01:31:31 -
> @@ -348,7 +348,7 @@ bin_file(int f)
>   pend = [n];
>   for (p = data; p < pend; ) {
>   LWCHAR c = step_char(, +1, pend);
> - if (ctldisp == OPT_ONPLUS && IS_CSI_START(c)) {
> + if (ctldisp == OPT_ONPLUS && c == ESC) {
>   do {
>   c = step_char(, +1, pend);
>   } while (p < pend && is_ansi_middle(c));
> Index: less.h
> ===
> RCS file: /cvs/src/usr.bin/less/less.h,v
> retrieving revision 1.28
> diff -u -p -r1.28 less.h
> --- less.h30 Dec 2018 23:09:58 -  1.28
> +++ less.h26 Feb 2019 01:31:31 -
> @@ -38,8 +38,6 @@
>  #define  IS_SPACE(c) isspace((unsigned char)(c))
>  #define  IS_DIGIT(c) isdigit((unsigned char)(c))
>  
> -#define  IS_CSI_START(c) (((LWCHAR)(c)) == ESC || (((LWCHAR)(c)) == CSI))
> -
>  #ifndef TRUE
>  #define  TRUE1
>  #endif
> @@ -151,9 +149,7 @@ struct textlist {
>  #define  AT_INDET(1 << 7)  /* Indeterminate: either bold or 
> underline */
>  
>  #define  CONTROL(c)  ((c)&037)
> -
>  #define  ESC CONTROL('[')
> -#define  CSI ((unsigned char)'\233')
>  
>  #define  S_INTERRUPT 01
>  #define  S_STOP  02
> Index: line.c
> ===
> RCS file: /cvs/src/usr.bin/less/line.c,v
> retrieving revision 1.23
> diff -u -p -r1.23 line.c
> --- line.c24 Feb 2019 04:54:36 -  1.23
> +++ line.c26 Feb 2019 01:31:31 -
> @@ -230,7 +230,7 @@ pshift(int shift)
>*/
>   while (shifted <= shift && from < curr) {
>   c = linebuf[from];
> - if (ctldisp == OPT_ONPLUS && IS_CSI_START(c)) {
> + if (ctldisp == OPT_ONPLUS && c == ESC) {
>   /* Keep cumulative effect.  */
>   linebuf[to] = c;
>   attr[to++] = attr[from++];
> @@ -463,17 +463,16 @@ backc(void)
>  static int
>  in_ansi_esc_seq(void)
>  {
> - char *p;
> + int i;
>  
>   /*
>* Search backwards for either an 

Re: start cleaning up UTF-8 processing in less(1)

[Nicholas Marriott]

> During the upcoming cleanup steps, let use retain full support for
> the first (ESC-[) syntax and lets us completely delete support for
> the second and third CSI syntaxes (single-byte CSI and UTF-8
> single-character two-byte CSI).
> 
> If you are OK with that plan, i'll send diffs implementing that.

Very little, if anything at all, uses 8-bit or UTF-8 CSI, getting rid of
it is a good idea.

Re: file(1) better ideas to recognize rust language code?

[Nicholas Marriott]

I would really like it if file(1) could use the upstream magic files,
but they now use a lot of regex.

Even if I just add the c-lang file, the difference is dramatic,
especially on large files:

$ time file magdir/* /etc/* /bin/* >/dev/null
0m01.05s real 0m00.87s user 0m00.16s system
$ time ./file magdir/* /etc/* /bin/* >/dev/null
0m04.89s real 0m04.63s user 0m00.25s system

$ time file ./post-magic >/dev/null
0m00.54s real 0m00.45s user 0m00.10s system
$ time ./file ./post-magic >/dev/null
0m13.31s real 0m13.20s user 0m00.08s system

If I link with PCRE instead (I am not suggesting we do this!):

$ time ./file magdir/* /etc/* /bin/* >/dev/null
0m00.96s real 0m00.78s user 0m00.17s system
$ time ./file ./post-magic >/dev/null
0m00.25s real 0m00.17s user 0m00.07s system

So the best way to improve our file(1) would probably be to make our
libc regex engine faster...


On Tue, Jan 15, 2019 at 07:27:15AM +, Nicholas Marriott wrote:
> Hi
> 
> I think I would avoid adding more of these at the moment, especially
> ones that aren't very specific (why is "package" Go and not Java?) and
> for languages that haven't been around very long, unless it is solving a
> specific problem.
> 
> Original file has moved these into the magic files and made them more
> sophisticated (Magdir/c-lang), but I doubt our regex code is fast enough
> to get away with this. It is mostly stuff like ^ and leading spaces or
> #s though - perhaps we could make the C searching code better though, I
> just copied what our old file version did. Not sure it is worth it.
> 
> 
> On Tue, Jan 15, 2019 at 02:00:11AM -0500, Ted Unangst wrote:
> > Matteo Niccoli wrote:
> > > Didn't find any other examples. At the moment rust code is recognized
> > > as ASCII C program text.
> > 
> > src/usr.bin/file/text.c has an array of special matches for text.
> > 
> > It has various omissions, though.
> > 
> >  is matched as SGML.
> > import means Java, but not python or go.
> > 
> > etc. I suppose it doesn't hurt to add a few more entries, but every entry
> > slows down file. So we shouldn't go too wild.
> > 
> > Anyway, this adds support for go by matching "package". It also removes two
> > entries that result in false positives if they match too soon.
> > 
> > Index: text.c
> > ===
> > RCS file: /cvs/src/usr.bin/file/text.c,v
> > retrieving revision 1.3
> > diff -u -p -r1.3 text.c
> > --- text.c  18 Apr 2017 14:16:48 -  1.3
> > +++ text.c  15 Jan 2019 06:58:36 -
> > @@ -31,14 +31,13 @@ static const char *text_words[][3] = {
> > { "import", "Java program", "text/x-java" },
> > { "\"libhdr\"", "BCPL program", "text/x-bcpl" },
> > { "\"LIBHDR\"", "BCPL program", "text/x-bcpl" },
> > -   { "//", "C++ program", "text/x-c++" },
> > { "virtual", "C++ program", "text/x-c++" },
> > { "class", "C++ program", "text/x-c++" },
> > { "public:", "C++ program", "text/x-c++" },
> > { "private:", "C++ program", "text/x-c++" },
> > -   { "/*", "C program", "text/x-c" },
> > { "#include", "C program", "text/x-c" },
> > { "char", "C program", "text/x-c" },
> > +   { "package", "Go program", "text/x-go" },
> > { "The", "English", "text/plain" },
> > { "the", "English", "text/plain" },
> > { "double", "C program", "text/x-c" },
> > 

Re: file(1) better ideas to recognize rust language code?

[Nicholas Marriott]

Hi

I think I would avoid adding more of these at the moment, especially
ones that aren't very specific (why is "package" Go and not Java?) and
for languages that haven't been around very long, unless it is solving a
specific problem.

Original file has moved these into the magic files and made them more
sophisticated (Magdir/c-lang), but I doubt our regex code is fast enough
to get away with this. It is mostly stuff like ^ and leading spaces or
#s though - perhaps we could make the C searching code better though, I
just copied what our old file version did. Not sure it is worth it.


On Tue, Jan 15, 2019 at 02:00:11AM -0500, Ted Unangst wrote:
> Matteo Niccoli wrote:
> > Didn't find any other examples. At the moment rust code is recognized
> > as ASCII C program text.
> 
> src/usr.bin/file/text.c has an array of special matches for text.
> 
> It has various omissions, though.
> 
>  is matched as SGML.
> import means Java, but not python or go.
> 
> etc. I suppose it doesn't hurt to add a few more entries, but every entry
> slows down file. So we shouldn't go too wild.
> 
> Anyway, this adds support for go by matching "package". It also removes two
> entries that result in false positives if they match too soon.
> 
> Index: text.c
> ===
> RCS file: /cvs/src/usr.bin/file/text.c,v
> retrieving revision 1.3
> diff -u -p -r1.3 text.c
> --- text.c18 Apr 2017 14:16:48 -  1.3
> +++ text.c15 Jan 2019 06:58:36 -
> @@ -31,14 +31,13 @@ static const char *text_words[][3] = {
>   { "import", "Java program", "text/x-java" },
>   { "\"libhdr\"", "BCPL program", "text/x-bcpl" },
>   { "\"LIBHDR\"", "BCPL program", "text/x-bcpl" },
> - { "//", "C++ program", "text/x-c++" },
>   { "virtual", "C++ program", "text/x-c++" },
>   { "class", "C++ program", "text/x-c++" },
>   { "public:", "C++ program", "text/x-c++" },
>   { "private:", "C++ program", "text/x-c++" },
> - { "/*", "C program", "text/x-c" },
>   { "#include", "C program", "text/x-c" },
>   { "char", "C program", "text/x-c" },
> + { "package", "Go program", "text/x-go" },
>   { "The", "English", "text/plain" },
>   { "the", "English", "text/plain" },
>   { "double", "C program", "text/x-c" },
> 

Re: update magic file for qcow

[Nicholas Marriott]

This looks OK and is probably the easiest thing to do.

Note that string/b does not have the same meaning for us because in
original file some eejit decided to change "b" to "w" and then use "b"
to mean something else, but I think it is harmless here.

They have split this out into a "virtual" file which looks fine, and
extended "msdos" which I think is alright too although it might need
some tweaks. So bringing those both in would be an option instead.





On Wed, Oct 03, 2018 at 08:37:10PM -0700, Carlos Cardenas wrote:
> Attached is patch from netbsd for updated qcow definitions.
> 
> Comments? Ok?
> 
> +--+
> Carlos

> Index: msdos
> ===
> RCS file: /home/los/cvs/src/usr.bin/file/magdir/msdos,v
> retrieving revision 1.6
> diff -u -p -r1.6 msdos
> --- msdos 29 Jan 2016 11:50:40 -  1.6
> +++ msdos 3 Oct 2018 05:25:21 -
> @@ -641,43 +641,77 @@
>  #
>  # Qemu Emulator Images
>  # Lines written by Friedrich Schwittay (f.schwit...@yousable.de)
> -# Made by reading sources and doing trial and error on existing
> -# qcow files
> -0   string  QFI Qemu Image, Format: Qcow
> +# Updated by Adam Buchbinder (adam.buchbin...@gmail.com)
> +# Made by reading sources, reading documentation, and doing trial and error
> +# on existing QCOW files
> +0string/bQFI\xFB QEMU QCOW Image
>  
>  # Uncomment the following line to display Magic (only used for debugging
>  # this magic number)
> -#>0 string  x   , Magic: %s
> +#>0  string/bx   , Magic: %s
>  
> -# There are currently 2 Versions: "1" and "2"
> -# I do not use Version 2 and therefore branch here
> -# but can assure: it works (tested on both versions)
> -# Also my Qemu 0.9.0 which uses this Version 2 refuses
> -# to start in its bios
> ->0x04   belong  2   , Version: 2
> ->0x04   belong  1   , Version: 1
> +# There are currently 2 Versions: "1" and "2".
> +# http://www.gnome.org/~markmc/qcow-image-format-version-1.html
> +>4   belong  1   (v1)
>  
> -# Using the existence of the Backing File Offset to Branch or not
> +# Using the existence of the Backing File Offset to determine whether
>  # to read Backing File Information
> ->>0xcbelong  >0  , Backing File( Offset: %lu
> ->>>(0xc.L)   string >\0 , Path: %s
> -
> -# Didn't get the trick here how qemu stores the "Size" at this Position
> -# There is actually something stored but nothing makes sense
> -# The header in the sources talks about it
> -#>>>16   lelong  x   , Size: %lu
> +>>12 belong   >0  \b, has backing file (
> +# Note that this isn't a null-terminated string; the length is actually
> +# (16.L). Assuming a null-terminated string happens to work usually, but it
> +# may spew junk until it reaches a \0 in some cases.
> +>>>(12.L) string >\0 \bpath %s
>  
>  # Modification time of the Backing File
>  # Really useful if you want to know if your backing
>  # file is still usable together with this image
> ->>>20bedate x   , Mtime: %s )
> +20   bedate >0   \b, mtime %s)
> +20   default x   \b)
> +
> +# Size is stored in bytes in a big-endian u64.
> +>>24 bequad  x\b, %lld bytes
>  
> -# Don't know how to calculate in Magicfiles
> -# Also: this Information is not reliably
> -#   stored in image-files
> ->>24 lelong  x   , Disk Size could be: %d * 256 bytes
> +# 1 for AES encryption, 0 for none.
> +>>36 belong  1   \b, AES-encrypted
>  
> -0string  QEVMQEMU's suspend to disk image
> +# http://www.gnome.org/~markmc/qcow-image-format.html
> +>4   belong  2   (v2)
> +# Using the existence of the Backing File Offset to determine whether
> +# to read Backing File Information
> +>>8  bequad  >0   \b, has backing file
> +# Note that this isn't a null-terminated string; the length is actually
> +# (16.L). Assuming a null-terminated string happens to work usually, but it
> +# may spew junk until it reaches a \0 in some cases. Also, since there's no
> +# .Q modifier, we just use the bottom four bytes as an offset. Note that if
> +# the file is over 4G, and the backing file path is stored after the first 
> 4G,
> +# the wrong filename will be printed. (This should be (8.Q), when that syntax
> +# is introduced.)
> +>>>(12.L) string >\0 (path %s)
> +>>24 bequad  x   \b, %lld bytes
> +>>32 belong  1   \b, AES-encrypted
> +
> +>4   belong  3   (v3)
> +# Using the existence of the Backing File Offset to determine whether
> +# to read Backing File Information
> +>>8  bequad  >0   \b, has backing file
> +# Note that this isn't a null-terminated string; the length is actually
> +# (16.L). Assuming a null-terminated string happens to work usually, but it
> +# may spew junk until it reaches a \0 in some cases. Also, since there's no
> +# .Q modifier, we just use the bottom four bytes as an offset. 

Re: Add bufferevent_setwatermark(3) to manual

[Nicholas Marriott]

ok nicm



On Sat, Sep 22, 2018 at 10:15:21AM +0200, Anton Lindqvist wrote:
> On Fri, Sep 21, 2018 at 06:36:54PM -0700, Geoff Hill wrote:
> > Hello tech,
> > 
> > I noticed the event(3) manual pages don't mention the
> > bufferevent_setwatermark(3) function and glosses over the details of
> > watermarks, even though there's a few programs in userland that set both
> > read and write watermarks. Looks like there was an effort in 2017
> > to add some documentation but it stalled.
> > 
> > Here's a patch that adds the function synopsis and a brief description
> > of how watermarks work separately for read and write. Mostly copied from
> > the function declaration comments in event.h.
> > 
> > ok?
> 
> Few nits below, otherwise it looks good. Anyone else willing to ok?
> 
> > 
> > Geoff Hill
> > 
> > 
> > Index: event.3
> > ===
> > RCS file: /cvs/src/lib/libevent/event.3,v
> > retrieving revision 1.54
> > diff -u -p -u -r1.54 event.3
> > --- event.3 26 Jul 2018 12:50:04 -  1.54
> > +++ event.3 22 Sep 2018 01:26:56 -
> > @@ -68,6 +68,7 @@
> >  .Nm bufferevent_enable ,
> >  .Nm bufferevent_disable ,
> >  .Nm bufferevent_settimeout ,
> > +.Nm bufferevent_setwatermark ,
> >  .Nm EVBUFFER_INPUT ,
> >  .Nm EVBUFFER_OUTPUT
> >  .Nd execute a function when a specific event occurs
> > @@ -156,6 +157,8 @@
> >  .Fn "bufferevent_disable" "struct bufferevent *bufev" "short event"
> >  .Ft void
> >  .Fn "bufferevent_settimeout" "struct bufferevent *bufev" "int 
> > timeout_read" "int timeout_write"
> > +.Ft void
> > +.Fn "bufferevent_setwatermark" "struct bufferevent *bufev" "short events" 
> > "size_t lowmark" "size_t highmark"
> >  .Ft "struct evbuffer *"
> >  .Fn "EVBUFFER_INPUT" "struct bufferevent *bufev"
> >  .Ft "struct evbuffer *"
> > @@ -492,10 +495,35 @@ and
> >  When read enabled the bufferevent will try to read from the file
> >  descriptor and call the read callback.
> >  The write callback is executed
> > -whenever the output buffer is drained below the write low watermark,
> > +whenever the output buffer is drained below the write
> > +.Fa "lowmark" ,
> >  which is
> >  .Va 0
> >  by default.
> > +.Pp
> > +The
> > +.Fn bufferevent_setwatermark
> > +function can set the the low and high watermarks
> 
> One `the' is enough.
> 
> > +for read and write events.
> > +.Fa "events"
> > +can be
> > +.Va EV_READ ,
> > +.Va EV_WRITE
> > +or both.
> > +When used with
> > +.Va EV_READ ,
> > +a bufferevent does not invoke the user read callback
> > +unless there is at least
> > +.Fa "lowmark"
> > +data in the buffer.
> > +If the read buffer is beyond
> > +.Fa "highmark" ,
> > +the bufferevent stops reading from the file descriptor.
> > +When used with
> > +.Va EV_WRITE,
> 
> Missing space between EV_WRITE and comma.
> 
> > +the user write callback is invoked whenever the buffered data
> > +falls below
> > +.Fa "lowmark" .
> >  .Pp
> >  The
> >  .Fn bufferevent_write

Re: ksh "clear-screen" editing command

[Nicholas Marriott]

Never mind, should have finished reading the later emails ;-).



On Sat, Jun 23, 2018 at 09:02:24AM +0100, Nicholas Marriott wrote:
> Hi
> 
> I think you should not pass NULL to the last argument of setupterm(),
> from terminfo(3):
> 
>   If errret is null, setupterm prints an error message upon finding an
>   error and exits.
> 
> 
> 
> On Sat, Jun 16, 2018 at 04:16:57PM -0600, Todd C. Miller wrote:
> > On Sat, 16 Jun 2018 14:50:40 +0200, Mark Kettenis wrote:
> > 
> > > To be honest, I find the whole idea of invoking an external program to
> > > clear the screen insane.
> > 
> > Linking with curses doesn't increase the size a huge amount since
> > we just need to query terminfo.
> > 
> > textdatabss dec hex
> > 529120  12584   57024   598728  922c8   /bin/ksh
> > 595671  21904   57024   674599  a4b27   ./obj/ksh
> > 
> >  - todd
> > 
> > Index: bin/ksh/Makefile
> > ===
> > RCS file: /cvs/src/bin/ksh/Makefile,v
> > retrieving revision 1.38
> > diff -u -p -u -r1.38 Makefile
> > --- bin/ksh/Makefile6 Jan 2018 16:28:58 -   1.38
> > +++ bin/ksh/Makefile16 Jun 2018 22:00:32 -
> > @@ -1,6 +1,9 @@
> >  #  $OpenBSD: Makefile,v 1.38 2018/01/06 16:28:58 millert Exp $
> >  
> >  PROG=  ksh
> > +DPADD+=${LIBCURSES}
> > +LDADD+=-lcurses
> > +
> >  SRCS=  alloc.c c_ksh.c c_sh.c c_test.c c_ulimit.c edit.c emacs.c 
> > eval.c \
> > exec.c expr.c history.c io.c jobs.c lex.c mail.c main.c \
> > misc.c path.c shf.c syn.c table.c trap.c tree.c tty.c var.c \
> > Index: bin/ksh/edit.c
> > ===
> > RCS file: /cvs/src/bin/ksh/edit.c,v
> > retrieving revision 1.65
> > diff -u -p -u -r1.65 edit.c
> > --- bin/ksh/edit.c  9 Apr 2018 17:53:36 -   1.65
> > +++ bin/ksh/edit.c  16 Jun 2018 22:09:17 -
> > @@ -138,10 +138,10 @@ x_flush(void)
> > shf_flush(shl_out);
> >  }
> >  
> > -void
> > +int
> >  x_putc(int c)
> >  {
> > -   shf_putc(c, shl_out);
> > +   return shf_putc(c, shl_out);
> >  }
> >  
> >  void
> > Index: bin/ksh/edit.h
> > ===
> > RCS file: /cvs/src/bin/ksh/edit.h,v
> > retrieving revision 1.11
> > diff -u -p -u -r1.11 edit.h
> > --- bin/ksh/edit.h  26 Jan 2016 17:39:31 -  1.11
> > +++ bin/ksh/edit.h  16 Jun 2018 22:09:27 -
> > @@ -37,7 +37,7 @@ extern X_chars edchars;
> >  /* edit.c */
> >  intx_getc(void);
> >  void   x_flush(void);
> > -void   x_putc(int);
> > +intx_putc(int);
> >  void   x_puts(const char *);
> >  bool   x_mode(bool);
> >  intpromptlen(const char *, const char **);
> > Index: bin/ksh/emacs.c
> > ===
> > RCS file: /cvs/src/bin/ksh/emacs.c,v
> > retrieving revision 1.84
> > diff -u -p -u -r1.84 emacs.c
> > --- bin/ksh/emacs.c 16 Jan 2018 17:17:18 -  1.84
> > +++ bin/ksh/emacs.c 16 Jun 2018 22:12:24 -
> > @@ -21,6 +21,10 @@
> >  #include 
> >  #include 
> >  #include 
> > +#ifndef SMALL
> > +# include 
> > +# include 
> > +#endif
> >  
> >  #include "sh.h"
> >  #include "edit.h"
> > @@ -28,6 +32,7 @@
> >  static Areaaedit;
> >  #defineAEDIT /* area for kill ring and macro defns */
> >  
> > +#undef CTRL
> >  #defineCTRL(x) ((x) == '?' ? 0x7F : (x) & 0x1F)/* 
> > ASCII */
> >  #defineUNCTRL(x)   ((x) == 0x7F ? '?' : (x) | 0x40)/* 
> > ASCII */
> >  
> > @@ -146,6 +151,7 @@ static int  isu8cont(unsigned char);
> >  /* proto's for keybindings */
> >  static int x_abort(int);
> >  static int x_beg_hist(int);
> > +static int x_clear_screen(int);
> >  static int x_comp_comm(int);
> >  static int x_comp_file(int);
> >  static int x_complete(int);
> > @@ -202,6 +208,7 @@ static int  x_debug_info(int);
> >  static const struct x_ftab x_ftab[] = {
> > { x_abort,  "abort",0 },
> > { x_beg_hist,   "beginning-of-history", 0 },
> > +   { x_clear_screen,   "clear-screen", 0 },
> > { x_comp_comm,  "comp

Re: ksh "clear-screen" editing command

[Nicholas Marriott]

Hi

I think you should not pass NULL to the last argument of setupterm(),
from terminfo(3):

  If errret is null, setupterm prints an error message upon finding an
  error and exits.
  


On Sat, Jun 16, 2018 at 04:16:57PM -0600, Todd C. Miller wrote:
> On Sat, 16 Jun 2018 14:50:40 +0200, Mark Kettenis wrote:
> 
> > To be honest, I find the whole idea of invoking an external program to
> > clear the screen insane.
> 
> Linking with curses doesn't increase the size a huge amount since
> we just need to query terminfo.
> 
> textdatabss dec hex
> 529120  12584   57024   598728  922c8   /bin/ksh
> 595671  21904   57024   674599  a4b27   ./obj/ksh
> 
>  - todd
> 
> Index: bin/ksh/Makefile
> ===
> RCS file: /cvs/src/bin/ksh/Makefile,v
> retrieving revision 1.38
> diff -u -p -u -r1.38 Makefile
> --- bin/ksh/Makefile  6 Jan 2018 16:28:58 -   1.38
> +++ bin/ksh/Makefile  16 Jun 2018 22:00:32 -
> @@ -1,6 +1,9 @@
>  #$OpenBSD: Makefile,v 1.38 2018/01/06 16:28:58 millert Exp $
>  
>  PROG=ksh
> +DPADD+=  ${LIBCURSES}
> +LDADD+=  -lcurses
> +
>  SRCS=alloc.c c_ksh.c c_sh.c c_test.c c_ulimit.c edit.c emacs.c 
> eval.c \
>   exec.c expr.c history.c io.c jobs.c lex.c mail.c main.c \
>   misc.c path.c shf.c syn.c table.c trap.c tree.c tty.c var.c \
> Index: bin/ksh/edit.c
> ===
> RCS file: /cvs/src/bin/ksh/edit.c,v
> retrieving revision 1.65
> diff -u -p -u -r1.65 edit.c
> --- bin/ksh/edit.c9 Apr 2018 17:53:36 -   1.65
> +++ bin/ksh/edit.c16 Jun 2018 22:09:17 -
> @@ -138,10 +138,10 @@ x_flush(void)
>   shf_flush(shl_out);
>  }
>  
> -void
> +int
>  x_putc(int c)
>  {
> - shf_putc(c, shl_out);
> + return shf_putc(c, shl_out);
>  }
>  
>  void
> Index: bin/ksh/edit.h
> ===
> RCS file: /cvs/src/bin/ksh/edit.h,v
> retrieving revision 1.11
> diff -u -p -u -r1.11 edit.h
> --- bin/ksh/edit.h26 Jan 2016 17:39:31 -  1.11
> +++ bin/ksh/edit.h16 Jun 2018 22:09:27 -
> @@ -37,7 +37,7 @@ extern X_chars edchars;
>  /* edit.c */
>  int  x_getc(void);
>  void x_flush(void);
> -void x_putc(int);
> +int  x_putc(int);
>  void x_puts(const char *);
>  bool x_mode(bool);
>  int  promptlen(const char *, const char **);
> Index: bin/ksh/emacs.c
> ===
> RCS file: /cvs/src/bin/ksh/emacs.c,v
> retrieving revision 1.84
> diff -u -p -u -r1.84 emacs.c
> --- bin/ksh/emacs.c   16 Jan 2018 17:17:18 -  1.84
> +++ bin/ksh/emacs.c   16 Jun 2018 22:12:24 -
> @@ -21,6 +21,10 @@
>  #include 
>  #include 
>  #include 
> +#ifndef SMALL
> +# include 
> +# include 
> +#endif
>  
>  #include "sh.h"
>  #include "edit.h"
> @@ -28,6 +32,7 @@
>  static   Areaaedit;
>  #define  AEDIT /* area for kill ring and macro defns */
>  
> +#undef CTRL
>  #define  CTRL(x) ((x) == '?' ? 0x7F : (x) & 0x1F)/* 
> ASCII */
>  #define  UNCTRL(x)   ((x) == 0x7F ? '?' : (x) | 0x40)/* 
> ASCII */
>  
> @@ -146,6 +151,7 @@ static intisu8cont(unsigned char);
>  /* proto's for keybindings */
>  static int   x_abort(int);
>  static int   x_beg_hist(int);
> +static int   x_clear_screen(int);
>  static int   x_comp_comm(int);
>  static int   x_comp_file(int);
>  static int   x_complete(int);
> @@ -202,6 +208,7 @@ static intx_debug_info(int);
>  static const struct x_ftab x_ftab[] = {
>   { x_abort,  "abort",0 },
>   { x_beg_hist,   "beginning-of-history", 0 },
> + { x_clear_screen,   "clear-screen", 0 },
>   { x_comp_comm,  "complete-command", 0 },
>   { x_comp_file,  "complete-file",0 },
>   { x_complete,   "complete", 0 },
> @@ -1004,12 +1011,19 @@ x_draw_line(int c)
>  {
>   x_redraw(-1);
>   return KSTD;
> +}
>  
> +static int
> +x_clear_screen(int c)
> +{
> + x_redraw(-2);
> + return KSTD;
>  }
>  
> -/* Redraw (part of) the line.  If limit is < 0, the everything is redrawn
> - * on a NEW line, otherwise limit is the screen column up to which needs
> - * redrawing.
> +/* Redraw (part of) the line.
> + * A non-negative limit is the screen column up to which needs
> + * redrawing. A limit of -1 redraws on a new line, while a limit
> + * of -2 (attempts to) clear the screen.
>   */
>  static void
>  x_redraw(int limit)
> @@ -1018,9 +1032,23 @@ x_redraw(int limit)
>   char*cp;
>  
>   x_adj_ok = 0;
> - if (limit == -1)
> + if (limit == -2) {
> + int ret = -1;
> +#ifndef SMALL
> + char *str, *term = str_val(global("TERM"));
> + if (term && *term) {
> + setupterm(term, 1, 

Re: fix file(1) memory leak

[Nicholas Marriott]

I think this is better?

Index: magic-test.c
===
RCS file: /cvs/src/usr.bin/file/magic-test.c,v
retrieving revision 1.25
diff -u -p -r1.25 magic-test.c
--- magic-test.c18 Apr 2017 14:16:48 -  1.25
+++ magic-test.c14 Jun 2018 07:16:41 -
@@ -1404,10 +1404,10 @@ magic_test(struct magic *m, const void *
if (*ms.out != '\0') {
if (flags & MAGIC_TEST_MIME) {
if (ms.mimetype != NULL)
-   return (xstrdup(ms.mimetype));
+   return (ms.mimetype);
return (NULL);
}
-   return (xstrdup(ms.out));
+   return (ms.out);
}
return (NULL);
 }



On Wed, Jun 13, 2018 at 11:00:58PM -0400, Bryan Steele wrote:
> magic_test returns a xstrdup'd string, which was then being xstrdup'd
> again without freeing the original copy (leaking memory).
> 
> casts added to avoid clang warning
>   warning: assigning to 'char *' from 'const char *' discards qualifiers
>   [-Wincompatible-pointer-types-discards-qualifiers]
> inf->result = s;
> 
> ok?
> 
> -Bryan.
> 
> Index: file/file.c
> ===
> RCS file: /cvs/src/usr.bin/file/file.c,v
> retrieving revision 1.66
> diff -u -p -u -r1.66 file.c
> --- file/file.c   15 Jan 2018 19:45:51 -  1.66
> +++ file/file.c   14 Jun 2018 02:55:32 -
> @@ -603,7 +603,7 @@ try_text(struct input_file *inf)
>  
>   s = magic_test(inf->m, inf->base, inf->size, flags);
>   if (s != NULL) {
> - inf->result = xstrdup(s);
> + inf->result = (char *)s;
>   return (1);
>   }
>  
> @@ -635,7 +635,7 @@ try_magic(struct input_file *inf)
>  
>   s = magic_test(inf->m, inf->base, inf->size, flags);
>   if (s != NULL) {
> - inf->result = xstrdup(s);
> + inf->result = (char *)s;
>   return (1);
>   }
>   return (0);
> 

Re: tmux fix when renaming session with no client attached

[Nicholas Marriott]

Fixed, thanks!




On Tue, Apr 10, 2018 at 04:31:38PM -0700, Ryan Freeman wrote:
> Hey,
> 
> After upgrading OpenBSD 6.2 -> 6.3, a program I am building for $DAYJOB
> started malfunctioning.
> 
> Basically, we use tmux to manage running sessions of this program, which
> does automated work on things with console ports.
> 
> The session is started in a detached state with temp session name derived
> from a date(1) stamp (date +%s) plus a few random digits:
> 
> $ tmux -S /var/someprog/default new-session -s tmp1234456789
> 
> After the program is running, a command is executed by the running
> program to give it a proper session id, fetched from a database:
> 
> exec /usr/bin/tmux -S /var/someprog/default rename-session $SESSIONID
> 
> Tmux in OpenBSD 6.2 worked a-okay, tmux in 6.3 would return an error
> indicating no current client.  This seems similar to here:
> 
> https://marc.info/?l=openbsd-cvs=152183263526828=2
> 
> I took a stab at fixing this in cmd-rename-session.c like was done
> for cmd-rename-window.c, and the rebuilt tmux seems happy renaming
> sessions from a program running within it again.
> 
> OK?  Flames? :-)
> 
> -ryan
> 
> Index: cmd-rename-session.c
> ===
> RCS file: /cvs/src/usr.bin/tmux/cmd-rename-session.c,v
> retrieving revision 1.27
> diff -u -p -r1.27 cmd-rename-session.c
> --- cmd-rename-session.c1 Mar 2018 12:53:08 -   1.27
> +++ cmd-rename-session.c10 Apr 2018 23:21:52 -
> @@ -47,7 +47,7 @@ static enum cmd_retval
>  cmd_rename_session_exec(struct cmd *self, struct cmdq_item *item)
>  {
> struct args *args = self->args;
> -   struct client   *c = cmd_find_client(item, NULL, 0);
> +   struct client   *c = cmd_find_client(item, NULL, 1);
> struct session  *s = item->target.s;
> char*newname;
> 

Re: less(1): `!' command

[Nicholas Marriott]

I don't think we should bring ! back.

I wanted to remove v and | (and some other stuff) shortly afterwards, but
several people objected.

I did suggest having a lightweight less in base for most people and adding
the full upstream less to ports for the stuff we don't want to maintain
(like we do for eg libevent) but other people didn't like that idea.



On 17 December 2017 at 15:48, kshe  wrote:

> On Sat, 16 Dec 2017 21:52:44 +, Theo de Raadt wrote:
> > > On Sat, 16 Dec 2017 19:39:27 +, Theo de Raadt wrote:
> > > > > On Sat, 16 Dec 2017 18:13:16 +, Jiri B wrote:
> > > > > > On Sat, Dec 16, 2017 at 04:55:44PM +, kshe wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > Would a patch to bring back the `!' command to less(1) be
> accepted?  The
> > > > > > > commit message for its removal explains that ^Z should be used
> instead,
> > > > > > > but that obviously does not work if less(1) is run from
> something else
> > > > > > > than an interactive shell, for example when reading manual
> pages from a
> > > > > > > vi(1) instance spawned directly by `xterm -e vi' in a window
> manager or
> > > > > > > by `neww vi' in a tmux(1) session.
> > > > > >
> > > > > > Why should less be able to spawn another programs? This would
> undermine
> > > > > > all pledge work.
> > > > >
> > > > > Because of at least `v' and `|', less(1) already is able to invoke
> > > > > arbitrary programs, and accordingly needs the "proc exec" promise,
> so
> > > > > bringing `!' back would not change anything from a security
> perspective
> > > > > (otherwise, I would obviously not have made such a proposition).
> > > > >
> > > > > In fact, technically, what I want to do is still currently
> possible:
> > > > > from any less(1) instance, one may use `v' to invoke vi(1), and
> then use
> > > > > vi(1)'s own `!' command as desired.  So the functionality of `!' is
> > > > > still there; it was only made more difficult to reach for no
> apparent
> > > > > reason.
> > > >
> > > > No apparent reason?
> > > >
> > > > Good you have an opinion.  I have a different opinion: We should look
> > > > for rarely used functionality and gut it.
> > >
> > > I completely agree, and I also completely agree with the rest of what
> > > you said.  However, in this particular case, the functionality of `!'
> is
> > > still fully (albeit indirectly) accessible, as shown above, and this is
> > > why its deletion, when not immediately followed by that of `|' and `v',
> > > made little sense for me.
> >
> > Oh, so you don't agree.  Or do you.  I can't tell.  You haven't made up
> > your mind enough to have a final position?
>
> In the case of less(1), the underlying functionality of `!' (invoking
> arbitrary programs) has not been removed at all, as `!' itself was only
> one way amongst others of doing that.  Therefore, I would have prefered
> that such an endeavour be conducted in steps at least as large as a
> pledge(2) category.  You may say this is absolutist, but, in the end,
> users might actually be more inclined to accept such removals if they
> come with, and thus are justified by, a real and immediate security
> benefit, like stricter pledge(2) promises, rather than some vague
> theoretical explanation about the global state of their software
> environment.
>
> > [...]
> >
> > > May I go ahead and prepare a patch to remove "proc exec" entirely?
> >
> > Sure you could try, and see who freaks out.  Exactly what the plan was
> > all along.
>
> The minimal diff below does that.  If it is accepted, further cleanups
> would need to follow (in particular, removing a few unused variables and
> functions), and of course the manual would also need some adjustments.
>
> Index: cmd.h
> ===
> RCS file: /cvs/src/usr.bin/less/cmd.h,v
> retrieving revision 1.10
> diff -u -p -r1.10 cmd.h
> --- cmd.h   6 Nov 2015 15:58:01 -   1.10
> +++ cmd.h   17 Dec 2017 12:23:00 -
> @@ -42,12 +42,12 @@
>  #defineA_FF_LINE   29
>  #defineA_BF_LINE   30
>  #defineA_VERSION   31
> -#defineA_VISUAL32
> +/* 32 unused */
>  #defineA_F_WINDOW  33
>  #defineA_B_WINDOW  34
>  #defineA_F_BRACKET 35
>  #defineA_B_BRACKET 36
> -#defineA_PIPE  37
> +/* 37 unused */
>  #defineA_INDEX_FILE38
>  #defineA_UNDO_SEARCH   39
>  #defineA_FF_SCREEN 40
> Index: command.c
> ===
> RCS file: /cvs/src/usr.bin/less/command.c,v
> retrieving revision 1.31
> diff -u -p -r1.31 command.c
> --- command.c   12 Jan 2017 20:32:01 -  1.31
> +++ command.c   17 Dec 2017 12:23:00 -
> @@ -241,12 +241,6 @@ exec_mca(void)
> /* If tag structure is loaded then clean it up. */
> 

Re: document capability dc in remote(5)

[Nicholas Marriott]

IIRC dc was removed and then added back.

ok nicm



On Mon, Oct 30, 2017 at 11:44:28PM +0100, Remi Locherer wrote:
> Hi,
> 
> in 2015 remote(5) was trimmed down when tip was removed. It looks like
> documentation for capability "dc" was also removed by accident. cu(1) still
> supports this (src/usr.bin/cu/cu.c):
> 
> 381 if (is_direct == -1 && cgetcap(cp, "dc", ':') != NULL)
> 382 is_direct = 1;
> 
> Below patch brings back documentation for "dc".
> 
> ok?
> 
> Remi
> 
> 
> Index: remote.5
> ===
> RCS file: /cvs/src/share/man/man5/remote.5,v
> retrieving revision 1.26
> diff -u -p -r1.26 remote.5
> --- remote.5  21 Sep 2017 07:51:43 -  1.26
> +++ remote.5  30 Oct 2017 22:39:44 -
> @@ -75,6 +75,12 @@ The baud rate used in establishing
>  a connection to the remote host.
>  This is a decimal number.
>  The default baud rate is 9600 baud.
> +.It Sy \
> +(bool)
> +This host is directly connected, and
> +.Xr cu 1
> +should not expect carrier detect to be high, nor should it exit if
> +carrier detect drops.
>  .It Sy \
>  (str)
>  Device to open to establish a connection.

Re: magic.5: Add missing types

[Nicholas Marriott]

Hi

On Thu, Jun 29, 2017 at 09:29:57PM +0200, Klemens Nanni wrote:
> While reading file(1)'s code in #openbsd-daily mulander noted that the
> 'name' and 'use' types were missing from magic(5).
> 
> I'm not entirely sure yet whether this is complete, so here's what I
> did:
> 
> magic(5) provided by devel/magic documents version 5.31 while base's
> magic(5) is at 4.24. Here are the types found in 5.31 but not ours as
> well as those of the missing ones that are actually implemented but
> undocumented as of now:
> 
>   $ grep -i "TYPE_($(grep -F 'It Dv' $(man -w magic) |
>   cut -d' ' -f3 | sort | uniq -u | paste -sd\| - |
>   tee /dev/stderr))" magic.h
>   beid3|beqwdate|clear|indirect|leid3|leqwdate|name|qwdate|use
>   MAGIC_TYPE_CLEAR,
>   MAGIC_TYPE_NAME,
>   MAGIC_TYPE_USE,
> 
> What about the current version being 4.21? We're clearly ahead of this,
> it seems magic(5) wasn't updated when nicm@ reimplemented things.
> 
> This patch documents the respective types.

Did you copy this text from upstream or write it?

Some comments inline.

> Feedback/OK?
> 
> Index: magic.5
> ===
> RCS file: /cvs/src/usr.bin/file/magic.5,v
> retrieving revision 1.17
> diff -u -p -r1.17 magic.5
> --- magic.5   24 Apr 2016 07:02:07 -  1.17
> +++ magic.5   29 Jun 2017 17:41:56 -
> @@ -218,6 +218,31 @@ This is intended to be used with the tes
> .Em x
> (which is always true) and a message that is to be used if there are
> no other matches.
> +.It Dv clear
> +This test is always true and clears the match flag for that continuation
> +level.

I would just say "that level" or "the test's level" not "that
continuation level" because there is no previous mention of
"continuation".

> +It is intended to be used with the default test.
> +.It Dv name
> +Define a
> +.Dq named

I don't think quotation marks are necessary here.

> +magic instance that can be called from another
> +.Dv use
> +magic entry, like a subroutine call.
> +Named instance direct magic offsets are relative to the offset of the
> +previous matched entry, but indirect offsets are relative to the
> +beginning of the file as usual.
> +Named magic entries always match.
> +.It Dv use
> +Recursively call the named magic starting from the current offset.
> +If the name of the referenced begins with a

"the referenced instance" would be better than "the referenced".

> +.Dv ^
> +then the endianness of the magic is switched; if the magic mentioned
> +.Dv leshort
> +for example,
> +it is treated as
> +.Dv beshort
> +and vice versa.
> +This is useful to avoid duplicating the rules for different endianness.
> .El
> .Pp
> Each top-level magic pattern (see below for an explanation of levels)

Re: file: replace fgetln with getline(3)

[Nicholas Marriott]

Brilliant, looks good to me, ok nicm



On Sun, Jul 02, 2017 at 06:23:35PM +0200, Ingo Schwarze wrote:
> Hi Nic,
> 
> Nicholas Marriott wrote on Sun, Jul 02, 2017 at 08:48:28AM +0100:
> 
> > Possibly fgetln(3) CAVEATS could benefit from some of what you have
> > written here, although it is all in the manual already just in several
> > places.
> 
> Inspecting the manual page again, i don't think it needs longer
> text.  I merely provided more detailed explanations here because i
> got the wrong impression that you were in denial of the potential
> dangers of fgetln(3) - though probably i should have relized how
> implausible that impression was, but anyway.
> 
> > Or at least a link to getline(3)?
> 
> Indeed.  I think that ought to take the form of a mild deprecation
> notice ot the top, though i don't see a need for bold face or any
> kind of strong wording.
> 
> While here, let's delete the irrelevant .Xr to putc(3) and instead
> link to fgetc(3) - which is relevant because it is a fundamental
> stdio I/O function and may invalidate the line pointer.
> 
> Also, add two lines to the example code for error checking.
> The example is already so long that the two lines don't make it
> much worse, and forgetting error checking is such a common oversight
> for this function that it should not be left out of an example even
> if the example focusses on something else.
> 
> OK?
>   Ingo
> 
> 
> Index: fgetln.3
> ===
> RCS file: /cvs/src/lib/libc/stdio/fgetln.3,v
> retrieving revision 1.17
> diff -u -r1.17 fgetln.3
> --- fgetln.3  13 Jan 2015 14:02:30 -  1.17
> +++ fgetln.3  2 Jul 2017 16:03:29 -
> @@ -38,6 +38,11 @@
>  .Ft char *
>  .Fn fgetln "FILE *stream" "size_t *len"
>  .Sh DESCRIPTION
> +Using this function is error-prone in multiple ways;
> +consider using the safer and more portable function
> +.Xr getline 3
> +instead.
> +.Pp
>  The
>  .Fn fgetln
>  function returns a pointer to the next line from the stream referenced by
> @@ -103,10 +108,11 @@
>  .Xr realloc 3 .
>  .Sh SEE ALSO
>  .Xr ferror 3 ,
> +.Xr fgetc 3 ,
>  .Xr fgets 3 ,
>  .Xr fopen 3 ,
>  .Xr fparseln 3 ,
> -.Xr putc 3
> +.Xr getline 3
>  .Sh HISTORY
>  The
>  .Fn fgetln
> @@ -139,4 +145,6 @@
>   printf("%s\en", buf);
>   }
>   free(lbuf);
> + if (ferror(fp))
> + err(1, "fgetln");
>  .Ed

Re: file: replace fgetln with getline(3)

[Nicholas Marriott]

Possibly fgetln(3) CAVEATS could benefit from some of what you have
written here, although it is all in the manual already just in several
places.

Or at least a link to getline(3)?



On Sun, Jul 02, 2017 at 12:26:19AM +0200, Ingo Schwarze wrote:
> Hi,
> 
> Nicholas Marriott wrote on Sat, Jul 01, 2017 at 09:48:08PM +0100:
> 
> > Safer how?
> 
>  1. fgetln(3) returns a char array that is not NUL-terminated.
> That is prone to buffer overrun bugs in general.
> 
> getline(3) always returns proper NUL-terminated C strings.
> 
>  2. The array returned from fgetln(3) contains a trailing '\n' in
> most cases, but if the stream ends without it, there won't be
> one, which aggravates the danger of item 1: if a careless
> programmer reads the buffer until '\n', they will get away with
> it on all valid text input files, so the problem is likely to
> go undetected during testing, but they may yet be vulnerable
> to buffer overrun on crafted input lacking the trailing newline.
> 
> With getline(3), lack of the trailing '\n' is much less of an
> issue because people will likely look for the terminating '\0'
> in the first place, not for the terminating '\n'.
> 
>  3. For this reason, the fgetln(3) manual page contains a lengthy
> CAVEATS section recommending a complicated ten-line idiom to
> NUL-terminate the buffer.  In general, if something requires
> a ten-line idiom, it can hardly be called safe.  Adding the
> missing two lines for error checking at the end, the complete
> example is essentially 19 lines.
> 
> EXAMPLES in fgetline(3) contains a code snipped doing about the
> same, and even adding two more lines to strip '\n', it is only
> 10 lines long including full initialization and error checking.
> So that's half the code, much more intuitive, and much simpler,
> without any manual malloc(3) or memcpy(3) or manually assigning
> '\0' to some position calculated from pointer arithmetics,
> which we all know is prone to off-by-one.
> 
>  4. With fgetln(3), the returned pointer becomes invalid after the
> next I/O operation on the stream.  So you say line = fgetln(..),
> do something else that you consider unrelated because you never
> use "line" in it, and the next time you access "line", boom.
> That can be very surprising.  In general, library interfaces
> passing out pointers to library-owned internal memory for the
> user to mess with tend to be somewhat risky.
> 
> By contrast, getline(3) copies the data to a dedicated buffer,
> which won't become invalid on I/O.  So the next time you use
> it, it is guaranteed to be still valid, even if that is much
> later and the file in question has long been closed.  Of course,
> if you pass the same buffer to getline(3) again, then after
> that, it will contain the *next* line; but that is not surprising
> at all, because that is exactly what you explicitly asked for.
> 
>  5. The fgetln(3) manual says that you are allowed to modify the
> returned buffer, but item 4 still applies, which aggravates the
> danger inherent in item 4.  Why would you modify the buffer,
> unless you want to do something else after that and then, yet
> later, look at your changes again?  But then you must be *very*
> careful to not do I/O on that file descriptor while doing
> the "something else", or you are in for a surprise.
> 
> getline(3) returns an allocated buffer for the user to free(3).
> That's an easy to understand operation mode with clearly
> assigned responsibilies, and it is so obvious that the user
> can alter the buffer - after all, it's their own buffer -
> that the manual doesn't even need to mention it.
> 
> Your code follows the CAVEATS section in the fgetln(3) manual quite
> closely, and i would have been quite surprised if you had screwed
> such a thing up.
> 
> But with most other people, i would have felt somewhat worried
> about their code if they had asked me "safer how?" *after* using
> fgetln(3)...  >:-o
> 
> > I don't mind too much, but you will also need to initialize size
> > to 0 (it is currently uninitialized) before calling getline().
> 
> Ouch.  No interface can be so simple that there is no way to
> misuse it dangerously, even if it sets fewer traps for the
> unwary than fgetln(3).
> 
> >> Index: magic-load.c
> >> ===
> >> RCS file: /cvs/src/usr.bin/file/magic-load.c,v
> >> retrieving revision 1.25
> >> diff -u -p -r1.25 magic-load.c
> >> --- magic-load.c  

Re: file: replace fgetln with getline(3)

[Nicholas Marriott]

I'm not wild about magic_load calling err(), but it is probably more
pain to do nicely than it is worth it for now. This is fine, ok nicm





On Sun, Jul 02, 2017 at 01:03:13AM -0400, Bryan Steele wrote:
> On Sat, Jul 01, 2017 at 08:48:18PM -0400, Bryan Steele wrote:
> > On Sat, Jul 01, 2017 at 06:53:29PM -0400, Bryan Steele wrote:
> > > On Sun, Jul 02, 2017 at 12:26:19AM +0200, Ingo Schwarze wrote:
> > > > I don't see a need for two char * variables here, that's an artifact
> > > > of the typical fgetln(3) complications.  Just
> > > > 
> > > > char*line = NULL;
> > > > size_t   linesize = 0;
> > > > ssize_t  slen;
> > > > 
> > > > while ((slen = getline(, , f)) != -1) {
> > > > if (line[slen - 1] == '\n')
> > > > /* etc. */
> > > > }
> > > > free(line);
> > > > if (ferror(f))
> > > > err(1, "%s", path);
> > > > 
> > > > ought to suffice.
> > > > 
> > > > Yours,
> > > >   Ingo
> > > 
> > > Hmm, there's a Segmentation fault that I can't seems to narrow
> > > down with the change you suggest. That's odd..
> > 
> > This appears to be because the loop passes around the line pointer
> > and increments it, fixing that would be a lager diff.. but I'd
> > still appreciate a cluestick if I missed something.
> > 
> > -Bryan.
> 
> Here's the final diff incorporating all the feedback recieved, except
> for eliminating the tmp variable as it is used to keep track of the
> getline pointer which gets munged each iteration.
> 
> -Bryan.
> 
> Index: magic-load.c
> ===
> RCS file: /cvs/src/usr.bin/file/magic-load.c,v
> retrieving revision 1.25
> diff -u -p -r1.25 magic-load.c
> --- magic-load.c  1 Jul 2017 14:34:29 -   1.25
> +++ magic-load.c  2 Jul 2017 04:58:52 -
> @@ -19,6 +19,7 @@
>  #include 
>  
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -1071,6 +1072,7 @@ magic_load(FILE *f, const char *path, in
>   struct magic_line   *ml = NULL, *parent, *parent0;
>   char*line, *tmp;
>   size_t   size;
> + ssize_t  slen;
>   u_intat, level, n, i;
>  
>   m = xcalloc(1, sizeof *m);
> @@ -1084,15 +1086,12 @@ magic_load(FILE *f, const char *path, in
>  
>   at = 0;
>   tmp = NULL;
> - while ((line = fgetln(f, ))) {
> - if (line[size - 1] == '\n')
> - line[size - 1] = '\0';
> - else {
> - tmp = xmalloc(size + 1);
> - memcpy(tmp, line, size);
> - tmp[size] = '\0';
> - line = tmp;
> - }
> + size = 0;
> + while ((slen = getline(, , f)) != -1) {
> + line = tmp;
> + if (line[slen - 1] == '\n')
> + line[slen - 1] = '\0';
> +
>   at++;
>  
>   while (isspace((u_char)*line))
> @@ -1168,6 +1167,8 @@ magic_load(FILE *f, const char *path, in
>   parent0 = ml;
>   }
>   free(tmp);
> + if (ferror(f))
> + err(1, "%s", path);
>  
>   return (m);
>  }
> 

Re: file: replace fgetln with getline(3)

[Nicholas Marriott]

Right, I wasn't clear - I didn't mean "why is getline better than
fgetln", I was asking "why do you want to change fgetln to getline in
this code".

It is fine if the answer is "I am worried that another developer might
copy or modify the code and mess it up" rather than "the way file is
using fgetln is not safe" but that was not clear to me.




On Sun, Jul 02, 2017 at 12:26:19AM +0200, Ingo Schwarze wrote:
> Hi,
> 
> Nicholas Marriott wrote on Sat, Jul 01, 2017 at 09:48:08PM +0100:
> 
> > Safer how?
> 
>  1. fgetln(3) returns a char array that is not NUL-terminated.
> That is prone to buffer overrun bugs in general.
> 
> getline(3) always returns proper NUL-terminated C strings.
> 
>  2. The array returned from fgetln(3) contains a trailing '\n' in
> most cases, but if the stream ends without it, there won't be
> one, which aggravates the danger of item 1: if a careless
> programmer reads the buffer until '\n', they will get away with
> it on all valid text input files, so the problem is likely to
> go undetected during testing, but they may yet be vulnerable
> to buffer overrun on crafted input lacking the trailing newline.
> 
> With getline(3), lack of the trailing '\n' is much less of an
> issue because people will likely look for the terminating '\0'
> in the first place, not for the terminating '\n'.
> 
>  3. For this reason, the fgetln(3) manual page contains a lengthy
> CAVEATS section recommending a complicated ten-line idiom to
> NUL-terminate the buffer.  In general, if something requires
> a ten-line idiom, it can hardly be called safe.  Adding the
> missing two lines for error checking at the end, the complete
> example is essentially 19 lines.
> 
> EXAMPLES in fgetline(3) contains a code snipped doing about the
> same, and even adding two more lines to strip '\n', it is only
> 10 lines long including full initialization and error checking.
> So that's half the code, much more intuitive, and much simpler,
> without any manual malloc(3) or memcpy(3) or manually assigning
> '\0' to some position calculated from pointer arithmetics,
> which we all know is prone to off-by-one.
> 
>  4. With fgetln(3), the returned pointer becomes invalid after the
> next I/O operation on the stream.  So you say line = fgetln(..),
> do something else that you consider unrelated because you never
> use "line" in it, and the next time you access "line", boom.
> That can be very surprising.  In general, library interfaces
> passing out pointers to library-owned internal memory for the
> user to mess with tend to be somewhat risky.
> 
> By contrast, getline(3) copies the data to a dedicated buffer,
> which won't become invalid on I/O.  So the next time you use
> it, it is guaranteed to be still valid, even if that is much
> later and the file in question has long been closed.  Of course,
> if you pass the same buffer to getline(3) again, then after
> that, it will contain the *next* line; but that is not surprising
> at all, because that is exactly what you explicitly asked for.
> 
>  5. The fgetln(3) manual says that you are allowed to modify the
> returned buffer, but item 4 still applies, which aggravates the
> danger inherent in item 4.  Why would you modify the buffer,
> unless you want to do something else after that and then, yet
> later, look at your changes again?  But then you must be *very*
> careful to not do I/O on that file descriptor while doing
> the "something else", or you are in for a surprise.
> 
> getline(3) returns an allocated buffer for the user to free(3).
> That's an easy to understand operation mode with clearly
> assigned responsibilies, and it is so obvious that the user
> can alter the buffer - after all, it's their own buffer -
> that the manual doesn't even need to mention it.
> 
> Your code follows the CAVEATS section in the fgetln(3) manual quite
> closely, and i would have been quite surprised if you had screwed
> such a thing up.
> 
> But with most other people, i would have felt somewhat worried
> about their code if they had asked me "safer how?" *after* using
> fgetln(3)...  >:-o
> 
> > I don't mind too much, but you will also need to initialize size
> > to 0 (it is currently uninitialized) before calling getline().
> 
> Ouch.  No interface can be so simple that there is no way to
> misuse it dangerously, even if it sets fewer traps for the
> unwary than fgetln(3).
> 
> >> Index: magic-load.c
> >> =

Re: file: replace fgetln with getline(3)

[Nicholas Marriott]

Right but we are already taking that into account, that is not a problem
here. Anyway, change it if you like, it is shorter anyway.


On Sat, Jul 01, 2017 at 05:03:41PM -0400, Bryan Steele wrote:
> On Sat, Jul 01, 2017 at 09:48:08PM +0100, Nicholas Marriott wrote:
> > Safer how?
> 
> I believe that's in reference to the CAVEATS mentioned in fgetln(3).
> 
> http://man.openbsd.org/fgetln.3#CAVEATS
> 
> getline(3) returns C strings.
> 
> > I don't mind too much, but you will also need to initialize size to 0
> > (it is currently uninitialized) before calling getline().
> > 
> > On Sat, Jul 01, 2017 at 04:35:39PM -0400, Bryan Steele wrote:
> > > On Sat, Jul 01, 2017 at 09:27:47PM +0100, Nicholas Marriott wrote:
> > > > 
> > > > What is the point of doing this? fgetln works fine.
> > > > 
> > > 
> > > There's been other examples of commits replacing fgetln for getline,
> > > while portability isn't a major concern, it's been suggested that
> > > getline may be safer.
> > > 
> > > http://marc.info/?l=openbsd-cvs=147231601627483=2
> > > 
> > > No worries if you want to keep it.
> > > 
> > > > 
> > > > 
> > > > On Sat, Jul 01, 2017 at 02:47:25PM -0400, Bryan Steele wrote:
> > > > > Seems to work, does this look right?
> > > > > 
> > > > > -Bryan.
> > > > > 
> > > > > Index: magic-load.c
> > > > > ===
> > > > > RCS file: /cvs/src/usr.bin/file/magic-load.c,v
> > > > > retrieving revision 1.25
> > > > > diff -u -p -r1.25 magic-load.c
> > > > > --- magic-load.c  1 Jul 2017 14:34:29 -   1.25
> > > > > +++ magic-load.c  1 Jul 2017 18:44:42 -
> > > > > @@ -1071,6 +1071,7 @@ magic_load(FILE *f, const char *path, in
> > > > >   struct magic_line   *ml = NULL, *parent, *parent0;
> > > > >   char*line, *tmp;
> > > > >   size_t   size;
> > > > > + ssize_t  slen;
> > > > >   u_intat, level, n, i;
> > > > >  
> > > > >   m = xcalloc(1, sizeof *m);
> > > > > @@ -1084,15 +1085,11 @@ magic_load(FILE *f, const char *path, in
> > > > >  
> > > > >   at = 0;
> > > > >   tmp = NULL;
> > > > > - while ((line = fgetln(f, ))) {
> > > > > - if (line[size - 1] == '\n')
> > > > > - line[size - 1] = '\0';
> > > > > - else {
> > > > > - tmp = xmalloc(size + 1);
> > > > > - memcpy(tmp, line, size);
> > > > > - tmp[size] = '\0';
> > > > > - line = tmp;
> > > > > - }
> > > > > + while ((slen = getline(, , f)) != -1) {
> > > > > + line = tmp;
> > > > > + if (line[slen - 1] == '\n')
> > > > > + line[slen - 1] = '\0';
> > > > > +
> > > > >   at++;
> > > > >  
> > > > >   while (isspace((u_char)*line))
> > > > 

Re: file: replace fgetln with getline(3)

[Nicholas Marriott]

Safer how?

I don't mind too much, but you will also need to initialize size to 0
(it is currently uninitialized) before calling getline().





On Sat, Jul 01, 2017 at 04:35:39PM -0400, Bryan Steele wrote:
> On Sat, Jul 01, 2017 at 09:27:47PM +0100, Nicholas Marriott wrote:
> > 
> > What is the point of doing this? fgetln works fine.
> > 
> 
> There's been other examples of commits replacing fgetln for getline,
> while portability isn't a major concern, it's been suggested that
> getline may be safer.
> 
> http://marc.info/?l=openbsd-cvs=147231601627483=2
> 
> No worries if you want to keep it.
> 
> > 
> > 
> > On Sat, Jul 01, 2017 at 02:47:25PM -0400, Bryan Steele wrote:
> > > Seems to work, does this look right?
> > > 
> > > -Bryan.
> > > 
> > > Index: magic-load.c
> > > ===
> > > RCS file: /cvs/src/usr.bin/file/magic-load.c,v
> > > retrieving revision 1.25
> > > diff -u -p -r1.25 magic-load.c
> > > --- magic-load.c  1 Jul 2017 14:34:29 -   1.25
> > > +++ magic-load.c  1 Jul 2017 18:44:42 -
> > > @@ -1071,6 +1071,7 @@ magic_load(FILE *f, const char *path, in
> > >   struct magic_line   *ml = NULL, *parent, *parent0;
> > >   char*line, *tmp;
> > >   size_t   size;
> > > + ssize_t  slen;
> > >   u_intat, level, n, i;
> > >  
> > >   m = xcalloc(1, sizeof *m);
> > > @@ -1084,15 +1085,11 @@ magic_load(FILE *f, const char *path, in
> > >  
> > >   at = 0;
> > >   tmp = NULL;
> > > - while ((line = fgetln(f, ))) {
> > > - if (line[size - 1] == '\n')
> > > - line[size - 1] = '\0';
> > > - else {
> > > - tmp = xmalloc(size + 1);
> > > - memcpy(tmp, line, size);
> > > - tmp[size] = '\0';
> > > - line = tmp;
> > > - }
> > > + while ((slen = getline(, , f)) != -1) {
> > > + line = tmp;
> > > + if (line[slen - 1] == '\n')
> > > + line[slen - 1] = '\0';
> > > +
> > >   at++;
> > >  
> > >   while (isspace((u_char)*line))
> > 

Re: file: close fd after test_file

[Nicholas Marriott]

I'm not sure I see much point in this but do it if you like. It will
potentially close STDIN_FILENO though, I think that is harmless at the
moment - or perhaps prepare_input should dup() it.



On Sat, Jul 01, 2017 at 03:26:27PM -0400, Bryan Steele wrote:
> I think I lost this part in my larger diff, make sure fds are closed
> after testing files.
> 
> -Bryan.
> 
> Index: file.c
> ===
> RCS file: /cvs/src/usr.bin/file/file.c,v
> retrieving revision 1.63
> diff -u -p -u -r1.63 file.c
> --- file.c28 Jun 2017 17:14:15 -  1.63
> +++ file.c1 Jul 2017 19:23:17 -
> @@ -208,6 +208,8 @@ main(int argc, char **argv)
>   for (idx = 0; idx < argc; idx++) {
>   inf[idx].m = m;
>   test_file([idx], width);
> + if (inf[idx].fd != -1)
> + close(inf[idx].fd);
>   }
>   exit(0);
>  }
> 

Re: file: replace fgetln with getline(3)

[Nicholas Marriott]

What is the point of doing this? fgetln works fine.




On Sat, Jul 01, 2017 at 02:47:25PM -0400, Bryan Steele wrote:
> Seems to work, does this look right?
> 
> -Bryan.
> 
> Index: magic-load.c
> ===
> RCS file: /cvs/src/usr.bin/file/magic-load.c,v
> retrieving revision 1.25
> diff -u -p -r1.25 magic-load.c
> --- magic-load.c  1 Jul 2017 14:34:29 -   1.25
> +++ magic-load.c  1 Jul 2017 18:44:42 -
> @@ -1071,6 +1071,7 @@ magic_load(FILE *f, const char *path, in
>   struct magic_line   *ml = NULL, *parent, *parent0;
>   char*line, *tmp;
>   size_t   size;
> + ssize_t  slen;
>   u_intat, level, n, i;
>  
>   m = xcalloc(1, sizeof *m);
> @@ -1084,15 +1085,11 @@ magic_load(FILE *f, const char *path, in
>  
>   at = 0;
>   tmp = NULL;
> - while ((line = fgetln(f, ))) {
> - if (line[size - 1] == '\n')
> - line[size - 1] = '\0';
> - else {
> - tmp = xmalloc(size + 1);
> - memcpy(tmp, line, size);
> - tmp[size] = '\0';
> - line = tmp;
> - }
> + while ((slen = getline(, , f)) != -1) {
> + line = tmp;
> + if (line[slen - 1] == '\n')
> + line[slen - 1] = '\0';
> +
>   at++;
>  
>   while (isspace((u_char)*line))

Re: small patch for file(1)

[Nicholas Marriott]

ok nicm, it should not fclose this because it didn't open it


On Sat, Jul 01, 2017 at 10:19:32AM -0400, Bryan Steele wrote:
> magic_load() is only called in main and fcloses magicfp immediately,
> this removes a redundant fclose.
> 
> Index: magic-load.c
> ===
> RCS file: /cvs/src/usr.bin/file/magic-load.c,v
> retrieving revision 1.24
> diff -u -p -u -r1.24 magic-load.c
> --- magic-load.c  18 Apr 2017 14:16:48 -  1.24
> +++ magic-load.c  1 Jul 2017 14:16:14 -
> @@ -1169,6 +1169,5 @@ magic_load(FILE *f, const char *path, in
>   }
>   free(tmp);
>  
> - fclose(f);
>   return (m);
>  }

Re: fix stdin input for file(1)

[Nicholas Marriott]

Ouch. ok nicm

You can add this regress test as well:

Index: regress/usr.bin/file//Makefile
===
RCS file: /cvs/src/regress/usr.bin/file/Makefile,v
retrieving revision 1.7
diff -u -p -r1.7 Makefile
--- regress/usr.bin/file//Makefile  1 May 2016 11:28:06 -   1.7
+++ regress/usr.bin/file//Makefile  28 Jun 2017 17:18:35 -
@@ -4,7 +4,7 @@ FILE=file
 
 REGRESS_TARGETS=t0  t1  t2  t3  t4  t5  t6  t7  \
t8  t9  t10 t11 t15 t17 t18 t19 \
-   t20 t21 t22 t30 t31 t32 t33
+   t20 t21 t22 t30 t31 t32 t33 stdin
 
 # .in:  input file
 # .out: desired result
@@ -17,6 +17,12 @@ all: ${REGRESS_TARGET}
@echo ${*}
@${FILE} ${.CURDIR}/${*}.in | \
sed -e "s@${.CURDIR}/@@" | \
+   diff - ${.CURDIR}/${*}.out || \
+   (echo "XXX ${*} failed" && false)
+
+stdin:
+   @echo ${*}
+   @${FILE} -  My latest commit broke support for stdin input, whoops.
> 
> ok?
> 
> Index: file.c
> ===
> RCS file: /cvs/src/usr.bin/file/file.c,v
> retrieving revision 1.62
> diff -u -p -u -r1.62 file.c
> --- file.c28 Jun 2017 15:42:49 -  1.62
> +++ file.c28 Jun 2017 16:42:40 -
> @@ -217,12 +217,16 @@ prepare_input(struct input_file *inf, co
>  {
>   int fd, mode, error;
>  
> + inf->path = path;
> +
>   if (strcmp(path, "-") == 0) {
>   if (fstat(STDIN_FILENO, >sb) == -1) {
>   inf->error = errno;
>   inf->fd = -1;
> + return;
>   }
>   inf->fd = STDIN_FILENO;
> + return;
>   }
>  
>   if (Lflag)
> @@ -232,6 +236,7 @@ prepare_input(struct input_file *inf, co
>   if (error == -1) {
>   inf->error = errno;
>   inf->fd = -1;
> + return;
>   }
>  
>   /* We don't need them, so don't open directories or symlinks. */
> @@ -245,7 +250,6 @@ prepare_input(struct input_file *inf, co
>   if (S_ISLNK(mode))
>   read_link(inf, path);
>   inf->fd = fd;
> - inf->path = path;
>  }
>  
>  static void
> 

Re: Attempt to simplify file(1)

[Nicholas Marriott]

Thanks. Comments inline.

On Tue, Jun 27, 2017 at 06:28:57AM -0400, Bryan Steele wrote:
> On Tue, Jun 27, 2017 at 01:20:59AM -0400, Bryan Steele wrote:
> > On Tue, Jun 27, 2017 at 12:26:08AM -0400, Bryan Steele wrote:
> > Some unintentional changes crept in, here's another diff..
> 
> Sorry, last diff broke width calculation.. 3rd times the charm.
> 
> Index: Makefile
> ===
> RCS file: /cvs/src/usr.bin/file/Makefile,v
> retrieving revision 1.16
> diff -u -p -u -r1.16 Makefile
> --- Makefile  4 Oct 2015 07:25:59 -   1.16
> +++ Makefile  27 Jun 2017 10:05:39 -
> @@ -5,9 +5,6 @@ SRCS=   file.c magic-dump.c magic-load.c
>   text.c xmalloc.c
>  MAN= file.1 magic.5
>  
> -LDADD=   -lutil
> -DPADD=   ${LIBUTIL}
> -
>  CDIAGFLAGS+= -Wno-long-long -Wall -W -Wnested-externs -Wformat=2
>  CDIAGFLAGS+= -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations
>  CDIAGFLAGS+= -Wwrite-strings -Wshadow -Wpointer-arith -Wsign-compare
> Index: file.c
> ===
> RCS file: /cvs/src/usr.bin/file/file.c,v
> retrieving revision 1.59
> diff -u -p -u -r1.59 file.c
> --- file.c18 Apr 2017 14:16:48 -  1.59
> +++ file.c27 Jun 2017 10:05:39 -
> @@ -29,12 +29,10 @@
>  #include 
>  #include 
>  #include 
> -#include 
>  #include 
>  #include 
>  #include 
>  #include 
> -#include 
>  #include 
>  #include 
>  #include 
> @@ -43,27 +41,16 @@
>  #include "magic.h"
>  #include "xmalloc.h"
>  
> -struct input_msg {
> - int idx;
> -
> - struct stat sb;
> - int error;
> -
> - charlink_path[PATH_MAX];
> - int link_error;
> - int link_target;
> -};
> -
> -struct input_ack {
> - int idx;
> -};
> -
>  struct input_file {
>   struct magic*m;
> - struct input_msg*msg;
>  
>   const char  *path;
> - int  fd;
> + struct stat  sb;
> + int  fd, error;

One member per line in structs please. Also you could reduce the amount
of space here now to one tab.

> +
> + char link_path[PATH_MAX];
> + int  link_error;
> + int  link_target;
>  
>   void*base;
>   size_t   size;
> @@ -75,15 +62,13 @@ extern char   *__progname;
>  
>  __dead void   usage(void);
>  
> -static intprepare_message(struct input_msg *, int, const char *);
> -static void   send_message(struct imsgbuf *, void *, size_t, int);
> -static intread_message(struct imsgbuf *, struct imsg *, pid_t);
> +static void   prepare_input(struct input_file *, const char *);
>  
> -static void   read_link(struct input_msg *, const char *);
> +static void   read_link(struct input_file *, const char *);
>  
> -static __dead void child(int, pid_t, int, char **);
> +static void   privdrop(void);
>  
> -static void   test_file(struct input_file *, size_t);
> +static void   test_file(struct input_file *, struct magic *, size_t);
>  
>  static inttry_stat(struct input_file *);
>  static inttry_empty(struct input_file *);
> @@ -120,14 +105,12 @@ usage(void)
>  int
>  main(int argc, char **argv)
>  {
> - int  opt, pair[2], fd, idx;
> + int  opt, idx;
>   char*home;
>   struct passwd   *pw;
> - struct imsgbuf   ibuf;
> - struct imsg  imsg;
> - struct input_msg msg;
> - struct input_ack*ack;
> - pid_tpid, parent;
> + struct magic*m;
> + struct input_file   *inf;
> + size_t   len, width = 0;
>  
>   tzset();
>  
> @@ -193,71 +176,48 @@ main(int argc, char **argv)
>   if (magicfp == NULL)
>   err(1, "%s", magicpath);
>  
> - parent = getpid();
> - if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, pair) != 0)
> - err(1, "socketpair");
> - switch (pid = fork()) {
> - case -1:
> - err(1, "fork");
> - case 0:
> - close(pair[0]);
> - child(pair[1], parent, argc, argv);
> + m = magic_load(magicfp, magicpath, cflag || Wflag);
> + if (cflag) {
> + magic_dump(m);
> + exit(0);

magic_load (which parses the magic file) is now before pledge and
privdrop which is wrong. You need to drop privs and pledge before
magic_load - if needed you can reduce the pledge further after
magic_load, but I think it only needs stdio anyway.

>   }
> - close(pair[1]);
> -
> - fclose(magicfp);
> - magicfp = NULL;
>  
> - if (cflag)
> - goto wait_for_child;
> -
> - imsg_init(, pair[0]);
> + inf = xcalloc(argc, sizeof *inf);
>   for (idx = 0; idx < argc; idx++) {
> - fd 

Re: ksh(1): don't output invalid UTF-8 characters

[Nicholas Marriott]

Having a look at ksh, I don't see how Anton's original diff is much
different from x_emacs() looping around x_e_getc() until it finishes a
long key input?

It would be better to stop reading early if an invalid UTF-8 byte is
input rather than always requiring exactly N bytes; he needs to fix his
u8len(); and I would want to test the other calls to x_e_getc(), such as
in x_search_char_forw() -- but I the general idea seems reasonable.



On Sat, May 20, 2017 at 12:04:35AM +0100, Nicholas Marriott wrote:
> On Fri, May 19, 2017 at 09:29:06PM +0200, Ingo Schwarze wrote:
> > On a side note, i don't think gnome-terminal and konsole are relevant.
> > I never installed them before and did so now for the first time for
> > testing, but they installed so many libraries that i feel uncomfortable
> > and unsafe using them even briefly and purely for testing purposes.
> > I will certainly pkg_delete them, and the libraries they pulled in, in
> > the near future.
> 
> You might not use them, and I don't use them, but they are very popular,
> as are half a dozen others that may well have different behaviours.
> 
> > > FWIW bash seems to do the replacement to U+FFFD itself before it sends
> > > it to the terminal, which means it is (more) predictable. I don't know
> > > if this is a sensible option for ksh.
> > 
> > That is not really an option.  This matters most while the multibyte
> > character is being typed, when the first byte is already being
> > processed but the second one not yet.  Replacing the byte with
> > U+FFFD, then later substituting the actual bytes back when all have
> > arrived, doesn't really make much sense to me.
> 
> It would help because terminals would know how to deal with the
> character.
> 
> So ksh is doing this now - if we pretend A,B,C are UTF-8 characters:
> 
> A,cursor left,B,cursor left,C
> 
> So first A appears, then B overwrites A, then C overwrites B.
> 
> Except A and B are invalid so the terminal may not do the right thing
> with them. It may draw no characters and not move the cursor (like
> tmux), or draw two and move the cursor by two (like rxvt-unicode appears
> to).
> 
> If you sent U+FFFD instead of the invalid characters, that is a known
> character and terminals should be guaranteed to do what ksh wants (move
> the cursor back one position to the right), until ksh overwrites it
> again.
> 
> > Hanging the shell until all expected bytes have arrived seems like
> > a bad idea, too.  You can see the misbehaviour that is causing in
> > programs like uniname(1) from the misc/uniutils package:
> 
> I don't really understand this. How does ksh handle cursor keys? What if
> when I press Left, the three bytes (\033OA) are split across two
> separate read()s? How does this avoid hanging ksh?

Re: ksh(1): don't output invalid UTF-8 characters

[Nicholas Marriott]

On Fri, May 19, 2017 at 09:29:06PM +0200, Ingo Schwarze wrote:
> On a side note, i don't think gnome-terminal and konsole are relevant.
> I never installed them before and did so now for the first time for
> testing, but they installed so many libraries that i feel uncomfortable
> and unsafe using them even briefly and purely for testing purposes.
> I will certainly pkg_delete them, and the libraries they pulled in, in
> the near future.

You might not use them, and I don't use them, but they are very popular,
as are half a dozen others that may well have different behaviours.

> > FWIW bash seems to do the replacement to U+FFFD itself before it sends
> > it to the terminal, which means it is (more) predictable. I don't know
> > if this is a sensible option for ksh.
> 
> That is not really an option.  This matters most while the multibyte
> character is being typed, when the first byte is already being
> processed but the second one not yet.  Replacing the byte with
> U+FFFD, then later substituting the actual bytes back when all have
> arrived, doesn't really make much sense to me.

It would help because terminals would know how to deal with the
character.

So ksh is doing this now - if we pretend A,B,C are UTF-8 characters:

A,cursor left,B,cursor left,C

So first A appears, then B overwrites A, then C overwrites B.

Except A and B are invalid so the terminal may not do the right thing
with them. It may draw no characters and not move the cursor (like
tmux), or draw two and move the cursor by two (like rxvt-unicode appears
to).

If you sent U+FFFD instead of the invalid characters, that is a known
character and terminals should be guaranteed to do what ksh wants (move
the cursor back one position to the right), until ksh overwrites it
again.

> Hanging the shell until all expected bytes have arrived seems like
> a bad idea, too.  You can see the misbehaviour that is causing in
> programs like uniname(1) from the misc/uniutils package:

I don't really understand this. How does ksh handle cursor keys? What if
when I press Left, the three bytes (\033OA) are split across two
separate read()s? How does this avoid hanging ksh?

Re: ksh(1): don't output invalid UTF-8 characters

[Nicholas Marriott]

Hi

On Fri, May 19, 2017 at 10:23:08PM +0200, Ingo Schwarze wrote:
> Hi Nicholas,
> 
> Nicholas Marriott wrote on Fri, May 19, 2017 at 07:04:53PM +0100:
> 
> > Perhaps I haven't understood what you are saying correctly,
> 
> What matters most is that sending an incomplete character
> followed by U+0008 (ASCII BACKSPACE) is a no-op, both in the sense
> that it doesn't change the line being edited and that it doesn't
> change the display.  All terminals you mentioned seem to conform
> to that according to my testing, except tmux.

They don't all do the same thing for me. I am doing this, the same as
ksh does:

printf 'a\010\342a\010\010\342\211a\010\010\342\211\240a\010\n'

And the terminals behave differently.

> > but I don't think it is possible to send control characters or
> > any other invalid UTF-8 bytes inside UTF-8 characters and safely
> > predict what the terminal will do. How about these examples:
> 
> If the invalid bytes are still present by the time the line is sent
> off for processing (like in your example printf '\343\203\n'), then
> it is indeed hard to predict what random terminals will do, though

I don't know what you mean by "sent off for processing".

I think you might be under some misapprehension.

\010 (ASCII BS) is just a cursor positioning sequence, all it does is
move the cursor one position to the left. \n the same, it moves the
cursor one line down.

The problem here is that ksh assumes a partial UTF-8 character (whether
one byte or two) will also move the cursor by one position, but that is
not always the case. Not for tmux, and - for me - not for other
terminals.

Are you thinking of ICANON? That is a kernel mechanism, and when it is
in use, the backspace will never reach the tmux. But ksh doesn't appear
to use it.

> i would argue that xterm's behaviour is correct (print one substitution
> glyph for each incomplete character, or if bytes don't form even
> incomplete characters, then one for each such invalid byte.

So you think the correct behaviour is to replace invalid sequences by
U+FEFF?

tmux could do that, but it won't fix ksh for other terminals. ksh is
making an assumption about how terminals will behave that is not
correct.

> urxvt is clearly broken:
> 
>  $ printf '\343\203x\n'
> 
> prints U+00E3 x linefeed; i have no idea what it does to garble
> 0xe383 into 0xc3a3.  Maybe some naive misparsing, or spewing out
> incomplete parsing state in some inconsistent way.
> 
> gnome-terminal and konsole print one replacement character for each
> invalid byte, even if bytes form an incomplete character.  Maybe
> not outright wrong, but arguably a bit confusing.

This doesn't sound like all the terminals doing the same thing.

> So yeah, if lines containing incomplete sequences *when they are
> sent off* misformat with tmux and gnome-terminal or konsole, i
> wouldn't call that tmux'es fault, and i agree there is little that
> can be done about it.

There is no "sent off". What you send is what you get, invalid
characters, backspaces, everything.

> 
> > Having tmux ignore the whole lot seems like a relatively sensible
> > course.
> 
> Well, what tmux currently does is making sure that everything gets
> broken in the maximum possible way on every terminal, even if the
> line that is finally sent off is completely correct.
> 
> > The only other alternative would be to substitute U+FFFD.
> 
> Why not just pass the bytes through?  I don't think it's the job
> of a terminal mutiplexer to mess with individual bytes.  It's the
> job of the final terminal doing the display to select glyphs and
> place them, for printable characters, for non-printable characters,
> for incomplete characters, and for invalid bytes.

No no, it is the job of tmux to interpret what it receives from the
application and make sure it shows the same no matter what terminal is
outside.

If terminals outside behave differently for the same output, tmux can't
keep its own internal state correct, so the display will be
mangled. tmux has to understand everything it receives, there is almost
no case where we can just pass through.

> 
> > But that seems iffy too - U+FFFD is width 1, but what if the
> > application is expecting a width 2?
> 
> By definition, incomplete characters and invalid bytes don't
> have a width, so it doesn't matter what the application wanted
> (for example, which character the user intended to type but
> didn't finish).  What matters is how wide the replacement
> glyph will look on the final terminal.  In that respect, we
> cannot help making an assumption, and "incomplete sequences
> and invalid bytes are displayed as U+FFFD (i.e. width 1)
> seems about the best we can do.  That may be slightly off
> for gnome-terminal and konsole, but i don't

Re: ksh(1): don't output invalid UTF-8 characters

[Nicholas Marriott]

ksh has problems for me with Anton's example in several terminals, not
just in tmux. Mostly the cursor seems to end up one character off rather
than in the prompt, which is less visibly incorrect perhaps, but still
wrong.

I don't know that ksh will be able to predict this reliably (not
uncommon with shells).

FWIW bash seems to do the replacement to U+FFFD itself before it sends
it to the terminal, which means it is (more) predictable. I don't know
if this is a sensible option for ksh.




On Fri, May 19, 2017 at 07:04:53PM +0100, Nicholas Marriott wrote:
> Hi
> 
> Perhaps I haven't understood what you are saying correctly, but I don't
> think it is possible to send control characters or any other invalid
> UTF-8 bytes inside UTF-8 characters and safely predict what the terminal
> will do. How about these examples:
> 
> printf '\343\203\010\217a\n'
> printf '\343\203\r\217b\n'
> printf '\343\203\n\217c\n'
> 
> Or with a width 1 character:
> 
> printf '\342\211\010\240\n'
> printf '\342\211\r\240b\n'
> printf '\342\211\n\240c\n'
> 
> I have four UTF-8 capable X terminals here and between them they do
> three different things (xterm, rxvt-unicode, gnome-terminal & konsole).
> 
> tmux can't forward this stuff to the terminal outside, because if it
> can't predict what that terminal will do, tmux's idea of the display
> will get out of sync with reality.
> 
> Having tmux ignore the whole lot seems like a relatively sensible
> course.
> 
> The only other alternative would be to substitute U+FFFD. But that seems
> iffy too - U+FFFD is width 1, but what if the application is expecting
> a width 2?
> 
> 
> 
> 
> On Fri, May 19, 2017 at 06:54:38PM +0200, Ingo Schwarze wrote:
> > Hi Anton,
> > 
> > Anton Lindqvist wrote on Fri, May 19, 2017 at 08:42:05AM +0200:
> > 
> > > 1. Run ksh under tmux.
> > > 
> > > 2. Input the following characters, without spaces:
> > > 
> > >a (any character) ^B (backward-char) ?? (any UTF-8 character)
> > > 
> > > 3. At this point, the prompt gets overwritten.
> > > 
> > > Since ksh read a single byte of input, it will display a partial UTF-8
> > > character before the whole character has been read. This is especially
> > > troublesome when the cursor is not placed at the end of the line. In the
> > > scenario above, after reading the first byte of '??' the following
> > > sequence will be displayed:
> > > 
> > >   0xc3 0x61 0x08
> > > 
> > > That is the first byte of '??' (0xc3), 'a' (0x61), '\b' (0x08). tmux
> > > does the right thing here, since 0xc3 is a valid UTF-8 start byte it
> > > expects it to be followed by a UTF-8 continuation byte which is not the
> > > case. The two first bytes (0xc3, 0x61) are discarded and the parser is
> > > reset to its initial state
> > 
> > I call that a bug in tmux.  At least for UTF-8, tmux should never reset
> > its parser.  Depending on the keyboard configuration, it may be possible
> > to enter single non-UTF-8 bytes.  For example, at the console, i can do
> > this:
> > 
> >  - type "printf x"
> >  - press Ctrl-V, Ctrl-Alt-Z
> >  - type "printf x | hexdump -C"
> > 
> > The result is:
> > 
> >     78 9a 78   |x.x|
> >   0003
> > 
> > All of the shell, the console, and xterm more or less handle such
> > stunts, even though admittedly, that is not the usual way of typing
> > in a binary file.  tmux is a terminal emulator, so it ought to cope,
> > too, and not reset its state.
> > 
> > Note that this is yet another instance where the concept of arbitrary
> > locales is utterly broken and insecure.  On some non-OpenBSD system
> > supporting such insecure stuff, if the user has set an arbitrary,
> > non-UTF-8, state-dependent locale and somehow manages to insert an
> > invalid byte into the input stream, the state of the input stream
> > becomes invalid, no further characters can be read from that terminal,
> > and *there is no way to recover*.  The only remaining half-secure
> > option is for the shell to exit, which may also not be very secure,
> > on a different level.
> > 
> > But with UTF-8, there is no problem whatsoever dealing with invalid
> > bytes, so tmux ought to cope.
> > 
> > > causing the backspace to be accepted and the
> > > first character in the prompt to be overwritten.
> > 
> > That's part of the reason why tmux must not reset its state:
> > The shell won't know about it, and the screen will become garbled.
> > 
> > > Below is diff

Re: ksh(1): don't output invalid UTF-8 characters

[Nicholas Marriott]

Hi

Perhaps I haven't understood what you are saying correctly, but I don't
think it is possible to send control characters or any other invalid
UTF-8 bytes inside UTF-8 characters and safely predict what the terminal
will do. How about these examples:

printf '\343\203\010\217a\n'
printf '\343\203\r\217b\n'
printf '\343\203\n\217c\n'

Or with a width 1 character:

printf '\342\211\010\240\n'
printf '\342\211\r\240b\n'
printf '\342\211\n\240c\n'

I have four UTF-8 capable X terminals here and between them they do
three different things (xterm, rxvt-unicode, gnome-terminal & konsole).

tmux can't forward this stuff to the terminal outside, because if it
can't predict what that terminal will do, tmux's idea of the display
will get out of sync with reality.

Having tmux ignore the whole lot seems like a relatively sensible
course.

The only other alternative would be to substitute U+FFFD. But that seems
iffy too - U+FFFD is width 1, but what if the application is expecting
a width 2?




On Fri, May 19, 2017 at 06:54:38PM +0200, Ingo Schwarze wrote:
> Hi Anton,
> 
> Anton Lindqvist wrote on Fri, May 19, 2017 at 08:42:05AM +0200:
> 
> > 1. Run ksh under tmux.
> > 
> > 2. Input the following characters, without spaces:
> > 
> >a (any character) ^B (backward-char) ?? (any UTF-8 character)
> > 
> > 3. At this point, the prompt gets overwritten.
> > 
> > Since ksh read a single byte of input, it will display a partial UTF-8
> > character before the whole character has been read. This is especially
> > troublesome when the cursor is not placed at the end of the line. In the
> > scenario above, after reading the first byte of '??' the following
> > sequence will be displayed:
> > 
> >   0xc3 0x61 0x08
> > 
> > That is the first byte of '??' (0xc3), 'a' (0x61), '\b' (0x08). tmux
> > does the right thing here, since 0xc3 is a valid UTF-8 start byte it
> > expects it to be followed by a UTF-8 continuation byte which is not the
> > case. The two first bytes (0xc3, 0x61) are discarded and the parser is
> > reset to its initial state
> 
> I call that a bug in tmux.  At least for UTF-8, tmux should never reset
> its parser.  Depending on the keyboard configuration, it may be possible
> to enter single non-UTF-8 bytes.  For example, at the console, i can do
> this:
> 
>  - type "printf x"
>  - press Ctrl-V, Ctrl-Alt-Z
>  - type "printf x | hexdump -C"
> 
> The result is:
> 
>     78 9a 78   |x.x|
>   0003
> 
> All of the shell, the console, and xterm more or less handle such
> stunts, even though admittedly, that is not the usual way of typing
> in a binary file.  tmux is a terminal emulator, so it ought to cope,
> too, and not reset its state.
> 
> Note that this is yet another instance where the concept of arbitrary
> locales is utterly broken and insecure.  On some non-OpenBSD system
> supporting such insecure stuff, if the user has set an arbitrary,
> non-UTF-8, state-dependent locale and somehow manages to insert an
> invalid byte into the input stream, the state of the input stream
> becomes invalid, no further characters can be read from that terminal,
> and *there is no way to recover*.  The only remaining half-secure
> option is for the shell to exit, which may also not be very secure,
> on a different level.
> 
> But with UTF-8, there is no problem whatsoever dealing with invalid
> bytes, so tmux ought to cope.
> 
> > causing the backspace to be accepted and the
> > first character in the prompt to be overwritten.
> 
> That's part of the reason why tmux must not reset its state:
> The shell won't know about it, and the screen will become garbled.
> 
> > Below is diff that make sure to read a whole UTF-8 character in
> > x_emacs() prior doing another iteration of the main-loop
> 
> I don't think we can do that.  What if there is no next byte?
> Then the shell will hang until, maybe considerably later, the next
> character is typed.  Also, we cannot rely on parsing the UTF-8 start
> byte (even if done correctly).  It tells the byte length of the
> character only for valid byte sequences, and the byte sequence need
> not be valid.
> 
> Besides, i think patching the shell to work around terminal bugs
> (or terminal emulator bugs) is the wrong approach in the first place.
> 
> Yours,
>   Ingo
> 

Re: event(3): mention bufferevent_setwatermark

[Nicholas Marriott]

I think it needs some text as well, there isn't much point in just
listing the function. bufferevent_settimeout is also missing a
description.

I'm away for the next two weeks but I'll look when I get back.



On Mon, May 15, 2017 at 07:41:57PM +0200, Anton Lindqvist wrote:
> Hi,
> The bufferevent_setwatermark function is not mentioned in event(3).
> Maybe the function deserves to be documented under the "BUFFERED EVENTS"
> section but I know too little about the API to determine if that would
> be useful. Some docs regarding the function can however be found in the
> event.h header.
> 
> Spotted while reading the tmux source.
> 
> Index: event.3
> ===
> RCS file: /cvs/src/lib/libevent/event.3,v
> retrieving revision 1.52
> diff -u -p -r1.52 event.3
> --- event.3   17 Jul 2016 11:21:07 -  1.52
> +++ event.3   15 May 2017 17:32:36 -
> @@ -67,7 +67,8 @@
>  .Nm bufferevent_read ,
>  .Nm bufferevent_enable ,
>  .Nm bufferevent_disable ,
> -.Nm bufferevent_settimeout
> +.Nm bufferevent_settimeout ,
> +.Nm bufferevent_setwatermark
>  .Nd execute a function when a specific event occurs
>  .Sh SYNOPSIS
>  .In sys/time.h
> @@ -154,6 +155,8 @@
>  .Fn "bufferevent_disable" "struct bufferevent *bufev" "short event"
>  .Ft void
>  .Fn "bufferevent_settimeout" "struct bufferevent *bufev" "int timeout_read" 
> "int timeout_write"
> +.Ft void
> +.Fn "bufferevent_setwatermark" "struct bufferevent *bufev" "short events" 
> "size_t lowmark" "size_t highmark"
>  .Sh DESCRIPTION
>  The
>  .Nm event
> 

Re: less(1): plug memory leak

[Nicholas Marriott]

looks good, ok nicm


On Mon, May 01, 2017 at 10:35:59AM +0200, Tobias Stoeckmann wrote:
> Hi,
> 
> On Mon, May 01, 2017 at 09:15:45AM +0200, Anton Lindqvist wrote:
> > While freeing tag entries, make sure to free the copied strings.
> 
> this patch looks good to me.
> 
> Have you reported this to the upstream less maintainers, as well?
> The original less and the forked one from Garrett D'Amore, which we
> use as foundation, have the same issue in them.
> 
> 
> Tobias
> 
> > Index: tags.c
> > ===
> > RCS file: /cvs/src/usr.bin/less/tags.c,v
> > retrieving revision 1.18
> > diff -u -p -r1.18 tags.c
> > --- tags.c  17 Sep 2016 15:06:41 -  1.18
> > +++ tags.c  30 Apr 2017 18:13:24 -
> > @@ -77,6 +77,8 @@ cleantags(void)
> >  */
> > while ((tp = taglist.tl_first) != TAG_END) {
> > TAG_RM(tp);
> > +   free(tp->tag_file);
> > +   free(tp->tag_pattern);
> > free(tp);
> > }
> > curtag = NULL;
> > 
> 

Re: Better error output for readlink(1)

[Nicholas Marriott]

On Tue, Apr 04, 2017 at 05:03:26PM -0500, Scott Cheloha wrote:
> > On Apr 4, 2017, at 4:46 PM, Nicholas Marriott <nicholas.marri...@gmail.com> 
> > wrote:
> > 
> > readlink is explicitly documented to silently exit 1 if run without -f,
> > and GNU readlink behaves the same way. I doubt that should change.
> 
> Yeah, I saw that.  I (incorrectly, I guess?) interpreted it to mean
> "does not print anything on the standard output."
> 
> My think is -- and maybe I'm being nitpicky, but -- the documentation
> says:
> 
> > If the -f option is not specified and readlink is invoked with an
> > argument other than the pathname of a symbolic link, it exits with
> > a nonzero exit code without printing anything.
> 
> In the current code, however, you could have insufficient permissions
> for a part of the path (EPERM), or an I/O failure (EIO), but otherwise
> specify a valid symbolic link, and still get the described behavior,
> which seems wrong to me.
> 
> If the documented "say nothing and exit 1" behavior when the target
> is not a symbolic link is sacred, then maybe this snippet is better:

There are likely to be existing scripts that rely on this behaviour.

> 
>   if (fflag) {
>   if (realpath(argv[0], buf) == NULL)
>   err(1, "%s", argv[0]);
>   } else {
> - if ((n = readlink(argv[0], buf, sizeof buf-1)) < 0)
> - exit(1);
> + if ((n = readlink(argv[0], buf, sizeof(buf) - 1)) == -1) {
> + if (errno != EINVAL)
> + warn("%s", argv[0]);

You will definitely also need to check for at least ENOENT and ENOTDIR,
possibly others too.

I think it would be better not to change the behaviour, except possibly
for ENAMETOOLONG.

> + return 1;
> + }
>   buf[n] = '\0';
>   }
> 
> That way you still say nothing if the issue is, in fact, that your target
> isn't a symbolic link, but you point out any other errors.
> 
> --
> Scott Cheloha

Re: Better error output for readlink(1)

[Nicholas Marriott]

Hi

readlink is explicitly documented to silently exit 1 if run without -f,
and GNU readlink behaves the same way. I doubt that should change.



On Tue, Apr 04, 2017 at 04:40:19PM -0500, Scott Cheloha wrote:
> This patch replaces a custom error message in readlink.c with err(3).
> The custom message and call to strlen(3) aren't needed because
> readlink(2) checks the length of the argument string and sets errno
> to ENAMETOOLONG if it is too long.
> 
> Dropping strlen(3) lets us drop string.h.
> 
> Using err(3) also lets us trivially report the myriad ways readlink(2)
> can fail.  At the moment we just exit 1, which can be misleading during
> interactive use.
> 
> While here, do other miscellaneous style(9)-type changes.
> 
> Any takers?
> 
> --
> Scott Cheloha
> 
> P.S. returning from main() is preferred over exit(3), right?
> 
> Index: readlink.c
> ===
> RCS file: /cvs/src/usr.bin/readlink/readlink.c,v
> retrieving revision 1.27
> diff -u -p -r1.27 readlink.c
> --- readlink.c9 Oct 2015 01:37:08 -   1.27
> +++ readlink.c4 Apr 2017 21:10:41 -
> @@ -32,17 +32,17 @@
>  #include 
>  #include 
>  #include 
> -#include 
>  #include 
>  
> -static void  usage(void);
> +static __dead void   usage(void);
>  
>  int
>  main(int argc, char *argv[])
>  {
>   char buf[PATH_MAX];
> - int n, ch, nflag = 0, fflag = 0;
> - extern int optind;
> + int ch, fflag, n, nflag;
> +
> + fflag = nflag = 0;
>  
>   if (pledge("stdio rpath", NULL) == -1)
>   err(1, "pledge");
> @@ -64,30 +64,26 @@ main(int argc, char *argv[])
>   if (argc != 1)
>   usage();
>  
> - n = strlen(argv[0]);
> - if (n > PATH_MAX - 1) {
> - fprintf(stderr,
> - "readlink: filename longer than PATH_MAX-1 (%d)\n",
> - PATH_MAX - 1);
> - exit(1);
> - }
> -
>   if (fflag) {
>   if (realpath(argv[0], buf) == NULL)
>   err(1, "%s", argv[0]);
>   } else {
> - if ((n = readlink(argv[0], buf, sizeof buf-1)) < 0)
> - exit(1);
> + if ((n = readlink(argv[0], buf, sizeof(buf) - 1)) == -1) {
> + if (errno == EINVAL)
> + errx(1, "%s: Not a symbolic link", argv[0]);
> + else
> + err(1, "%s", argv[0]);
> + }
>   buf[n] = '\0';
>   }
>  
>   printf("%s", buf);
>   if (!nflag)
>   putchar('\n');
> - exit(0);
> + return 0;
>  }
>  
> -static void
> +static __dead void
>  usage(void)
>  {
>   (void)fprintf(stderr, "usage: readlink [-fn] file\n");
> 

Re: vmd 4/5: replace openpty(4) with a local function

[Nicholas Marriott]

Hi

I think putting these in libutil is a good idea. tmux could use
them. I'd like to have openptmfd() as you suggest as well - it'd be nice
to hide PATH_PTMDEV as well as the PTMGET.

Life would be a lot easier for portable if there was fdforkpty() as
well.



On Mon, Feb 27, 2017 at 07:00:03PM +0100, Reyk Floeter wrote:
> On Mon, Feb 27, 2017 at 10:19:28AM -0700, Theo de Raadt wrote:
> > > On Mon, Feb 27, 2017 at 10:55:31AM +0100, Reyk Floeter wrote:
> > > > The following diff is not really needed without just yet, but:
> > > > - openening /dev/ptm in advance might allow better pledge in the future
> > > > - customizing "openpty" will allow to do what we need next
> > > > Since openpty(4) is libutil and not libc, it should be fine not using 
> > > > it.
> > > > 
> > > > OK?
> > > > 
> > > > Replace openpty(3) with local function that uses pre-opened 
> > > > /dev/ptm fd
> > > > 
> > > > This allows more flexibility for upcoming changes and better pledge.
> > > > We also didn't use half of the features of libutil's openpty 
> > > > function.
> > > > Additionally, make sure that the ttys are closed correctly on 
> > > > shutdown.
> > 
> > This is related to the change that happened in tmux, when PTM ioctl was
> > locked down in pledge ioctl.  The primary purpose of the change, was that
> > review of the PTM ioctl codepaths showed it was a lot of risk to keep in
> > pledge "tty" programs, so it makes sense that some of them get restructured.
> > 
> > That means the libutil functions aren't always used.  We should consider
> > whether raw ioctl PTMGET in programs is "good" or "bad".  vmd is openbsd
> > only, but tmux now has this in the openbsd-specific code.
> > 
> > If this goes PTMGET approach goes any further, we should think about a
> > new interface in libutil which hides the ioctl in a differnet way, and
> > convert programs to it.  Just a thought..
> > 
> 
> I had a diff for libutil but millert@ (or was it nicm@?) told me that
> it is better to keep it as a local change.  I cannot remember and
> unfortunately I cannot find the mail where it was mentioned.
> 
> I attached the old diff for libutil as a reference.
> 
> To fully abstract /dev/ptm in libutil, the API below would have to be
> extended to have another function to open /dev/ptm in libutil as well,
> eg. (better names would be desired):
> 
>   fd = openptmfd()
>   pledge()
>   fdopenpty(fd, ...)
>   fdopenpty(fd, ...)
>   fdopenpty(fd, ...)
>   fdopenpty(fd, ...)
> 
> Reyk
> 
> Index: lib/libutil/openpty.3
> ===
> RCS file: /cvs/src/lib/libutil/openpty.3,v
> retrieving revision 1.17
> diff -u -p -u -p -r1.17 openpty.3
> --- lib/libutil/openpty.3 28 Aug 2015 19:54:06 -  1.17
> +++ lib/libutil/openpty.3 25 Jan 2017 08:40:46 -
> @@ -44,6 +44,8 @@
>  .Ft int
>  .Fn openpty "int *amaster" "int *aslave" "char *name" "struct termios 
> *termp" "struct winsize *winp"
>  .Ft int
> +.Fn fdopenpty "int ptmfd" "int *amaster" "int *aslave" "char *name" "struct 
> termios *termp" "struct winsize *winp"
> +.Ft int
>  .Fn login_tty "int fd"
>  .Ft pid_t
>  .Fn forkpty "int *amaster" "char *name" "struct termios *termp" "struct 
> winsize *winp"
> @@ -88,6 +90,15 @@ the caller, permissions are set to corre
>  uses of that device are revoked (see
>  .Xr revoke 2
>  for details).
> +.Pp
> +The
> +.Fn fdopenpty
> +function works like
> +.Fn openpty
> +but expects a pre-opened
> +.Pa /dev/ptm
> +file descriptor
> +.Fa ptmfd .
>  .Pp
>  The
>  .Fn login_tty
> Index: lib/libutil/pty.c
> ===
> RCS file: /cvs/src/lib/libutil/pty.c,v
> retrieving revision 1.20
> diff -u -p -u -p -r1.20 pty.c
> --- lib/libutil/pty.c 30 Aug 2016 14:44:45 -  1.20
> +++ lib/libutil/pty.c 25 Jan 2017 08:40:46 -
> @@ -57,11 +57,30 @@ openpty(int *amaster, int *aslave, char 
>   fd = open(PATH_PTMDEV, O_RDWR|O_CLOEXEC);
>   if (fd == -1)
>   return (-1);
> - if ((ioctl(fd, PTMGET, ) == -1)) {
> +
> + if (fdopenpty(fd, amaster, aslave, name, termp, winp) == -1) {
>   close(fd);
>   return (-1);
>   }
>   close(fd);
> +
> + return (0);
> +}
> +
> +int
> +fdopenpty(int ptmfd, int *amaster, int *aslave, char *name,
> +struct termios *termp, struct winsize *winp)
> +{
> + int master, slave;
> + struct ptmget ptm;
> +
> + /*
> +  * Use /dev/ptm and the PTMGET ioctl to get a properly set up and
> +  * owned pty/tty pair.
> +  */
> + if ((ioctl(ptmfd, PTMGET, ) == -1))
> + return (-1);
> +
>   master = ptm.cfd;
>   slave = ptm.sfd;
>   if (name) {
> Index: lib/libutil/util.h
> ===
> RCS file: /cvs/src/lib/libutil/util.h,v
> retrieving revision 1.34
> diff -u -p -u -p -r1.34 util.h
> --- lib/libutil/util.h3 Jun 

Re: ksh tab completion: ^_: unexpected `^'

[Nicholas Marriott]

This is incoherent and I don't see what it has to do with the issue, you
will need to provide a diff yourself.



On Thu, Sep 08, 2016 at 09:18:19AM -0400, sven falempin wrote:
> This does not include UTF8 basic character,
> 
> so if someone do  
> 
> And it want to do it again for that file ... svik , does not work.
> 
> This problem should be address in isalnum, i guess, i think some c++
> lib did it already,
> and i have a headache everytime i want to use \w in a regexp.
> 
> current $ ./a.out ??lol
> NOP
> _
> ~
> current $ ./a.out elol
> elol_
> ~
> current $ cat /tmp/foo.c
> #include 
> 
> int main(int argc, char** argv) {
>   if (isalnum(argv[1][0] )) printf("%s", argv[1]); else printf ("NOP\n");
> }
> 
> On Thu, Sep 8, 2016 at 6:08 AM, Stuart Henderson <s...@spacehopper.org> wrote:
> > On 2016/09/08 10:45, Nicholas Marriott wrote:
> >> Yeah we probably shouldn't bother to look for commands that aren't 
> >> [A-Za-z0-9_-]:
> >
> > I don't think - is necessary, it's not a valid character for an array
> > name so it can't be used here anyway. Otherwise OK.
> >
> >> Index: edit.c
> >> ===
> >> RCS file: /cvs/src/bin/ksh/edit.c,v
> >> retrieving revision 1.56
> >> diff -u -p -r1.56 edit.c
> >> --- edit.c7 Sep 2016 04:42:31 -   1.56
> >> +++ edit.c8 Sep 2016 09:45:21 -
> >> @@ -584,9 +584,8 @@ x_try_array(const char *buf, int buflen,
> >>  int *nwords, char ***words)
> >>  {
> >>   const char *cmd, *cp;
> >> - int cmdlen, n;
> >> + int cmdlen, n, i, slen;
> >>   char *name, *s;
> >> - size_t slen;
> >>   struct tbl *v, *vp;
> >>
> >>   *nwords = 0;
> >> @@ -604,6 +603,10 @@ x_try_array(const char *buf, int buflen,
> >>   cmdlen = 0;
> >>   while (cmd + cmdlen < want && !isspace((u_char)cmd[cmdlen]))
> >>   cmdlen++;
> >> + for (i = 0; i < cmdlen; i++) {
> >> + if (!isalnum((u_char)cmd[i]) && cmd[i] != '_' && cmd[i] != 
> >> '-')
> >> + return 0;
> >> + }
> >>
> >>   /* Take a stab at argument count from here. */
> >>   n = 1;
> >>
> >>
> >> On Thu, Sep 08, 2016 at 09:43:53AM +0100, Stuart Henderson wrote:
> >> > I just ran into this which was introduced with custom completions
> >> > (I haven't setup any complete_* arrays).
> >> >
> >> > $ ag "(foo)[^_]" /u
> >> >
> >> > results in
> >
> 
> 
> 
> -- 
> -
> () ascii ribbon campaign - against html e-mail
> /\

Re: ksh tab completion: ^_: unexpected `^'

[Nicholas Marriott]

Yeah we probably shouldn't bother to look for commands that aren't 
[A-Za-z0-9_-]:

Index: edit.c
===
RCS file: /cvs/src/bin/ksh/edit.c,v
retrieving revision 1.56
diff -u -p -r1.56 edit.c
--- edit.c  7 Sep 2016 04:42:31 -   1.56
+++ edit.c  8 Sep 2016 09:45:21 -
@@ -584,9 +584,8 @@ x_try_array(const char *buf, int buflen,
 int *nwords, char ***words)
 {
const char *cmd, *cp;
-   int cmdlen, n;
+   int cmdlen, n, i, slen;
char *name, *s;
-   size_t slen;
struct tbl *v, *vp;
 
*nwords = 0;
@@ -604,6 +603,10 @@ x_try_array(const char *buf, int buflen,
cmdlen = 0;
while (cmd + cmdlen < want && !isspace((u_char)cmd[cmdlen]))
cmdlen++;
+   for (i = 0; i < cmdlen; i++) {
+   if (!isalnum((u_char)cmd[i]) && cmd[i] != '_' && cmd[i] != '-')
+   return 0;
+   }
 
/* Take a stab at argument count from here. */
n = 1;


On Thu, Sep 08, 2016 at 09:43:53AM +0100, Stuart Henderson wrote:
> I just ran into this which was introduced with custom completions
> (I haven't setup any complete_* arrays).
> 
> $ ag "(foo)[^_]" /u
> 
> results in
> 
> ksh: ^_: unexpected `^'
> 

Re: tmux(1): dealing with broken wcwidth(3)

[Nicholas Marriott]

On Thu, Apr 28, 2016 at 06:49:58PM +0200, Ingo Schwarze wrote:
> Hi Nic,
> 
> Nicholas Marriott wrote on Thu, Apr 28, 2016 at 01:11:44PM +0100:
> 
> > tmux is not some sort of terminal firewall. Of course we try to avoid
> > anything obviously stupid, but we also want stuff that works outside
> > tmux to also work inside.
> [...]
> > Assuming a width of 1 where we can't get a width from the system
> > is not a big step and I do not think it poses any more of a
> > security risk than just running the application without tmux.
> 
> Oh.  That does look like a valid argument that i missed.
> 
> Note, though, that most terminals display nothing at all when being
> fed a non-printable character (except that, of course, there are

If this was just guaranteed nonprintable characters, there would be no
issue. But the problem is that some platforms are missing genuine UTF-8
characters and since many terminals do not use wcwidth(), it means that
users who used older versions of tmux had working characters that now do
not work even though the terminal is happy to show them.

I think if nonprintable characters show up, it will be because a user
printed garbage and they will know they printed garbage and not expect
sensible output.

However I think you are right and there is no reason for this to be in
OpenBSD, I'll move it into portable tmux only. Since it is only a couple
of lines it is not a big issue.

> various control characters that change terminal state and can have
> extensive effects on how subsequent text will be printed).  So i'm
> not quite convinced the value 1 is better than the value 0.  Then
> again, i don't worry that much which value you use, i mostly worried
> about treating non-printables as printable at all.

I don't think it is tmux's role to protect the terminal and there is no
way it can do it. Sometimes we just have to assume that the terminal
will behave sensibly.

> 
> > tmux is /not/ getting the width with wcwidth() for some sort of
> > protective sanity check, it is getting it so it can keep its own state
> > correct. If we can't get the width, using the best guess of what the
> > terminal outside will do is fair and will fix more problems than it
> > causes.
> 
> In that case, it might make sense to add a comment explaining that
> non-printable characters are intentionally passed through, because
> doing so is no more dangerous than not using tmux(1) in the first
> place.
> 
> Given that failing to check the wcwidth(3) return value against -1,
> or mishandling that case in some way, is a very common and usually
> dangerous error, such a comment can help to avoid serious confusion
> of code auditors, like the one you witnessed on my part.  Saying
> "hope for the best", as you do now, rings all the alarms instead.
> 
> Yours,
>   Ingo

Re: tmux(1): dealing with broken wcwidth(3)

[Nicholas Marriott]

Hi

On Thu, Apr 28, 2016 at 01:47:27PM +0200, Ingo Schwarze wrote:
> Hi Nic,
> 
> Nicholas Marriott wrote on Wed, Apr 27, 2016 at 03:36:25AM -0600:
> 
> > CVSROOT:/cvs
> > Module name:src
> > Changes by: n...@cvs.openbsd.org2016/04/27 03:36:25
> > 
> > Modified files:
> > usr.bin/tmux   : utf8.c 
> > 
> > Log message:
> > Loads of platforms appear to have old or broken Unicode character type
> > information and are missing widths for relatively common Unicode
> > characters (so mbtowc() works, but wcwidth() fails). So if wcwidth()
> > returns -1, assume a width of 1 instead of ignoring the character.
> 
> I consider this is a backwards approach to portability.
> OpenBSD software ought to be developed to work correctly according
> to standards (as long as the standards aren't badly broken, which
> isn't the case here).
> 
> If *other* operating systems are broken, that should be dealt with
> in portable versions of OpenBSD software, not in OpenBSD itself.
> We don't want #ifdef in the code on cvs.openbsd.org to deal with
> brokenness in other operating systems.  We definitely don't want to
> cripple OpenBSD base tools just because other systems are broken.
> Due to afresh1@'s good work, we have a good wcwidth(3), and we don't
> want to punish OpenBSD users for other system's deficiencies.

I don't agree, this change is small and relatively harmless and we do
not need another difference to portable for it.

> 
> This particular change looks not only outright wrong, but even
> dangerous.  Treating non-printable characters as printable can
> cause printing them to the terminal, which may even result in
> vulnerabilities if you are unlucky.

tmux is not some sort of terminal firewall. Of course we try to avoid
anything obviously stupid, but we also want stuff that works outside
tmux to also work inside.

tmux is /not/ getting the width with wcwidth() for some sort of
protective sanity check, it is getting it so it can keep its own state
correct. If we can't get the width, using the best guess of what the
terminal outside will do is fair and will fix more problems than it
causes.

Note that we do not accept characters we deem to be invalid or where
mbtowc() fails. But assuming a width of 1 where we can't get a width
from the system is not a big step and I do not think it poses any more
of a security risk than just running the application without tmux.

> 
> I strongly object to code saying
> 
>   width = wcwidth(wc);
>   if (width < 0)
> width = 1;
> 
> It's an insecure idiom and should not be used.  In particular
> not on OpenBSD.

I do not agree.

> 
> If you have to support a system where wcwidth(3) is broken,
> the safest way is to
> 
>   #define wcwidth(c) (iswprint(c) ? 1 : -1)
> 
> on that system only, disabling double-width character support, but
> not exposing the system to printability vulnerabilities.  You should

I think this is a poor idea and will break many working characters for
no reason. Not to mention it would be difficult to detect a broken
wcwidth() in the first place.

> not treat non-printable characters as printable even there.  If
> iswprint(3) is broken, too, you should not support UTF-8 on that
> system at all but strictly limit tmux(1) to 7-bit ASCII for security
> reasons.  Of course, be careful to correctly treat C0 ASCII control
> codes in that case.

Part of the reason for using the system locale is to avoid having to
support multiple character sets.

Re: Any reason there's no way to persist pledge(2) state across exec?

[Nicholas Marriott]

Hi

What's the use for this? What program could use it?


On Sun, Apr 10, 2016 at 08:48:08AM -0700, Brennan Vincent wrote:
> Subject basically says it all. I think some could find it useful to have
> `pledge` promises optionally persist even after the process calls
> execve. This could, for example, be implemented with an `exec_noreset`
> pledge that gives access to the same syscalls as `exec`, but with this
> restricted behavior.
> 
> Is there a good technically reason this can't or shouldn't be done, or
> has it simply not been implemented yet?
> 

Re: [file] file -i doesn't detect epub correctly

[Nicholas Marriott]

The only thing I see that we do not support is the indirect on the last
line, I suggest just commenting it.



On Fri, Mar 04, 2016 at 07:01:57PM +, Stuart Henderson wrote:
> On 2016/03/04 19:50, Dmitrij D. Czarkoff wrote:
> > Jiri B said:
> > > file -i *.epub returns 'application/x-not-regular-file' or 
> > > 'application/zip'
> > > and it should return 'application/epub+zip' (at least this is on Fedora).
> > 
> > Below comes the patch that changes the way zip archives are treated in
> > magic file:
> 
> Can you try just pulling across magdir/archive from the other "file"?
> Unless there is a special case I think it is better to keep in sync.
> 
> e.g.
> 
> Index: archive
> ===
> RCS file: /cvs/src/usr.bin/file/magdir/archive,v
> retrieving revision 1.6
> diff -u -p -r1.6 archive
> --- archive   24 Apr 2009 18:54:34 -  1.6
> +++ archive   4 Mar 2016 19:00:39 -
> @@ -13,6 +13,11 @@
>  257  string  ustar\040\040\0 GNU tar archive
>  !:mime   application/x-tar # encoding: gnu
>  
> +# Incremental snapshot gnu-tar format from:
> +# http://www.gnu.org/software/tar/manual/html_node/Snapshot-Files.html
> +0string  GNU\ tar-   GNU tar incremental snapshot data
> +>&0  regex   [0-9]\.[0-9]+-[0-9]+version %s
> +
>  # cpio archives
>  #
>  # Yes, the top two "cpio archive" formats *are* supposed to just be "short".
> @@ -32,12 +37,66 @@
>  0string  070701  ASCII cpio archive (SVR4 with no CRC)
>  0string  070702  ASCII cpio archive (SVR4 with CRC)
>  
> -# Debian package (needs to go before regular portable archives)
> +#
> +# Various archive formats used by various versions of the "ar"
> +# command.
> +#
> +
> +#
> +# Original UNIX archive formats.
> +# They were written with binary values in host byte order, and
> +# the magic number was a host "int", which might have been 16 bits
> +# or 32 bits.  We don't say "PDP-11" or "VAX", as there might have
> +# been ports to little-endian 16-bit-int or 32-bit-int platforms
> +# (x86?) using some of those formats; if none existed, feel free
> +# to use "PDP-11" for little-endian 16-bit and "VAX" for little-endian
> +# 32-bit.  There might have been big-endian ports of that sort as
> +# well.
> +#
> +0leshort 0177555 very old 16-bit-int little-endian 
> archive
> +0beshort 0177555 very old 16-bit-int big-endian archive
> +0lelong  0177555 very old 32-bit-int little-endian 
> archive
> +0belong  0177555 very old 32-bit-int big-endian archive
> +
> +0leshort 0177545 old 16-bit-int little-endian archive
> +>2   string  __.SYMDEF   random library
> +0beshort 0177545 old 16-bit-int big-endian archive
> +>2   string  __.SYMDEF   random library
> +0lelong  0177545 old 32-bit-int little-endian archive
> +>4   string  __.SYMDEF   random library
> +0belong  0177545 old 32-bit-int big-endian archive
> +>4   string  __.SYMDEF   random library
> +
> +#
> +# From "pdp" (but why a 4-byte quantity?)
> +#
> +0lelong  0x39bed PDP-11 old archive
> +0lelong  0x39bee PDP-11 4.0 archive
> +
> +#
> +# XXX - what flavor of APL used this, and was it a variant of
> +# some ar archive format?  It's similar to, but not the same
> +# as, the APL workspace magic numbers in pdp.
> +#
> +0long0100554 apl workspace
> +
> +#
> +# System V Release 1 portable(?) archive format.
> +#
> +0string  =   System V Release 1 ar archive
> +!:mime   application/x-archive
> +
> +#
> +# Debian package; it's in the portable archive format, and needs to go
> +# before the entry for regular portable archives, as it's recognized as
> +# a portable archive whose first member has a name beginning with
> +# "debian".
>  #
>  0string  =!\ndebian
> -!:mime   application/x-debian-package
>  >8   string  debian-splitpart of multipart Debian package
> +!:mime   application/vnd.debian.binary-package
>  >8   string  debian-binary   Debian binary package
> +!:mime   application/vnd.debian.binary-package
>  >8   string  !debian
>  >68  string  >\0 (format %s)
>  # These next two lines do not work, because a bzip2 Debian archive
> @@ -49,18 +108,14 @@
>  #>84 string  gz  \b, uses gzip compression
>  #>136ledate  x   created: %s
>  
> -# other archives
> -0long0177555 very old archive
> -0short   0177555 very old PDP-11 archive
> -0long0177545 old archive
> -0short   0177545 old PDP-11 archive
> -0long0100554 apl workspace
> -0string  =   

Re: [patch] add UTF-8 support to column(1)

[Nicholas Marriott]

Hi

I don't know why it is off in xterm but oxtabs is the default in
ttydefaults.h and appears to on for mostly everything else, including ssh
and tmux (which just uses what forkpty gives it).
Hi Martijn,

Martijn van Duren wrote on Sat, Feb 27, 2016 at 01:22:53PM +0100:

> Here's my attempt to implement UTF-8 support in column(1).
> Besides the general UTF-8 conversions it does several other things
> to make it behave properly.

Two general remarks:

 1. This column(1) code seems to be relatively low quality.
When aiming at functional changes to low-quality code,
it is sometimes useful to keep cleanup and functional
changes seperate, in particular if either of them alone
already cause a large diff (mixing minor cleanup into a
functional diff is OK when the diff remains reasonably
easy to read, though).

 2. When you want to do several things that require large changes,
proceeding in steps can make review easier.

> Full changelist is as follow:
> - Make separator and input full UTF-8 aware.
> - Do proper character width count.

I agree with these two goals.

> This also fixes some indentation issues where the old code assumed
> that a tab also was one column wide.

I agree with your assessment that the old code mishandles tab
characters in the input.  There are three cases:

 1. In -t mode when '\t' is in -s (which it is by default).
There is no problem because the tabs get replaced with NULs
by strtok(3).
 2. In -t mode when '\t' is not in -s.
 3. In non-t mode.
What should happen in these two cases seems arguable.
One option would be to calculate the width of a tab such
that it advances to the next multiple of eight.
Another option would be to just replace each tab with
a single blank on the output side.
While the first option is closer to what happens now, the
second one seems more useful: When you explicitly change the
columnation and say that tabs in the input do not delimit
columns, it doesn't seem very useful to have them signify
subcolumns inside the new columns.

> - Replace tabs between columns with spaces.

I'm not convinced that should be changed.  In any case, it is unrelated
to UTF-8.

> The old code worked fine, but with UTF-8 and oxtabs in stty enabled
> the column positioning can get way off, and we can't expect everyone
> to run with "stty -oxtabs". Found with the help of nicm@.

On the console, UTF-8 doesn't work anyway, at all.  In an xterm(1),
it seems to me that "stty -oxtabs" is the default, so keeping the
tabs works by default.  If somebody manually sets "stty oxtabs"
(knowing that it cannot possibly work with UTF-8), they can easily
pipe column(1) output through expand(1).  Of course, we need to fix
expand(1), but that's a seperate matter.  I'm not sure we should
incorporate expand(1) functionality into random utilities that don't
yet have special tab handling just because there are some rare
corner cases where tabs might break.


Regarding the design of the program:  There are two very different
modes, -t and non-t.  In -t mode, total line widths are irrelevant
but calculated in input() anyway.  With UTF-8, that becomes more
expensive and harder to bear.  In non-t mode, total line widths
are the only widths that matter.

I think the ideal top-level data structure is as follows:

 - A list of input lines, stored as a pointer to the first
   line, either NULL-terminated or together with a variable
   storing the number of lines.
 - Each line is a list of fields, again stored as a pointer
   to the first field, this time definitely NULL-terminated
   for simplicity.
 - Each field is a struct containing a char * and an int width.

The input() function should generate this complete data structure
using getline(3) and mbtowc(3) in a single loop and remember the
maximum field width.  For the non-t case, only one field will be
generated per input line.

That way, we only need to parse the input once, we only need to
calculate widths once, we only need heavy malloc(3) lifting in one
function, and UTF-8 handling will only infect one single function,
input().  Probably, we only need one single call to mbtowc(3), one
single call to wcwidth(3), and no calls to *mbs*() functions except
one in main() for parsing and one in input() for using the -s option.
Probably, all the other functions can get away with functions like
fputs(3) or printf("...%s...").

I think it would be reasonable to do this refactoring in a first step,
maybe together with renaming length to width, then implement UTF-8
support in a second step.  Does that make sense to you?

I'm adding some specific comments in-line.  If any of my comments
do not seem helpful to you, just say so, i may have overlooked
something.

Yours,
  Ingo


> Index: column.c
> ===
> RCS file: /cvs/src/usr.bin/column/column.c,v
> retrieving revision 1.22
> diff -u -p -r1.22 column.c
> --- column.c  3 Nov 2015 

Re: tmux/window-copy.c - copy mode scroll behaviour

[Nicholas Marriott]

Hi

On Thu, Feb 11, 2016 at 08:05:54PM +0100, Michal Mazurek wrote:
> Hello,
> 
> I know the code is now locked, but I'd like to ask for comments on the
> following diff.
> 
> I believe that page-up and halfpage-up should have the same effect as
> running scroll-up (^Y) the equivalent number of times. There are two
> differences:
> - with line select active ('V' by default) pgup/pgdown messes up the
> selection. (steps to reproduce: select a line with 'V', press pgup,
> scroll down with ^E).
> - the 'x' position of the cursor doesn't change, even if the current
> line length becomes shorter.

I like the current behaviour, it is nice that page up then page down
will leave things unchanged no matter what line it ended up on.

> 
> The diff below modifies window_copy_pageup() and adds window_copy_pagedown().
> These changes remove the differences.
> 
> Both nvi and mg exhibit different behaviours from each other, from what
> is implemented now, and what I'm proposing. mg always moves the cursor
> to position 0. nvi retains the position if the new lines length is equal
> or longer, or moves the position to the first printable character
> otherwise.
> 
> If it looks OK I'd like to keep working on it, make
> MODEKEYCOPY_HALFPAGEUP and MODEKEYCOPY_HALFPAGEDOWN use these functions
> as well.
> 
> Index: window-copy.c
> ===
> RCS file: /cvs/src/usr.bin/tmux/window-copy.c,v
> retrieving revision 1.145
> diff -u -p -r1.145 window-copy.c
> --- window-copy.c 5 Feb 2016 10:20:06 -   1.145
> +++ window-copy.c 11 Feb 2016 18:22:59 -
> @@ -26,6 +26,7 @@
>  
>  struct screen *window_copy_init(struct window_pane *);
>  void window_copy_free(struct window_pane *);
> +void window_copy_pagedown(struct window_pane *);
>  void window_copy_resize(struct window_pane *, u_int, u_int);
>  void window_copy_key(struct window_pane *, struct client *, struct session *,
>   key_code, struct mouse_event *);
> @@ -324,15 +325,80 @@ window_copy_pageup(struct window_pane *w
>  {
>   struct window_copy_mode_data*data = wp->modedata;
>   struct screen   *s = >screen;
> - u_intn;
> + u_intn, ox, oy;
> +
> + oy = screen_hsize(data->backing) + data->cy - data->oy;
> + ox = window_copy_find_length(wp, oy);
> +
> + if (s->sel.lineflag == LINE_SEL_LEFT_RIGHT && oy == data->sely)
> + window_copy_other_end(wp);
> +
> + if (data->cx != ox) {
> + data->lastcx = data->cx;
> + data->lastsx = ox;
> + }
> + data->cx = data->lastcx;
>  
>   n = 1;
>   if (screen_size_y(s) > 2)
>   n = screen_size_y(s) - 2;
> +
>   if (data->oy + n > screen_hsize(data->backing))
>   data->oy = screen_hsize(data->backing);
>   else
>   data->oy += n;
> +
> + if (!data->screen.sel.flag || !data->rectflag) {
> + u_int py = screen_hsize(data->backing) + data->cy - data->oy;
> + u_int px = window_copy_find_length(wp, py);
> + if ((data->cx >= data->lastsx && data->cx != px) || data->cx > 
> px)
> + window_copy_cursor_end_of_line(wp);
> + }
> +
> + window_copy_update_selection(wp, 1);
> + window_copy_redraw_screen(wp);
> +}
> +
> +void
> +window_copy_pagedown(struct window_pane *wp)
> +{
> + struct window_copy_mode_data*data = wp->modedata;
> + struct screen   *s = >screen;
> + u_intn, ox, oy;
> +
> + oy = screen_hsize(data->backing) + data->cy - data->oy;
> + ox = window_copy_find_length(wp, oy);
> +
> + if (s->sel.lineflag == LINE_SEL_RIGHT_LEFT && oy == data->sely)
> + window_copy_other_end(wp);
> +
> + if (data->cx != ox) {
> + data->lastcx = data->cx;
> + data->lastsx = ox;
> + }
> + data->cx = data->lastcx;
> +
> + n = 1;
> + if (screen_size_y(s) > 2)
> + n = screen_size_y(s) - 2;
> +
> + if (data->oy < n)
> + data->oy = 0;
> + else
> + data->oy -= n;
> +
> + if (!data->screen.sel.flag || !data->rectflag) {
> + u_int py = screen_hsize(data->backing) + data->cy - data->oy;
> + u_int px = window_copy_find_length(wp, py);
> + if ((data->cx >= data->lastsx && data->cx != px) || data->cx > 
> px)
> + window_copy_cursor_end_of_line(wp);
> + }
> +
> + if (data->scroll_exit && data->oy == 0) {
> + window_pane_reset_mode(wp);
> + return;
> + }
> +
>   window_copy_update_selection(wp, 1);
>   window_copy_redraw_screen(wp);
>  }
> @@ -479,21 +545,8 @@ window_copy_key(struct window_pane *wp, 
>   window_copy_pageup(wp);
>   break;
>   case MODEKEYCOPY_NEXTPAGE:
> - n = 1;
> - if (screen_size_y(s) 

Re: I have a program I wish to submit for the base

[Nicholas Marriott]

Firstly, I don't think we need this in base and I think there is little
to no chance of it being taken, even if the code is improved.

Secondly:

- The code is still miles off style(9) and isn't really a consistent
  style within itself either.

- Forking uname(1)? What? No offence, but that is hilarious :-). Why
  fork uname(1) for uname(3) but not date(1) for gettimeofday(2)?

- Why would you fork sed either?

I think C is the wrong tool for this. Why not write a shell, perl, or
python script?

Then if people start to use it you could make a port.



On Fri, Jan 29, 2016 at 01:34:30AM -0600, Luke Small wrote:
> I think I fixed all your suggestions. I don't strictly adhere to kernel
> normal in the use of comments and I parse command-line arguments without
> using getopt(3), but the method is robust.
> 
> -Luke
> 
>  
>  o I definitely don't think camel case will be accepted
> 
>  o I'm pretty sure strtonum(3) is strongly preferred over strtod(3) et al.

> /*
>  * Copyright (c) 2016 Luke N. Small
>  *
>  * Permission to use, copy, modify, and distribute this software for any
>  * purpose with or without fee is hereby granted, provided that the above
>  * copyright notice and this permission notice appear in all copies.
>  *
>  * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
>  * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
>  * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
>  * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
>  * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
>  * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
>  * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
>  */
>  
> 
> /* Special thanks to Dan Mclaughlin for the ftp to sed idea
>  * 
>  * ftp -o - http://www.openbsd.org/ftp.html | \
>  * sed -n \
>  *  -e 's:$::' \
>  *-e 's:  \([^<]*\)<.*:\1:p' \
>  *-e 's:^\(   [hfr].*\):\1:p'
>  */
>  
>  
> #define EVENT_NOPOLL
> #define EVENT_NOSELECT
> 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> 
> struct mirror_st
> {
>   char * country_title;
>   char * mirror;
>   char * install_path;
>   double diff;
>   struct mirror_st * next;
> };
> 
> int ftp_cmp (const void * a, const void * b)
> {
>   struct mirror_st ** one = (struct mirror_st **)a;
>   struct mirror_st ** two = (struct mirror_st **)b;
> 
>   if ( (*one)->diff < (*two)->diff )
>   return -1;
>   if ( (*one)->diff > (*two)->diff )
>   return 1;
>   return 0;
> }
> 
> int country_cmp (const void * a, const void * b)
> {
>   struct mirror_st ** one = (struct mirror_st **)a;
>   struct mirror_st ** two = (struct mirror_st **)b;
>   
>   // list the USA mirrors first, it will subsort correctly
>   int8_t temp = !strncmp("USA", (*one)->country_title, 3);
>   if (temp != !strncmp("USA", (*two)->country_title, 3))
>   {
>   if (temp)
>   return -1;
>   return 1;
>   }
> 
>   return strcmp( (*one)->country_title, (*two)->country_title ) ;
> }
> 
> 
> double get_time_diff(struct timeval a, struct timeval b)
> {
> long sec;
> long usec;
> sec = b.tv_sec - a.tv_sec;
> usec = b.tv_usec - a.tv_usec;
> if (usec < 0)
> {
>   --sec;
>   usec += 100;
> }
> return sec + ((double)usec / 100.0);
> }
> 
> void manpage(char * a)
> {
>   errx(1, "%s [-s timeout] [-n maximum_mirrors_written]", a);
> }
> 
> 
> int main(int argc, char *argv[])
> {
>   pid_t ftp_pid, sed_pid, uname_pid;
>   int ftp_to_sed[2];
>   int sed_to_parent[2];
>   int uname_to_parent[2];
>   char uname_r[5], uname_m[20], character;
>   int i;
>   double s = 7;
>   int position, num, c, n = 5000;
>   FILE *input;
> 
>   if (argc > 1)
>   {
>   if (argc % 2 == 0)
>   manpage(argv[0]);
>   
>   position = 0;
>   while (++position < argc)
>   {
>   if (strlen(argv[position]) != 2)
>   manpage(argv[0]);
>   
>   if (!strcmp(argv[position], "-s"))
>   {
>   ++position;
>   c = -1;
>   i = 0;
>   while ((character = argv[position][++c]) != 
> '\0')
>   {
>   if (character == '.')
>   ++i;
>   
>   if ( ((character < '0' || character > 
> '9') && character != '.') || i > 1 )
>   {
>   

Re: Don't wrap the cursor in tmux in copy mode

[Nicholas Marriott]

Hi

tmux copy mode is not meant to work like vi, it is meant to work like
emacs, but emacs does do this so we can change it.

However, your diff is wrong. It prevents the cursor moving back when it
is at 0,0 _on screen_.

But if there is more than one screen worth of history, a left at 0,0 on
screen should scroll up one line in the history.

You can do something like "jot 1000" to generate plenty of history.



On Tue, Jan 26, 2016 at 04:32:39PM +0100, Michal Mazurek wrote:
> If the cursor is in position 0, 0 in copy mode, and the left arrow is
> pressed, the cursor position is wrapped to the end of line. A similar
> situation occurs at the last position and the right arrow. The following
> patch prevents that, and makes the behaviour more consistent with other
> software, such as nvi:
> 
> 
> Index: window-copy.c
> ===
> RCS file: /cvs/src/usr.bin/tmux/window-copy.c,v
> retrieving revision 1.144
> diff -u -p -r1.144 window-copy.c
> --- window-copy.c 19 Jan 2016 15:59:12 -  1.144
> +++ window-copy.c 26 Jan 2016 11:43:38 -
> @@ -1776,10 +1776,10 @@ window_copy_cursor_left(struct window_pa
>  {
>   struct window_copy_mode_data*data = wp->modedata;
>  
> - if (data->cx == 0) {
> + if (data->cx == 0 && data->cy > 0) {
>   window_copy_cursor_up(wp, 0);
>   window_copy_cursor_end_of_line(wp);
> - } else {
> + } else if (data->cx > 0) {
>   window_copy_update_cursor(wp, data->cx - 1, data->cy);
>   if (window_copy_update_selection(wp, 1))
>   window_copy_redraw_lines(wp, data->cy, 1);
> @@ -1799,10 +1799,10 @@ window_copy_cursor_right(struct window_p
>   px = window_copy_find_length(wp, py);
>   }
>  
> - if (data->cx >= px) {
> + if (data->cx >= px && data->cy < screen_size_y(>screen) - 1) {
>   window_copy_cursor_start_of_line(wp);
>   window_copy_cursor_down(wp, 0);
> - } else {
> + } else if (data->cx < px) {
>   window_copy_update_cursor(wp, data->cx + 1, data->cy);
>   if (window_copy_update_selection(wp, 1))
>   window_copy_redraw_lines(wp, data->cy, 1);
> 
> 
> Also remove two unused variables (no binary change):
> 
> Index: cmd-swap-pane.c
> ===
> RCS file: /cvs/src/usr.bin/tmux/cmd-swap-pane.c,v
> retrieving revision 1.25
> diff -u -p -r1.25 cmd-swap-pane.c
> --- cmd-swap-pane.c   19 Jan 2016 15:59:12 -  1.25
> +++ cmd-swap-pane.c   26 Jan 2016 15:26:14 -
> @@ -45,28 +45,23 @@ const struct cmd_entry cmd_swap_pane_ent
>  enum cmd_retval
>  cmd_swap_pane_exec(struct cmd *self, struct cmd_q *cmdq)
>  {
> - struct winlink  *src_wl, *dst_wl;
>   struct window   *src_w, *dst_w;
>   struct window_pane  *tmp_wp, *src_wp, *dst_wp;
>   struct layout_cell  *src_lc, *dst_lc;
>   u_intsx, sy, xoff, yoff;
>  
> - dst_wl = cmdq->state.tflag.wl;
> - dst_w = dst_wl->window;
> + dst_w = cmdq->state.tflag.wl->window;
>   dst_wp = cmdq->state.tflag.wp;
> - src_wl = cmdq->state.sflag.wl;
> - src_w = src_wl->window;
> + src_w = cmdq->state.sflag.wl->window;
>   src_wp = cmdq->state.sflag.wp;
>   server_unzoom_window(dst_w);
>  
>   if (args_has(self->args, 'D')) {
> - src_wl = dst_wl;
>   src_w = dst_w;
>   src_wp = TAILQ_NEXT(dst_wp, entry);
>   if (src_wp == NULL)
>   src_wp = TAILQ_FIRST(_w->panes);
>   } else if (args_has(self->args, 'U')) {
> - src_wl = dst_wl;
>   src_w = dst_w;
>   src_wp = TAILQ_PREV(dst_wp, window_panes, entry);
>   if (src_wp == NULL)
> 
> -- 
> Michal Mazurek
> 

Re: Simplify less(1) off_t formatting

[Nicholas Marriott]

I like the idea, but I don't like calling them ap_off_t and offttoa, I'd
just keep ap_pos and postoa and remove the linenum functions.



On Tue, Jan 12, 2016 at 11:50:51PM -0500, Michael McConville wrote:
> I'm working on bigger simplifications for less's string formatting, but
> this is a good start. We've removed the off_t aliases for linenums and
> positions, so we now have two identical sets of off_t formatting and
> appending functions. The below diff unifies them.
> 
> As you can see, there is more cleanup to be done.
> 
> Thoughts?
> 
> 
> Index: less.h
> ===
> RCS file: /cvs/src/usr.bin/less/less.h,v
> retrieving revision 1.21
> diff -u -p -r1.21 less.h
> --- less.h12 Jan 2016 17:48:04 -  1.21
> +++ less.h13 Jan 2016 04:43:15 -
> @@ -205,6 +205,5 @@ struct textlist {
>  #include "funcs.h"
>  
>  /* Functions not included in funcs.h */
> -void postoa(off_t, char *, size_t);
> -void linenumtoa(off_t, char *, size_t);
> +void offttoa(off_t, char *, size_t);
>  void inttoa(int, char *, size_t);
> Index: line.c
> ===
> RCS file: /cvs/src/usr.bin/less/line.c,v
> retrieving revision 1.16
> diff -u -p -r1.16 line.c
> --- line.c12 Jan 2016 17:48:04 -  1.16
> +++ line.c13 Jan 2016 04:43:15 -
> @@ -178,7 +178,7 @@ plinenum(off_t pos)
>   char buf[INT_STRLEN_BOUND(pos) + 2];
>   int n;
>  
> - linenumtoa(linenum, buf, sizeof (buf));
> + offttoa(linenum, buf, sizeof(buf));
>   n = strlen(buf);
>   if (n < MIN_LINENUM_WIDTH)
>   n = MIN_LINENUM_WIDTH;
> Index: output.c
> ===
> RCS file: /cvs/src/usr.bin/less/output.c,v
> retrieving revision 1.14
> diff -u -p -r1.14 output.c
> --- output.c  12 Jan 2016 17:48:04 -  1.14
> +++ output.c  13 Jan 2016 04:43:15 -
> @@ -148,8 +148,7 @@ funcname(type num, char *buf, size_t len
>   (void) strlcpy(buf, s, len);\
>  }
>  
> -TYPE_TO_A_FUNC(postoa, off_t)
> -TYPE_TO_A_FUNC(linenumtoa, off_t)
> +TYPE_TO_A_FUNC(offttoa, off_t)
>  TYPE_TO_A_FUNC(inttoa, int)
>  
>  /*
> @@ -173,7 +172,7 @@ iprint_linenum(off_t num)
>  {
>   char buf[INT_STRLEN_BOUND(num)];
>  
> - linenumtoa(num, buf, sizeof (buf));
> + offttoa(num, buf, sizeof(buf));
>   putstr(buf);
>   return (strlen(buf));
>  }
> Index: prompt.c
> ===
> RCS file: /cvs/src/usr.bin/less/prompt.c,v
> retrieving revision 1.19
> diff -u -p -r1.19 prompt.c
> --- prompt.c  12 Jan 2016 23:01:23 -  1.19
> +++ prompt.c  13 Jan 2016 04:43:15 -
> @@ -116,23 +116,11 @@ ap_char(char c)
>   * Append a off_t (as a decimal integer) to the end of the message.
>   */
>  static void
> -ap_pos(off_t pos)
> +ap_off_t(off_t pos)
>  {
>   char buf[INT_STRLEN_BOUND(pos) + 2];
>  
> - postoa(pos, buf, sizeof buf);
> - ap_str(buf);
> -}
> -
> -/*
> - * Append a line number to the end of the message.
> - */
> -static void
> -ap_linenum(off_t linenum)
> -{
> - char buf[INT_STRLEN_BOUND(linenum) + 2];
> -
> - linenumtoa(linenum, buf, sizeof buf);
> + offttoa(pos, buf, sizeof(buf));
>   ap_str(buf);
>  }
>  
> @@ -245,7 +233,7 @@ protochar(int c, int where)
>   case 'b':   /* Current byte offset */
>   pos = curr_byte(where);
>   if (pos != -1)
> - ap_pos(pos);
> + ap_off_t(pos);
>   else
>   ap_quest();
>   break;
> @@ -255,7 +243,7 @@ protochar(int c, int where)
>   case 'd':   /* Current page number */
>   linenum = currline(where);
>   if (linenum > 0 && sc_height > 1)
> - ap_linenum(PAGE_NUM(linenum));
> + ap_off_t(PAGE_NUM(linenum));
>   else
>   ap_quest();
>   break;
> @@ -266,13 +254,13 @@ protochar(int c, int where)
>   ap_quest();
>   } else if (len == 0) {
>   /* An empty file has no pages. */
> - ap_linenum(0);
> + ap_off_t(0);
>   } else {
>   linenum = find_linenum(len - 1);
>   if (linenum <= 0)
>   ap_quest();
>   else
> - ap_linenum(PAGE_NUM(linenum));
> + ap_off_t(PAGE_NUM(linenum));
>   }
>   break;
>   case 'E':   /* Editor name */
> @@ -293,7 +281,7 @@ protochar(int c, int where)
>   case 'l':   /* Current line number */
>   linenum = currline(where);
>   if (linenum != 0)
> - ap_linenum(linenum);
> +   

Re: Simplify less(1) off_t formatting

[Nicholas Marriott]

Looks good, ok nicm




On Wed, Jan 13, 2016 at 05:42:56PM -0500, Michael McConville wrote:
> Nicholas Marriott wrote:
> > I like the idea, but I don't like calling them ap_off_t and offttoa,
> > I'd just keep ap_pos and postoa and remove the linenum functions.
> 
> Does this look good?
> 
> 
> Index: less.h
> ===
> RCS file: /cvs/src/usr.bin/less/less.h,v
> retrieving revision 1.21
> diff -u -p -r1.21 less.h
> --- less.h12 Jan 2016 17:48:04 -  1.21
> +++ less.h13 Jan 2016 22:40:49 -
> @@ -206,5 +206,4 @@ struct textlist {
>  
>  /* Functions not included in funcs.h */
>  void postoa(off_t, char *, size_t);
> -void linenumtoa(off_t, char *, size_t);
>  void inttoa(int, char *, size_t);
> Index: line.c
> ===
> RCS file: /cvs/src/usr.bin/less/line.c,v
> retrieving revision 1.16
> diff -u -p -r1.16 line.c
> --- line.c12 Jan 2016 17:48:04 -  1.16
> +++ line.c13 Jan 2016 22:40:49 -
> @@ -178,7 +178,7 @@ plinenum(off_t pos)
>   char buf[INT_STRLEN_BOUND(pos) + 2];
>   int n;
>  
> - linenumtoa(linenum, buf, sizeof (buf));
> + postoa(linenum, buf, sizeof(buf));
>   n = strlen(buf);
>   if (n < MIN_LINENUM_WIDTH)
>   n = MIN_LINENUM_WIDTH;
> Index: output.c
> ===
> RCS file: /cvs/src/usr.bin/less/output.c,v
> retrieving revision 1.14
> diff -u -p -r1.14 output.c
> --- output.c  12 Jan 2016 17:48:04 -  1.14
> +++ output.c  13 Jan 2016 22:40:50 -
> @@ -149,7 +149,6 @@ funcname(type num, char *buf, size_t len
>  }
>  
>  TYPE_TO_A_FUNC(postoa, off_t)
> -TYPE_TO_A_FUNC(linenumtoa, off_t)
>  TYPE_TO_A_FUNC(inttoa, int)
>  
>  /*
> @@ -173,7 +172,7 @@ iprint_linenum(off_t num)
>  {
>   char buf[INT_STRLEN_BOUND(num)];
>  
> - linenumtoa(num, buf, sizeof (buf));
> + postoa(num, buf, sizeof(buf));
>   putstr(buf);
>   return (strlen(buf));
>  }
> Index: prompt.c
> ===
> RCS file: /cvs/src/usr.bin/less/prompt.c,v
> retrieving revision 1.19
> diff -u -p -r1.19 prompt.c
> --- prompt.c  12 Jan 2016 23:01:23 -  1.19
> +++ prompt.c  13 Jan 2016 22:40:50 -
> @@ -120,19 +120,7 @@ ap_pos(off_t pos)
>  {
>   char buf[INT_STRLEN_BOUND(pos) + 2];
>  
> - postoa(pos, buf, sizeof buf);
> - ap_str(buf);
> -}
> -
> -/*
> - * Append a line number to the end of the message.
> - */
> -static void
> -ap_linenum(off_t linenum)
> -{
> - char buf[INT_STRLEN_BOUND(linenum) + 2];
> -
> - linenumtoa(linenum, buf, sizeof buf);
> + postoa(pos, buf, sizeof(buf));
>   ap_str(buf);
>  }
>  
> @@ -255,7 +243,7 @@ protochar(int c, int where)
>   case 'd':   /* Current page number */
>   linenum = currline(where);
>   if (linenum > 0 && sc_height > 1)
> - ap_linenum(PAGE_NUM(linenum));
> + ap_pos(PAGE_NUM(linenum));
>   else
>   ap_quest();
>   break;
> @@ -266,13 +254,13 @@ protochar(int c, int where)
>   ap_quest();
>   } else if (len == 0) {
>   /* An empty file has no pages. */
> - ap_linenum(0);
> + ap_pos(0);
>   } else {
>   linenum = find_linenum(len - 1);
>   if (linenum <= 0)
>   ap_quest();
>   else
> - ap_linenum(PAGE_NUM(linenum));
> + ap_pos(PAGE_NUM(linenum));
>   }
>   break;
>   case 'E':   /* Editor name */
> @@ -293,7 +281,7 @@ protochar(int c, int where)
>   case 'l':   /* Current line number */
>   linenum = currline(where);
>   if (linenum != 0)
> - ap_linenum(linenum);
> + ap_pos(linenum);
>   else
>   ap_quest();
>   break;
> @@ -303,7 +291,7 @@ protochar(int c, int where)
>   (linenum = find_linenum(len)) <= 0)
>   ap_quest();
>   else
> - ap_linenum(linenum-1);
> + ap_pos(linenum-1);
>   break;
>   case 'm':   /* Number of files */
>   n = ntags();

Re: ksh another home/end pair

[Nicholas Marriott]

tmux is doing the right thing and changing what it generates would break
other programs.

ksh should use terminfo to work out what to expect, but we have always
resisted linking ksh against ncurses.

Your change is the best we can do without making ksh use ncurses.



On Wed, Dec 30, 2015 at 01:19:24AM -0500, Ted Unangst wrote:
> Mark Kettenis wrote:
> > > From: "Ted Unangst" 
> > > Date: Tue, 29 Dec 2015 12:11:25 -0500
> > > 
> > > In tmux, home and end send different bytes. I don't know why, but I want
> > > things to just work. We already have two different keys here, so what's 
> > > one
> > > more? (how many can there be...?)
> > 
> > Isn't that somehowa tmux bug?  I mean, isn't tmux supposed to be
> > compatible with some standard terminal type?
> 
> It could be. I don't know why tmux chooses a different set of bytes to
> represent home/end than xterm. I mostly don't care.
> 
> There was a thread on misc about this where it was suggested I either
> reconfigure ksh or tmux. Or both. But that's silly. This should work out of
> the box.
> 
> If somebody would like to propose a diff for tmux, I'd be happy with that, but
> I'm not sure where to begin myself. ksh already appears to be trying to
> support a nonconflicting superset of terminal types without resorting to
> termcap, so it seemed logical to fix here.
> 
> http://marc.info/?l=openbsd-misc=144906064413240=2
> 

Re: ksh rename global e

[Nicholas Marriott]

yes please, ok nicm



On Tue, Dec 29, 2015 at 12:15:25PM -0500, Ted Unangst wrote:
> I'm slowly trimming down some of the -Wshadow warnings in bin and one big
> offender is ksh. Namely, it has a local variable e that shadows a global e.
> 
> -struct env *e;
> +struct env *genv;
> 
> Normally I rename the local, but in this case I think the global deserves a
> better name. Note that this variable is also used as part of a macro ATEMP.
> Accidentally using this macro with a local of the same in scope would be
> ungood.
> 
> 
> Index: c_ksh.c
> ===
> RCS file: /cvs/src/bin/ksh/c_ksh.c,v
> retrieving revision 1.47
> diff -u -p -r1.47 c_ksh.c
> --- c_ksh.c   14 Dec 2015 13:59:42 -  1.47
> +++ c_ksh.c   29 Dec 2015 14:58:28 -
> @@ -710,7 +710,7 @@ c_typeset(char **wp)
>   /* list variables and attributes */
>   flag = fset | fclr; /* no difference at this point.. */
>   if (func) {
> - for (l = e->loc; l; l = l->next) {
> + for (l = genv->loc; l; l = l->next) {
>   for (p = ktsort(>funs); (vp = *p++); ) {
>   if (flag && (vp->flag & flag) == 0)
>   continue;
> @@ -723,7 +723,7 @@ c_typeset(char **wp)
>   }
>   }
>   } else {
> - for (l = e->loc; l; l = l->next) {
> + for (l = genv->loc; l; l = l->next) {
>   for (p = ktsort(>vars); (vp = *p++); ) {
>   struct tbl *tvp;
>   int any_set = 0;
> @@ -1271,15 +1271,15 @@ c_getopts(char **wp)
>   return 1;
>   }
>  
> - if (e->loc->next == NULL) {
> + if (genv->loc->next == NULL) {
>   internal_errorf(0, "c_getopts: no argv");
>   return 1;
>   }
>   /* Which arguments are we parsing... */
>   if (*wp == NULL)
> - wp = e->loc->next->argv;
> + wp = genv->loc->next->argv;
>   else
> - *--wp = e->loc->next->argv[0];
> + *--wp = genv->loc->next->argv[0];
>  
>   /* Check that our saved state won't cause a core dump... */
>   for (argc = 0; wp[argc]; argc++)
> Index: c_sh.c
> ===
> RCS file: /cvs/src/bin/ksh/c_sh.c,v
> retrieving revision 1.57
> diff -u -p -r1.57 c_sh.c
> --- c_sh.c14 Dec 2015 13:59:42 -  1.57
> +++ c_sh.c29 Dec 2015 15:01:42 -
> @@ -30,7 +30,7 @@ c_label(char **wp)
>  int
>  c_shift(char **wp)
>  {
> - struct block *l = e->loc;
> + struct block *l = genv->loc;
>   int n;
>   long val;
>   char *arg;
> @@ -208,7 +208,7 @@ c_dot(char **wp)
>   /* Set positional parameters? */
>   if (wp[builtin_opt.optind + 1]) {
>   argv = wp + builtin_opt.optind;
> - argv[0] = e->loc->argv[0]; /* preserve $0 */
> + argv[0] = genv->loc->argv[0]; /* preserve $0 */
>   for (argc = 0; argv[argc + 1]; argc++)
>   ;
>   } else {
> @@ -529,7 +529,7 @@ c_exitreturn(char **wp)
>   /* need to tell if this is exit or return so trap exit will
>* work right (POSIX)
>*/
> - for (ep = e; ep; ep = ep->oenv)
> + for (ep = genv; ep; ep = ep->oenv)
>   if (STOP_RETURN(ep->type)) {
>   how = LRETURN;
>   break;
> @@ -570,7 +570,7 @@ c_brkcont(char **wp)
>   }
>  
>   /* Stop at E_NONE, E_PARSE, E_FUNC, or E_INCL */
> - for (ep = e; ep && !STOP_BRKCONT(ep->type); ep = ep->oenv)
> + for (ep = genv; ep && !STOP_BRKCONT(ep->type); ep = ep->oenv)
>   if (ep->type == E_LOOP) {
>   if (--quit == 0)
>   break;
> @@ -605,7 +605,7 @@ int
>  c_set(char **wp)
>  {
>   int argi, setargs;
> - struct block *l = e->loc;
> + struct block *l = genv->loc;
>   char **owp = wp;
>  
>   if (wp[1] == NULL) {
> @@ -808,20 +808,20 @@ c_exec(char **wp)
>   int i;
>  
>   /* make sure redirects stay in place */
> - if (e->savefd != NULL) {
> + if (genv->savefd != NULL) {
>   for (i = 0; i < NUFILE; i++) {
> - if (e->savefd[i] > 0)
> - close(e->savefd[i]);
> + if (genv->savefd[i] > 0)
> + close(genv->savefd[i]);
>   /*
>* For ksh keep anything > 2 private,
>* for sh, let them be (POSIX says what
>* happens is unspecified and the bourne shell
>* keeps them open).
>*/
> - if (!Flag(FSH) && i > 2 && e->savefd[i])
> + if (!Flag(FSH) && i > 2 && 

Re: utf8 input in el_gets(3)

[Nicholas Marriott]

Go for it, IIRC there are a few new strcpy that will need care to change
to strlcpy but otherwise it should be easy enough.



On Thu, Dec 24, 2015 at 04:30:37PM -0500, Christian Heckendorf wrote:
> * Michael McConville  [24.12.2015. @16:19:03 -0500]:
> 
> > Christian Heckendorf wrote:
> > > A couple of somewhat recent changes in NetBSD's libedit permit
> > > el_gets(3) to accept multibyte characters if the locale supports
> > > it.
> > > 
> > > A notable user of this function is sftp(1) which will allow users to
> > > input multibyte characters, in filenames for example, once the diff
> > > is applied.
> > 
> > I remember someone recently mentioning that libedit is due for an
> > update. Maybe we should try to include this in a full sync.
> > 
> 
> Assuming someone isn't already working on it, I volunteer.
> 
> > > Index: eln.c
> > > ===
> > > RCS file: /cvs/src/lib/libedit/eln.c,v
> > > retrieving revision 1.4
> > > diff -u -p -r1.4 eln.c
> > > --- eln.c 20 May 2014 11:59:03 -  1.4
> > > +++ eln.c 24 Dec 2015 19:34:09 -
> > > @@ -74,9 +74,18 @@ el_gets(EditLine *el, int *nread)
> > >  {
> > >   const wchar_t *tmp;
> > >  
> > > - el->el_flags |= IGNORE_EXTCHARS;
> > > + if (!(el->el_flags & CHARSET_IS_UTF8))
> > > + el->el_flags |= IGNORE_EXTCHARS;
> > >   tmp = el_wgets(el, nread);
> > > - el->el_flags &= ~IGNORE_EXTCHARS;
> > > + if (tmp != NULL) {
> > > + size_t nwread = 0;
> > > + int i;
> > > + for (i = 0; i < *nread; i++)
> > > + nwread += ct_enc_width(tmp[i]);
> > > + *nread = (int)nwread;
> > > + }
> > > + if (!(el->el_flags & CHARSET_IS_UTF8))
> > > + el->el_flags &= ~IGNORE_EXTCHARS;
> > >   return ct_encode_string(tmp, >el_lgcyconv);
> > >  }
> > >  
> > > 
> > 
> 

Re: less.1: Update for LESSCHARSET removal

[Nicholas Marriott]

On Mon, Nov 23, 2015 at 11:58:19AM +0100, Theo Buehler wrote:
> On Mon, Nov 23, 2015 at 01:19:28AM -0500, Michael Reed wrote:
> 
> [...]
> >  .Nm
> [...]
> > +will determine the character set to use from the environment (see
> > +.Xr locale 1 ) .
> 
> I don't think this is correct.
> 
> The only call to setlocale(1) is 'setlocale(LC_ALL, "");' in charset.c.
> The LC_* environment variables are mentioned in the manual but appear to
> be unused, so I think we should delete them from the manual as well.

But setlocale uses them. I think the diff is fine.

Re: lesskey.1: Don't mention email address

[Nicholas Marriott]

ok nicm


On Mon, Nov 23, 2015 at 11:45:28AM +0100, Theo Buehler wrote:
> On Mon, Nov 23, 2015 at 01:24:19AM -0500, Michael Reed wrote:
> > This was already done for less.1, as OpenBSD's less(1) differs
> > substantially from upstream.
> > 
> Agreed.  While there, I think we should also remove the commented-out
> CAVEATS section on MS-DOS and OS/2 quirks.
> 
> ok?
> 
> Index: usr.bin/less/lesskey.1
> ===
> RCS file: /var/cvs/src/usr.bin/less/lesskey.1,v
> retrieving revision 1.13
> diff -u -p -r1.13 lesskey.1
> --- usr.bin/less/lesskey.15 Nov 2015 22:08:44 -   1.13
> +++ usr.bin/less/lesskey.123 Nov 2015 10:41:10 -
> @@ -448,10 +448,3 @@ file.
>  .Xr less 1
>  .Sh AUTHORS
>  .An Mark Nudelman
> -.Pp
> -Send bug reports or comments to
> -.Aq Mt bug\-l...@gnu.org .
> -.\" .Sh CAVEATS
> -.\" On MS-DOS and OS/2 systems, certain keys send a sequence of characters
> -.\" which start with a NUL character (0).
> -.\" This NUL character should be represented as \e340 in a lesskey file.
> 

Re: less.1: Update for LESSCHARSET removal

[Nicholas Marriott]

sure, ok nicm


On Mon, Nov 23, 2015 at 12:56:14PM +0100, Theo Buehler wrote:
> On Mon, Nov 23, 2015 at 11:39:55AM +0000, Nicholas Marriott wrote:
> > On Mon, Nov 23, 2015 at 11:58:19AM +0100, Theo Buehler wrote:
> > > On Mon, Nov 23, 2015 at 01:19:28AM -0500, Michael Reed wrote:
> > > 
> > > [...]
> > > >  .Nm
> > > [...]
> > > > +will determine the character set to use from the environment (see
> > > > +.Xr locale 1 ) .
> > > 
> > > I don't think this is correct.
> > > 
> > > The only call to setlocale(1) is 'setlocale(LC_ALL, "");' in charset.c.
> > > The LC_* environment variables are mentioned in the manual but appear to
> > > be unused, so I think we should delete them from the manual as well.
> > 
> > But setlocale uses them. I think the diff is fine.
> > 
> Hm.  This isn't mentioned in the setlocale manual page, but you're
> right.  Thank you for the correction.
> 
> So ok to commit as it is?
> 

Re: int -> size_t in ksh

[Nicholas Marriott]

Just look at the trouble changing one int->size_t is causing in lex ;-).


On Sun, Nov 22, 2015 at 08:18:55AM +0000, Nicholas Marriott wrote:
> On Sun, Nov 22, 2015 at 01:59:07AM +, B?hler  Theo wrote:
> > > It seems apparent that they never go negative. All instances are used to
> > > manipulate strlen() values.
> > >
> > > ok?
> > 
> > the first hunk is fine with me. I don't understand the second one:
> > why only convert w? Aren't all three ints guaranteed to be unsigned there?
> 
> Don't change i to size_t, it is not a size. Personally I wouldn't change
> this, int instead of size_t is everywhere in code this old, it would be
> a lot of churn, I don't think it necessarily improves anything and the
> risk of a mistake is high.

Re: less.1: secure mode is not a compile-time option

[Nicholas Marriott]

I just applied it actually, thanks



On Sun, Nov 15, 2015 at 09:04:58PM +, Nicholas Marriott wrote:
> ok nicm
> 
> 
> On Sun, Nov 15, 2015 at 03:56:12PM -0500, Michael Reed wrote:
> > From https://github.com/gdamore/less-fork/blob/master/CHANGES#L35-L37
> > 
> >  * SECURE mode (limited features) is removed - this version of less is 
> > always
> >fully featured.  (It was a compile-time option.)  Download official less
> >if you need restricted mode. (Hint: zones or chroot are better 
> > solutions!)
> > 
> > 
> > 
> > Index: less.1
> > ===
> > RCS file: /cvs/src/usr.bin/less/less.1,v
> > retrieving revision 1.49
> > diff -u -p -r1.49 less.1
> > --- less.1  8 Nov 2015 09:56:38 -   1.49
> > +++ less.1  15 Nov 2015 20:54:16 -
> > @@ -1756,8 +1756,6 @@ Metacharacters in filenames, such as "*"
> >  .It " "
> >  Filename completion (TAB, ^L).
> >  .El
> > -.Pp
> > -Less can also be compiled to be permanently in "secure" mode.
> >  .Sh COMPATIBILITY WITH MORE
> >  If the environment variable
> >  .Ev LESS_IS_MORE
> > 

Re: less.1: secure mode is not a compile-time option

[Nicholas Marriott]

ok nicm


On Sun, Nov 15, 2015 at 03:56:12PM -0500, Michael Reed wrote:
> From https://github.com/gdamore/less-fork/blob/master/CHANGES#L35-L37
> 
>  * SECURE mode (limited features) is removed - this version of less is always
>fully featured.  (It was a compile-time option.)  Download official less
>if you need restricted mode. (Hint: zones or chroot are better solutions!)
> 
> 
> 
> Index: less.1
> ===
> RCS file: /cvs/src/usr.bin/less/less.1,v
> retrieving revision 1.49
> diff -u -p -r1.49 less.1
> --- less.18 Nov 2015 09:56:38 -   1.49
> +++ less.115 Nov 2015 20:54:16 -
> @@ -1756,8 +1756,6 @@ Metacharacters in filenames, such as "*"
>  .It " "
>  Filename completion (TAB, ^L).
>  .El
> -.Pp
> -Less can also be compiled to be permanently in "secure" mode.
>  .Sh COMPATIBILITY WITH MORE
>  If the environment variable
>  .Ev LESS_IS_MORE
> 

Re: More less(1)

[Nicholas Marriott]

ok nicm


On Wed, Nov 11, 2015 at 11:45:07PM -0500, Michael McConville wrote:
> That helper function I just tweaked has a total of one usage. Maybe just
> inline it?
> 
> 
> Index: opttbl.c
> ===
> RCS file: /cvs/src/usr.bin/less/opttbl.c,v
> retrieving revision 1.16
> diff -u -p -r1.16 opttbl.c
> --- opttbl.c  12 Nov 2015 03:13:37 -  1.16
> +++ opttbl.c  12 Nov 2015 04:43:11 -
> @@ -466,18 +466,6 @@ findopt(int c)
>  }
>  
>  /*
> - *
> - */
> -static int
> -is_optchar(unsigned char c)
> -{
> - if (isupper(c) || islower(c) || c == '-')
> - return (1);
> - else
> - return (0);
> -}
> -
> -/*
>   * Find an option in the option table, given its option name.
>   * p_optname is the (possibly partial) name to look for, and
>   * is updated to point after the matched name.
> @@ -512,7 +500,10 @@ findopt_name(char **p_optname, char **p_
>*/
>   for (uppercase = 0;  uppercase <= 1;  uppercase++) {
>   len = sprefix(optname, oname->oname, uppercase);
> - if (len <= 0 || is_optchar(optname[len])) {
> + if (len <= 0 ||
> + isupper((unsigned char)optname[len]) ||
> + islower((unsigned char)optname[len]) ||
> + optname[len] == '-') {
>   /*
>* We didn't use all of the option name.
>*/
> 

Re: more utf8 less ebcdic

[Nicholas Marriott]

How about just lose chardef[] entirely now?


Index: charset.c
===
RCS file: /cvs/src/usr.bin/less/charset.c,v
retrieving revision 1.16
diff -u -p -r1.16 charset.c
--- charset.c   7 Nov 2015 18:06:38 -   1.16
+++ charset.c   11 Nov 2015 09:59:02 -
@@ -23,29 +23,10 @@
 
 int utf_mode = 0;
 
-#defineIS_BINARY_CHAR  01
-#defineIS_CONTROL_CHAR 02
-
-static char chardef[256];
 static const char *binfmt = NULL;
 static const char *utfbinfmt = NULL;
 int binattr = AT_STANDOUT;
 
-static void
-ilocale(void)
-{
-   int c;
-
-   for (c = 0; c < sizeof (chardef); c++) {
-   if (isprint(c))
-   chardef[c] = 0;
-   else if (iscntrl(c))
-   chardef[c] = IS_CONTROL_CHAR;
-   else
-   chardef[c] = IS_BINARY_CHAR|IS_CONTROL_CHAR;
-   }
-}
-
 static int
 checkfmt(const char *s)
 {
@@ -146,29 +127,18 @@ attr:
 }
 
 /*
- *
- */
-static void
-set_charset(void)
-{
-   char *s;
-
-   s = nl_langinfo(CODESET);
-   if (s && strcasecmp(s, "utf-8") == 0)
-   utf_mode = 1;
-
-   ilocale();
-}
-
-/*
  * Initialize charset data structures.
  */
 void
 init_charset(void)
 {
+   char *s;
+
setlocale(LC_ALL, "");
 
-   set_charset();
+   s = nl_langinfo(CODESET);
+   if (s && strcasecmp(s, "utf-8") == 0)
+   utf_mode = 1;
 
setbinfmt("LESSBINFMT", , "*s<%02X>");
setbinfmt("LESSUTFBINFMT", , "");
@@ -183,7 +153,7 @@ binary_char(LWCHAR c)
if (utf_mode)
return (is_ubin_char(c));
c &= 0377;
-   return (chardef[c] & IS_BINARY_CHAR);
+   return (!isprint((unsigned char)c) && !iscntrl((unsigned char)c));
 }
 
 /*
@@ -193,7 +163,7 @@ int
 control_char(LWCHAR c)
 {
c &= 0377;
-   return (chardef[c] & IS_CONTROL_CHAR);
+   return (iscntrl((unsigned char)c));
 }
 
 /*

Re: [PATCH] add F13-F24 and colour support to console terminfo

[Nicholas Marriott]

Perhaps it should go in FAQ section 7, or somewhere else, I don't know
where. Note pccon is only for some platforms.


On Mon, Nov 09, 2015 at 08:50:12AM +, Tati Chevron wrote:
> On Mon, Nov 09, 2015 at 08:23:08AM +0000, Nicholas Marriott wrote:
> >On Sun, Nov 08, 2015 at 09:01:52PM +, Tati Chevron wrote:
> >>Is there any reason why the OpenBSD installer still defaults to a
> >>terminal type of vt-220 for the VGA framebuffer console, and doesn't
> >>even suggest pccon as an alternative?
> >
> >In the installer itself? vt220 works fine and reliably and - importantly
> >- is small and fits on the ramdisks. Who needs to press F13 in the
> >installer?
> 
> Agreed, I didn't mean the installer itself, that does work fine with vt220.
> 
> >If you mean in /etc/ttys, this has been discussed before and mostly the
> >reason it has stayed is that terminfo entries propagate very slowly and
> >vt220 is available everywhere, no matter if the user ssh to some ancient
> >OS. It's easy to change anyway.
> 
> Ah, then this is a documentation issue - I always assumed that the terminal 
> type specified in the installer was propogated to /etc/ttys during the 
> install, and used as a default for the console.
> 
> It is indeed easy to change, but the existance of pccon isn't exactly widely 
> documented.  To be honest, I knew about it, but seeing that it was broken and 
> not used by default in the installer, nor on the console after installation, 
> I assumed that it was abandoned and unmaintained.
> 
> -- 
> Tati Chevron
> Perl and FORTRAN specialist.
> SWABSIT development and migration department.
> http://www.swabsit.com

Re: [PATCH] add F13-F24 and colour support to console terminfo

[Nicholas Marriott]

On Sun, Nov 08, 2015 at 09:01:52PM +, Tati Chevron wrote:
> Hi,
> 
> On Sun, Nov 08, 2015 at 08:18:55PM +, Nicholas Marriott wrote:
> >Hi
> >
> >I think it should be Shift-F1 etc, but since this is a general problem
> >with the xterm entry not just on OpenBSD, I suggest you discuss it with
> >upstream. The terminfo database is maintained as part of ncurses.
> 
> OK, I'll look into that, and maintain a local patch here for X for the time 
> being.
> 
> Is there any reason why the OpenBSD installer still defaults to a
> terminal type of vt-220 for the VGA framebuffer console, and doesn't
> even suggest pccon as an alternative?

In the installer itself? vt220 works fine and reliably and - importantly
- is small and fits on the ramdisks. Who needs to press F13 in the
installer?

If you mean in /etc/ttys, this has been discussed before and mostly the
reason it has stayed is that terminfo entries propagate very slowly and
vt220 is available everywhere, no matter if the user ssh to some ancient
OS. It's easy to change anyway.

Re: less(1) cleanup

[Nicholas Marriott]

ok nicm



On Sun, Nov 08, 2015 at 11:42:27PM -0500, Michael McConville wrote:
> Here, I:
> 
>  1) remove a useless comment
>  2) remove a needless void* cast
>  3) remove a bunch of NULL-checks for free()
>  4) simplify a little associated logic
> 
> Call me a one trick pony. 
> 
> ok?
> 
> 
> Index: cmdbuf.c
> ===
> RCS file: /cvs/src/usr.bin/less/cmdbuf.c,v
> retrieving revision 1.11
> diff -u -p -r1.11 cmdbuf.c
> --- cmdbuf.c  9 Nov 2015 04:10:57 -   1.11
> +++ cmdbuf.c  9 Nov 2015 04:38:23 -
> @@ -918,13 +918,8 @@ init_compl(void)
>   char *word;
>   char c;
>  
> - /*
> -  * Get rid of any previous tk_text.
> -  */
> - if (tk_text != NULL) {
> - free(tk_text);
> - tk_text = NULL;
> - }
> + free(tk_text);
> + tk_text = NULL;
>   /*
>* Find the original (uncompleted) word in the command buffer.
>*/
> @@ -939,8 +934,7 @@ init_compl(void)
>   /*
>* Save the original (uncompleted) word
>*/
> - if (tk_original != NULL)
> - free(tk_original);
> + free(tk_original);
>   tk_original = ecalloc(cp-word+1, sizeof (char));
>   (void) strncpy(tk_original, word, cp-word);
>   /*
> @@ -954,12 +948,11 @@ init_compl(void)
>   tk_text = fcomplete(word);
>   } else {
>   char *qword = shell_quote(word+1);
> - if (qword == NULL) {
> + if (qword == NULL)
>   tk_text = fcomplete(word+1);
> - } else {
> + else
>   tk_text = fcomplete(qword);
> - free(qword);
> - }
> + free(qword);
>   }
>   *cp = c;
>  }
> Index: command.c
> ===
> RCS file: /cvs/src/usr.bin/less/command.c,v
> retrieving revision 1.22
> diff -u -p -r1.22 command.c
> --- command.c 7 Nov 2015 18:07:44 -   1.22
> +++ command.c 9 Nov 2015 04:38:23 -
> @@ -198,8 +198,7 @@ exec_mca(void)
>*/
>   while (*cbuf == '+' || *cbuf == ' ')
>   cbuf++;
> - if (every_first_cmd != NULL)
> - free(every_first_cmd);
> + free(every_first_cmd);
>   if (*cbuf == '\0')
>   every_first_cmd = NULL;
>   else
> Index: line.c
> ===
> RCS file: /cvs/src/usr.bin/less/line.c,v
> retrieving revision 1.14
> diff -u -p -r1.14 line.c
> --- line.c6 Nov 2015 15:50:33 -   1.14
> +++ line.c9 Nov 2015 04:38:23 -
> @@ -98,10 +98,8 @@ expand_linebuf(void)
>   char *new_buf = realloc(linebuf, new_size);
>   char *new_attr = realloc(attr, new_size);
>   if (new_buf == NULL || new_attr == NULL) {
> - if (new_attr != NULL)
> - free(new_attr);
> - if (new_buf != NULL)
> - free(new_buf);
> + free(new_attr);
> + free(new_buf);
>   return (1);
>   }
>   linebuf = new_buf;
> Index: option.c
> ===
> RCS file: /cvs/src/usr.bin/less/option.c,v
> retrieving revision 1.12
> diff -u -p -r1.12 option.c
> --- option.c  7 Nov 2015 18:07:44 -   1.12
> +++ option.c  9 Nov 2015 04:38:23 -
> @@ -307,8 +307,7 @@ scan_option(char *s)
>*/
>   if (o->ofunc != NULL)
>   (*o->ofunc)(INIT, str);
> - if (str != NULL)
> - free(str);
> + free(str);
>   }
>  }
>  
> Index: search.c
> ===
> RCS file: /cvs/src/usr.bin/less/search.c,v
> retrieving revision 1.11
> diff -u -p -r1.11 search.c
> --- search.c  6 Nov 2015 15:50:33 -   1.11
> +++ search.c  9 Nov 2015 04:38:23 -
> @@ -97,12 +97,10 @@ set_pattern(struct pattern_info *info, c
>   else if (compile_pattern(pattern, search_type, >compiled) < 0)
>   return (-1);
>   /* Pattern compiled successfully; save the text too. */
> - if (info->text != NULL)
> - free(info->text);
> + free(info->text);
>   info->text = NULL;
> - if (pattern != NULL) {
> + if (pattern != NULL)
>   info->text = estrdup(pattern);
> - }
>   info->search_type = search_type;
>  
>   /*
> @@ -291,7 +289,7 @@ clr_hlist(struct hilite *anchor)
>  
>   for (hl = anchor->hl_first; hl != NULL; hl = nexthl) {
>   nexthl = hl->hl_next;
> - free((void*)hl);
> + free(hl);
>   }
>   anchor->hl_first = NULL;
>   prep_startpos = prep_endpos = -1;
> 

Re: unify xmalloc (was Re: [patch] cvs: retire xfree())

[Nicholas Marriott]

Hmm yes you are right, in that case, go for it.


On Sun, Nov 08, 2015 at 02:00:17PM +0100, Joerg Sonnenberger wrote:
> On Sun, Nov 08, 2015 at 08:36:52AM +0000, Nicholas Marriott wrote:
> > On Sat, Nov 07, 2015 at 08:42:14PM -0500, Ted Unangst wrote:
> > > Tobias Stoeckmann wrote:
> > > > Is this okay for ssh and tmux, which are out to be very portable?
> > > > Nicholas mentioned that malloc is not required to set errno. I've also
> > > > checked the standard and it's just an extension. Although at worst,
> > > > the user sees a wrong error message...
> > > 
> > > Are they portable to not-posix? posix dictates that malloc set errno.
> > 
> > It is optional in SUSv3:
> > 
> > RETURN VALUE
> > 
> >  Upon successful completion with size not equal to 0, malloc() shall
> >  return a pointer to the allocated space. If size is 0, either
> >  a null pointer or a unique pointer that can be successfully
> >  passed to free() shall be returned. Otherwise, it shall return
> >  a null pointer ^[CX] [Option Start] and set errno to indicate
> >  the error. [Option End]
> 
> Not really, that just means that the "and set errno" part is not
> included in ISO C itself. The markup is a bit confusing, but CX is not
> an option in the "pick it or not sense".
> 
> Joerg
> 

Re: [PATCH] add F13-F24 and colour support to console terminfo

[Nicholas Marriott]

If this is different from pccon*, you should fix those or add a variant
rather than adding completely new entries.



On Sun, Nov 08, 2015 at 11:55:23AM +, Tati Chevron wrote:
> The terminfo entries used for X and for the console by a default install,
> are broken for keyboards with function keys beyond F12, and for keyboards
> with five cursor movement keys instead of the more common four.
> 
> In addition, the VGA console supports colour, but the terminfo entry used
> by default does not define the necessary escape sequences.
> 
> Please find attached a patch against -current which defines three new
> terminfo entries to fix these problems.
> 
> --- termtypes.master.1.55.distSun Nov  8 11:40:24 2015
> +++ termtypes.master.1.55.custom  Sun Nov  8 11:43:06 2015
> @@ -291,6 +291,51 @@
> # There are no guarantees anywhere.  Svaha!
> #
> 
> + CUSTOM ENTRIES FOR OPENBSD-5.8
> +#
> +# Three entries to allow the use of F1-F5, F13-F24 and colour on the
> +# OpenBSD VGA framebuffer console, and the use of F13-F24 with the default
> +# X config.
> +#
> +# In addition, the, 'home', key on a PC-122 keyboard is defined as, 'home',
> +# on the console, to match the definition in X:
> +#
> +# On a standard PC keyboard, there are four cursor keys in an inverted, 'T',
> +# whereas on a PC-122 keyboard there are five cursor keys:
> +#
> +#  --- --- Legend
> +#  |^| |^| --
> +#  --- ---
> +#  --- --- --- --- --- --- ^ - UP
> +#  |<| |V| |>| |<| |H| |>| V - DOWN
> +#  --- --- --- --- --- --- < - LEFT
> +#  --- > - RIGHT
> +#  |V| H - HOME
> +#  ---
> +#
> +#  PC Keyboard   PC-122 Keyboard
> +#
> +# The new entries in this file are as follows:
> +# tati - OpenBSD console with F1-F24, 5-key cursor-key block, and colour.
> +# tatimono - As above, but without colour support
> +# tatiX - X11 xterm with F1-F24, (colour and 5-key cursor-key block were
> +# already supported by the existing xterm entry.
> +
> +tatimono|OpenBSD console with F1-F24 without colour support,
> + kf1=\E[11~, kf2=\E[12~, kf3=\E[13~, kf4=\E[14~, kf5=\E[15~,
> + kf15=\E[28~, kf16=\E[29~, kf21=\E[35~, kf22=\E[36~,
> + kf23=\E[37~, kf24=\E[38~, khome=\E[7~, khlp=, krdo=, use=vt220,
> +
> +tati|OpenBSD console with F1-F24 and colour,
> + use=tatimono, use=ecma+color,
> +
> +tatiX|vt220-like entry that fixes behaviour of F13-F24 in X on OpenBSD,
> + kf13=\E[25~, kf14=\E[26~,
> + kf15=\E[28~, kf16=\E[29~, kf17=\E[31~, kf18=\E[32~,
> + kf19=\E[33~, kf20=\E[34~, kf21=\E[42~, kf22=\E[43~,
> + kf23=\E[44~, kf24=\E[45~, khlp=, krdo=, use=xterm,
> +
> +
>  ANSI, UNIX CONSOLE, AND SPECIAL TYPES
> #
> # This section describes terminal classes and brands that are still
> 
> -- 
> Tati Chevron
> Perl and FORTRAN specialist.
> SWABSIT development and migration department.
> http://www.swabsit.com
> 

Re: [PATCH] add F13-F24 and colour support to console terminfo

[Nicholas Marriott]

On Sun, Nov 08, 2015 at 04:12:18PM +, Tati Chevron wrote:
> On Sun, Nov 08, 2015 at 02:09:03PM +0000, Nicholas Marriott wrote:
> >If this is different from pccon*, you should fix those or add a variant
> >rather than adding completely new entries.
> 
> OK.  The attached diff does the following:
> 
> * Add F13-F24 to pccon.
>  These were previously undefined.
> * Change the sgr definition to match the vt220 entry.
>  This makes the, 'intense', colours work, giving us 16 instead of 8.
> * Define the bold sequence.
>  Otherwise, E.G. 'echo foo | less', the ~ in column 0 are displayed as
>  reverse video instead of bold, (as they do in xterm).

I think these are fine.

Can you please send these upstream to Thomas Dickey or
bug-ncur...@gnu.org?

> * Add a new entry, 'pcconX', that adds F13-F24 to the standrd xterm entry

Not this though - pccon* is for the console, not for use in xterm. I
think you might want xterm-vt220?

Although I don't know why our xterm function keys should be different
from stock xterm.

> 
> --- termtypes.master.1.55.distSun Nov  8 15:20:37 2015
> +++ termtypes.master  Sun Nov  8 15:42:04 2015
> @@ -2184,13 +2184,16 @@
>   kf4=\E[14~, kf5=\E[15~, kf6=\E[17~, kf7=\E[18~, kf8=\E[19~,
>   kf9=\E[20~, khome=\E[7~, kich1=\E[2~, knp=\E[6~, kpp=\E[5~,
>   krfr=^R,
> + kf13=\E[25~, kf14=\E[26~, kf15=\E[28~, kf16=\E[29~,
> + kf17=\E[31~, kf18=\E[32~, kf19=\E[33~, kf20=\E[34~,
> + kf21=\E[35~, kf22=\E[36~, kf23=\E[37~, kf24=\E[38~,
> pccon+sgr+acs0|sgr and simple ASCII pseudographics for OpenBSD PC console,
>   acsc=+>\,<-\^.v0#`+a\:f\\h#i#j+k+l+m+n+o~p-q-r-s_t+u+v+w+x|y#z#{*|!}#~o,
>   sgr=\E[0%?%p1%p3%|%t;7%;m, sgr0=\E[m,
> pccon+sgr+acs|sgr and default ASCII pseudographics for OpenBSD PC console,
>   acsc=++\,\,--..00``aaffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
>   enacs=\E)0$<5>, rmacs=\E(B$<5>,
> - sgr=\E[0%?%p1%p3%|%t;7%;m%?%p9%t\E(0%e\E(B%;$<5>,
> + 
> sgr=\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p4%t;5%;%?%p1%p3%|%t;7%;m%?%p9%t\E(0%e\E(B%;$<2>,
>   sgr0=\E[m\E(B$<5>, smacs=\E(0$<5>,
> pccon+colors|ANSI colors for OpenBSD PC console,
>   bce,
> @@ -2199,7 +2202,7 @@
> pccon+base|base capabilities for OpenBSD PC console,
>   am, km, mc5i, msgr, npc, nxon, xenl, xon,
>   cols#80, it#8, lines#24,
> - bel=^G, clear=\E[H\E[J, cr=^M, cub1=^H, cud1=^J, cuf1=\E[C,
> + bel=^G, bold=\E[1m, clear=\E[H\E[J, cr=^M, cub1=^H, cud1=^J, cuf1=\E[C,
>   cup=\E[%i%p1%d;%p2%dH, cuu1=\E[A, dch=\E[%p1%dP,
>   dch1=\E[P, dl1=\E[M, ech=\E[%p1%dX, ed=\E[J, el=\E[K,
>   el1=\E[1K, home=\E[H, ht=^I, hts=\EH, ich=\E[%p1%d@,
> @@ -2214,6 +2217,11 @@
>   use=pccon+base, use=pccon+sgr+acs, use=pccon+keys,
> pccon|OpenBSD PC console,
>   use=pccon-m, use=pccon+colors,
> +pcconX|xterm-like entry that fixes behaviour of F13-F24 in X on OpenBSD,
> + kf13=\E[25~, kf14=\E[26~,
> + kf15=\E[28~, kf16=\E[29~, kf17=\E[31~, kf18=\E[32~,
> + kf19=\E[33~, kf20=\E[34~, kf21=\E[42~, kf22=\E[43~,
> + kf23=\E[44~, kf24=\E[45~, khlp=, krdo=, use=xterm,
> 
>  NetBSD consoles
> #
> 
> -- 
> Tati Chevron
> Perl and FORTRAN specialist.
> SWABSIT development and migration department.
> http://www.swabsit.com

Re: [PATCH] add F13-F24 and colour support to console terminfo

[Nicholas Marriott]

Hi

I think it should be Shift-F1 etc, but since this is a general problem
with the xterm entry not just on OpenBSD, I suggest you discuss it with
upstream. The terminfo database is maintained as part of ncurses.


On Sun, Nov 08, 2015 at 07:21:02PM +, Tati Chevron wrote:
> On Sun, Nov 08, 2015 at 04:50:21PM +0000, Nicholas Marriott wrote:
> >Can you please send these upstream to Thomas Dickey or
> >bug-ncur...@gnu.org?
> 
> Done.
> 
> >>* Add a new entry, 'pcconX', that adds F13-F24 to the standrd xterm entry
> >
> >Not this though - pccon* is for the console, not for use in xterm. I
> >think you might want xterm-vt220?
> 
> Hu, no, xterm-vt220 doesn't define F21-F24, and also has an incorrect 
> definition for the, 'home', key.
> 
> I can make a diff for this, obviously, should I just add it as a variation?
> 
> >Although I don't know why our xterm function keys should be different
> >from stock xterm.
> 
> Since F13-F24 don't exist on standard PC keyboards, traditionally terminal 
> emulators have mapped shift-F1 to F13, shift-F2 to F14, etc.
> 
> In recent times, some 122-key terminal keyboards, which are designed to be 
> used with terminal emulators like that, actually send sequences such as 
> Shift-F1 when the physical F13 key is pressed, instead of the, 'real', 
> sequence.
> 
> If you look in xterm+pcf2, which is the fragment included in the default 
> xterm terminfo entry that we use, you'll see that F13-F24 are indeed defined 
> as Shift-F1 - Shift-F12.
> 
> Unfortunately, even this doesn't work correctly for us, because F1-F4 don't 
> follow the normal sequence for historical reasons.
> 
> So, we should do something to make sure that the default terminal emulation 
> used for xterm on a default install, has a sensible configuration for 
> F13-F24.  Either support the, 'real', sequences from a 122-key keyboard, or 
> support the shifted F1-F12 keys as F13-F24, or preferably, both.
> 
> Shall I make a diff to fix this?
> 
> -- 
> Tati Chevron
> Perl and FORTRAN specialist.
> SWABSIT development and migration department.
> http://www.swabsit.com

Re: unify xmalloc (was Re: [patch] cvs: retire xfree())

[Nicholas Marriott]

This is fine with me, I think it was better without errno, but using it
can't do any harm. It is an extension TO set it, not to not set it, but
I am pretty sure it only happens on platforms I don't care about :-).

I suggest you check with djm or dtucker for ssh in case they do care, or
there are any other problem for portable.



On Sun, Nov 08, 2015 at 01:39:32AM +0100, Tobias Stoeckmann wrote:
> Here's an updated diff:
> 
> - use "overflow" error message for snprintf and friends
> - use err instead of errx for out of memory conditions
> - if fatal() doesn't print error string, use ": %s", strerror(errno)
> 
> Is this okay for ssh and tmux, which are out to be very portable?
> Nicholas mentioned that malloc is not required to set errno. I've also
> checked the standard and it's just an extension. Although at worst,
> the user sees a wrong error message...
> 
> Index: usr.bin/cvs/xmalloc.c
> ===
> RCS file: /cvs/src/usr.bin/cvs/xmalloc.c,v
> retrieving revision 1.12
> diff -u -p -u -p -r1.12 xmalloc.c
> --- usr.bin/cvs/xmalloc.c 5 Nov 2015 09:48:21 -   1.12
> +++ usr.bin/cvs/xmalloc.c 8 Nov 2015 00:27:13 -
> @@ -13,6 +13,8 @@
>   * called by a name other than "ssh" or "Secure Shell".
>   */
>  
> +#include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -30,7 +32,8 @@ xmalloc(size_t size)
>   fatal("xmalloc: zero size");
>   ptr = malloc(size);
>   if (ptr == NULL)
> - fatal("xmalloc: out of memory (allocating %lu bytes)", (u_long) 
> size);
> + fatal("xmalloc: allocating %zu bytes: %s",
> + size, strerror(errno));
>   return ptr;
>  }
>  
> @@ -41,12 +44,10 @@ xcalloc(size_t nmemb, size_t size)
>  
>   if (size == 0 || nmemb == 0)
>   fatal("xcalloc: zero size");
> - if (SIZE_MAX / nmemb < size)
> - fatal("xcalloc: nmemb * size > SIZE_MAX");
>   ptr = calloc(nmemb, size);
>   if (ptr == NULL)
> - fatal("xcalloc: out of memory (allocating %lu bytes)",
> - (u_long)(size * nmemb));
> + fatal("xcalloc: allocating %zu * %zu bytes: %s",
> + nmemb, size, strerror(errno));
>   return ptr;
>  }
>  
> @@ -54,28 +55,23 @@ void *
>  xreallocarray(void *ptr, size_t nmemb, size_t size)
>  {
>   void *new_ptr;
> - size_t new_size = nmemb * size;
>  
> - if (new_size == 0)
> - fatal("xrealloc: zero size");
> - if (SIZE_MAX / nmemb < size)
> - fatal("xrealloc: nmemb * size > SIZE_MAX");
> - new_ptr = realloc(ptr, new_size);
> + if (nmemb == 0 || size == 0)
> + fatal("xreallocarray: zero size");
> + new_ptr = reallocarray(ptr, nmemb, size);
>   if (new_ptr == NULL)
> - fatal("xrealloc: out of memory (new_size %lu bytes)",
> - (u_long) new_size);
> + fatal("xreallocarray: allocating %zu * %zu bytes: %s",
> + nmemb, size, strerror(errno));
>   return new_ptr;
>  }
>  
>  char *
>  xstrdup(const char *str)
>  {
> - size_t len;
>   char *cp;
>  
> - len = strlen(str) + 1;
> - cp = xmalloc(len);
> - strlcpy(cp, str, len);
> + if ((cp = strdup(str)) == NULL)
> + fatal("xstrdup: %s", strerror(errno));
>   return cp;
>  }
>  
> @@ -90,23 +86,26 @@ xasprintf(char **ret, const char *fmt, .
>   va_end(ap);
>  
>   if (i < 0 || *ret == NULL)
> - fatal("xasprintf: could not allocate memory");
> + fatal("xasprintf: %s", strerror(errno));
>  
> - return (i);
> + return i;
>  }
>  
>  int
> -xsnprintf(char *str, size_t size, const char *fmt, ...)
> +xsnprintf(char *str, size_t len, const char *fmt, ...)
>  {
>   va_list ap;
>   int i;
>  
> + if (len > INT_MAX)
> + fatal("xsnprintf: len > INT_MAX");
> +
>   va_start(ap, fmt);
> - i = vsnprintf(str, size, fmt, ap);
> + i = vsnprintf(str, len, fmt, ap);
>   va_end(ap);
>  
> - if (i == -1 || i >= (int)size)
> + if (i < 0 || i >= (int)len)
>   fatal("xsnprintf: overflow");
>  
> - return (i);
> + return i;
>  }
> Index: usr.bin/diff/xmalloc.c
> ===
> RCS file: /cvs/src/usr.bin/diff/xmalloc.c,v
> retrieving revision 1.8
> diff -u -p -u -p -r1.8 xmalloc.c
> --- usr.bin/diff/xmalloc.c25 Sep 2015 16:16:26 -  1.8
> +++ usr.bin/diff/xmalloc.c8 Nov 2015 00:27:13 -
> @@ -27,9 +27,11 @@ xmalloc(size_t size)
>  {
>   void *ptr;
>  
> + if (size == 0)
> + errx(2, "xmalloc: zero size");
>   ptr = malloc(size);
>   if (ptr == NULL)
> - err(2, "xmalloc %zu", size);
> + err(2, "xmalloc: allocating %zu bytes", size);
>   return ptr;
>  }
>  
> @@ -40,8 +42,7 @@ xcalloc(size_t nmemb, size_t size)
>  
>   ptr = calloc(nmemb, size);
>   if 

Re: unify xmalloc (was Re: [patch] cvs: retire xfree())

[Nicholas Marriott]

On Sat, Nov 07, 2015 at 08:42:14PM -0500, Ted Unangst wrote:
> Tobias Stoeckmann wrote:
> > Is this okay for ssh and tmux, which are out to be very portable?
> > Nicholas mentioned that malloc is not required to set errno. I've also
> > checked the standard and it's just an extension. Although at worst,
> > the user sees a wrong error message...
> 
> Are they portable to not-posix? posix dictates that malloc set errno.

It is optional in SUSv3:

RETURN VALUE

 Upon successful completion with size not equal to 0, malloc() shall
 return a pointer to the allocated space. If size is 0, either
 a null pointer or a unique pointer that can be successfully
 passed to free() shall be returned. Otherwise, it shall return
 a null pointer ^[CX] [Option Start] and set errno to indicate
 the error. [Option End]

Re: unify xmalloc (was Re: [patch] cvs: retire xfree())

[Nicholas Marriott]

Hi

On Sat, Nov 07, 2015 at 04:39:09PM -0500, Michael McConville wrote:
> Nicholas Marriott wrote:
> > Looks good, ok nicm
> 
> Reviewing now, generally looks good.
> 
> A few things:
> 
> I don't understand the motive for all the err() -> errx() and fatal() ->
> fatalx() changes. If I came across these, I probably would have

malloc and friends are not guaranteed to set errno (they do on OpenBSD
but it is not required).

> suggested the reverse. err(1, "xstrdup") is a lot cleaner than a long
> custom error message, IMO. I don't know how much value is in showing the
> size of the failed allocation, either - thoughts on that? I'm fine with
> a little less uniformity for simplicity's sake.

I think it is better if they are all the same, and showing the size is
useful (for example, if the size is crazy it could be an overflow bug
rather than out of memory).

> 
> Also, I'm seeing a couple "could not allocate memory" messages added to
> *snprintf() functions. They write to a supplied buffer, no?

Yes you are right, it should be a different message.

> 
> We should probably pass the SSH changes by open...@openssh.com.
> 
> This is valuable work - thanks.
> 
> > On Thu, Nov 05, 2015 at 05:35:22PM +0100, Tobias Stoeckmann wrote:
> > > On Thu, Nov 05, 2015 at 03:57:26PM +, Nicholas Marriott wrote:
> > > > I like this a lot.
> > > > 
> > > > There are some trivial differences in the various xmalloc.h as well, and
> > > > I think you could make the style consistent within the files (eg "return
> > > > i" in xasprintf and xsnprintf).
> > > 
> > > Oh yes, forgot to check the header files. Updated diff below, including
> > > the return (i) vs. return i change.
> > > 
> > > Index: usr.bin/cvs/xmalloc.c
> > > ===
> > > RCS file: /cvs/src/usr.bin/cvs/xmalloc.c,v
> > > retrieving revision 1.12
> > > diff -u -p -u -p -r1.12 xmalloc.c
> > > --- usr.bin/cvs/xmalloc.c 5 Nov 2015 09:48:21 -   1.12
> > > +++ usr.bin/cvs/xmalloc.c 5 Nov 2015 16:32:21 -
> > > @@ -13,6 +13,7 @@
> > >   * called by a name other than "ssh" or "Secure Shell".
> > >   */
> > >  
> > > +#include 
> > >  #include 
> > >  #include 
> > >  #include 
> > > @@ -30,7 +31,7 @@ xmalloc(size_t size)
> > >   fatal("xmalloc: zero size");
> > >   ptr = malloc(size);
> > >   if (ptr == NULL)
> > > - fatal("xmalloc: out of memory (allocating %lu bytes)", (u_long) 
> > > size);
> > > + fatal("xmalloc: out of memory (allocating %zu bytes)", size);
> > >   return ptr;
> > >  }
> > >  
> > > @@ -41,12 +42,10 @@ xcalloc(size_t nmemb, size_t size)
> > >  
> > >   if (size == 0 || nmemb == 0)
> > >   fatal("xcalloc: zero size");
> > > - if (SIZE_MAX / nmemb < size)
> > > - fatal("xcalloc: nmemb * size > SIZE_MAX");
> > >   ptr = calloc(nmemb, size);
> > >   if (ptr == NULL)
> > > - fatal("xcalloc: out of memory (allocating %lu bytes)",
> > > - (u_long)(size * nmemb));
> > > + fatal("xcalloc: out of memory (allocating %zu * %zu bytes)",
> > > + nmemb, size);
> > >   return ptr;
> > >  }
> > >  
> > > @@ -54,28 +53,23 @@ void *
> > >  xreallocarray(void *ptr, size_t nmemb, size_t size)
> > >  {
> > >   void *new_ptr;
> > > - size_t new_size = nmemb * size;
> > >  
> > > - if (new_size == 0)
> > > - fatal("xrealloc: zero size");
> > > - if (SIZE_MAX / nmemb < size)
> > > - fatal("xrealloc: nmemb * size > SIZE_MAX");
> > > - new_ptr = realloc(ptr, new_size);
> > > + if (nmemb == 0 || size == 0)
> > > + fatal("xreallocarray: zero size");
> > > + new_ptr = reallocarray(ptr, nmemb, size);
> > >   if (new_ptr == NULL)
> > > - fatal("xrealloc: out of memory (new_size %lu bytes)",
> > > - (u_long) new_size);
> > > + fatal("xreallocarray: out of memory "
> > > + "(allocating %zu * %zu bytes)", nmemb, size);
> > >   return new_ptr;
> > >  }
> > >  
> > >  char *
> > >  xstrdup(const char *str)
> > >  {
> > >

Re: more utf8 less ebcdic

[Nicholas Marriott]

ok nicm



On Fri, Nov 06, 2015 at 10:40:19PM -0500, Ted Unangst wrote:
> Ted Unangst wrote:
> > Oops, that was the big bomb diff. We want to keep the nl_langinfo and some
> > charset support. Just remove the environment variable and pare down some of
> > the stranger charsets.
> 
> here's better working second half of the big bomb. we can init the charset 
> table
> with the ctype functions, and the utf-8 code actually does something quite
> different (with various amounts of correctness).
> 
> Index: charset.c
> ===
> RCS file: /cvs/src/usr.bin/less/charset.c,v
> retrieving revision 1.15
> diff -u -p -r1.15 charset.c
> --- charset.c 7 Nov 2015 03:30:52 -   1.15
> +++ charset.c 7 Nov 2015 03:36:07 -
> @@ -23,37 +23,6 @@
>  
>  int utf_mode = 0;
>  
> -/*
> - * Predefined character sets,
> - * selected by the LESSCHARSET environment variable.
> - */
> -struct charset {
> - char *name;
> - int *p_flag;
> - char *desc;
> -} charsets[] = {
> - /* BEGIN CSTYLED */
> - { "ascii",  NULL,   "8bcccbcc18b95.b" },
> - { "utf-8",  _mode,   "8bcccbcc18b95.b126.bb" },
> - { NULL, NULL, NULL }
> - /* END CSTYLED */
> -};
> -
> -/*
> - * Support "locale charmap"/nl_langinfo(CODESET) values, as well as others.
> - */
> -struct cs_alias {
> - char *name;
> - char *oname;
> -} cs_aliases[] = {
> - { "UTF-8",  "utf-8" },
> - { "ANSI_X3.4-1968", "ascii" },
> - { "US-ASCII",   "ascii" },
> - { "646","ascii" },
> - { "C",  "ascii" },
> - { NULL, NULL }
> -};
> -
>  #define  IS_BINARY_CHAR  01
>  #define  IS_CONTROL_CHAR 02
>  
> @@ -62,109 +31,6 @@ static const char *binfmt = NULL;
>  static const char *utfbinfmt = NULL;
>  int binattr = AT_STANDOUT;
>  
> -
> -/*
> - * Define a charset, given a description string.
> - * The string consists of 256 letters,
> - * one for each character in the charset.
> - * If the string is shorter than 256 letters, missing letters
> - * are taken to be identical to the last one.
> - * A decimal number followed by a letter is taken to be a
> - * repetition of the letter.
> - *
> - * Each letter is one of:
> - *   . normal character
> - *   b binary character
> - *   c control character
> - */
> -static void
> -ichardef(char *s)
> -{
> - char *cp;
> - int n;
> - char v;
> -
> - n = 0;
> - v = 0;
> - cp = chardef;
> - while (*s != '\0') {
> - switch (*s++) {
> - case '.':
> - v = 0;
> - break;
> - case 'c':
> - v = IS_CONTROL_CHAR;
> - break;
> - case 'b':
> - v = IS_BINARY_CHAR|IS_CONTROL_CHAR;
> - break;
> -
> - case '0': case '1': case '2': case '3': case '4':
> - case '5': case '6': case '7': case '8': case '9':
> - n = (10 * n) + (s[-1] - '0');
> - continue;
> -
> - default:
> - error("invalid chardef", NULL_PARG);
> - quit(QUIT_ERROR);
> - /*NOTREACHED*/
> - }
> -
> - do {
> - if (cp >= chardef + sizeof (chardef)) {
> - error("chardef longer than 256", NULL_PARG);
> - quit(QUIT_ERROR);
> - /*NOTREACHED*/
> - }
> - *cp++ = v;
> - } while (--n > 0);
> - n = 0;
> - }
> -
> - while (cp < chardef + sizeof (chardef))
> - *cp++ = v;
> -}
> -
> -/*
> - * Define a charset, given a charset name.
> - * The valid charset names are listed in the "charsets" array.
> - */
> -static int
> -icharset(char *name, int no_error)
> -{
> - struct charset *p;
> - struct cs_alias *a;
> -
> - if (name == NULL || *name == '\0')
> - return (0);
> -
> - /* First see if the name is an alias. */
> - for (a = cs_aliases;  a->name != NULL;  a++) {
> - if (strcmp(name, a->name) == 0) {
> - name = a->oname;
> - break;
> - }
> - }
> -
> - for (p = charsets;  p->name != NULL;  p++) {
> - if (strcmp(name, p->name) == 0) {
> - ichardef(p->desc);
> - if (p->p_flag != NULL)
> - *(p->p_flag) = 1;
> - return (1);
> - }
> - }
> -
> - if (!no_error) {
> - error("invalid charset name", NULL_PARG);
> - quit(QUIT_ERROR);
> - }
> - return (0);
> -}
> -
> -/*
> - * Define a charset, given a locale name.
> - */
>  static void
>  ilocale(void)
>  {
> @@ -287,17 +153,10 @@ set_charset(void)
>  {
>   char *s;
>  
> - /*
> - 

Re: save less

[Nicholas Marriott]

Hmm I thought this was deliberate but looking at where estrdup is used
it is obviously not.  ok nicm



On Fri, Nov 06, 2015 at 11:07:26PM -0500, Ted Unangst wrote:
> less has a peculiar estrdup function. unlike ecalloc etc., it only prints an
> error but doesn't quit. But the callers don't seem to check for null. And in
> many places they call a function called save() instead.
> 
> It is clearer to make estrdup() quit and use it everywhere.
> 
> 
> Index: cmdbuf.c
> ===
> RCS file: /cvs/src/usr.bin/less/cmdbuf.c,v
> retrieving revision 1.9
> diff -u -p -r1.9 cmdbuf.c
> --- cmdbuf.c  6 Nov 2015 15:50:33 -   1.9
> +++ cmdbuf.c  7 Nov 2015 04:04:50 -
> @@ -688,7 +688,7 @@ cmd_addhist(struct mlist *mlist, const c
>* Save the command and put it at the end of the history list.
>*/
>   ml = ecalloc(1, sizeof (struct mlist));
> - ml->string = save(cmd);
> + ml->string = estrdup(cmd);
>   ml->next = mlist;
>   ml->prev = mlist->prev;
>   mlist->prev->next = ml;
> @@ -1207,7 +1207,7 @@ histfile_name(void)
>   if (strcmp(name, "-") == 0 || strcmp(name, "/dev/null") == 0)
>   /* $LESSHISTFILE == "-" means don't use history file */
>   return (NULL);
> - return (save(name));
> + return (estrdup(name));
>   }
>  
>   /* Otherwise, file is in $HOME if enabled. */
> Index: command.c
> ===
> RCS file: /cvs/src/usr.bin/less/command.c,v
> retrieving revision 1.21
> diff -u -p -r1.21 command.c
> --- command.c 6 Nov 2015 15:58:01 -   1.21
> +++ command.c 7 Nov 2015 04:04:50 -
> @@ -203,7 +203,7 @@ exec_mca(void)
>   if (*cbuf == '\0')
>   every_first_cmd = NULL;
>   else
> - every_first_cmd = save(cbuf);
> + every_first_cmd = estrdup(cbuf);
>   break;
>   case A_OPT_TOGGLE:
>   toggle_option(curropt, opt_lower, cbuf, optflag);
> Index: decode.c
> ===
> RCS file: /cvs/src/usr.bin/less/decode.c,v
> retrieving revision 1.11
> diff -u -p -r1.11 decode.c
> --- decode.c  6 Nov 2015 15:58:01 -   1.11
> +++ decode.c  7 Nov 2015 04:04:50 -
> @@ -680,9 +680,9 @@ add_hometable(char *envname, char *def_f
>   PARG parg;
>  
>   if (envname != NULL && (filename = lgetenv(envname)) != NULL)
> - filename = save(filename);
> + filename = estrdup(filename);
>   else if (sysvar)
> - filename = save(def_filename);
> + filename = estrdup(def_filename);
>   else
>   filename = homefile(def_filename);
>   if (filename == NULL)
> Index: edit.c
> ===
> RCS file: /cvs/src/usr.bin/less/edit.c,v
> retrieving revision 1.13
> diff -u -p -r1.13 edit.c
> --- edit.c6 Nov 2015 15:50:33 -   1.13
> +++ edit.c7 Nov 2015 04:04:50 -
> @@ -232,7 +232,7 @@ edit_ifile(IFILE ifile)
>   return (0);
>   }
>  
> - filename = save(get_filename(ifile));
> + filename = estrdup(get_filename(ifile));
>   /*
>* See if LESSOPEN specifies an "alternate" file to open.
>*/
> Index: filename.c
> ===
> RCS file: /cvs/src/usr.bin/less/filename.c,v
> retrieving revision 1.18
> diff -u -p -r1.18 filename.c
> --- filename.c6 Nov 2015 15:50:33 -   1.18
> +++ filename.c7 Nov 2015 04:04:50 -
> @@ -643,14 +643,14 @@ open_altfile(char *filename, int *pf, vo
>   if (returnfd > 1 && status == 0) {
>   *pfd = NULL;
>   *pf = -1;
> - return (save(FAKE_EMPTYFILE));
> + return (estrdup(FAKE_EMPTYFILE));
>   }
>   return (NULL);
>   }
>   ch_ungetchar(c);
>   *pfd = (void *) fd;
>   *pf = f;
> - return (save("-"));
> + return (estrdup("-"));
>   }
>   cmd = readfd(fd);
>   pclose(fd);
> Index: funcs.h
> ===
> RCS file: /cvs/src/usr.bin/less/funcs.h,v
> retrieving revision 1.10
> diff -u -p -r1.10 funcs.h
> --- funcs.h   6 Nov 2015 15:09:07 -   1.10
> +++ funcs.h   7 Nov 2015 04:04:50 -
> @@ -9,7 +9,6 @@
>  struct mlist;
>  struct loption;
>  
> -extern char *save(const char *);
>  extern void *ecalloc(int, unsigned int);
>  /*PRINTFLIKE1*/
>  extern char *easprintf(const char *, ...);
> Index: ifile.c
> 

Re: save less

[Nicholas Marriott]

On Sat, Nov 07, 2015 at 12:12:55AM -0500, Michael McConville wrote:
> Ted Unangst wrote:
> > less has a peculiar estrdup function. unlike ecalloc etc., it only
> > prints an error but doesn't quit. But the callers don't seem to check
> > for null. And in many places they call a function called save()
> > instead.
> > 
> > It is clearer to make estrdup() quit and use it everywhere.
> 
> ok mmcc@
> 
> That said, error()'s name very strongly implies that it quits, so we
> should look into fixing that too.

I don't think it implies anything of the sort, an error is still an
error if it doesn't kill the program.

Re: [patch] cvs: retire xfree()

[Nicholas Marriott]

Applied, thanks.

I don't know why cvs and rcs xmalloc.c has ended up so different.


On Thu, Nov 05, 2015 at 11:50:51AM +0800, Michael W. Bombardieri wrote:
> Hi tech@,
> 
> Function xfree() was previously removed from rcs, so drop it from
> opencvs too...
> 
> http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/rcs/xmalloc.c?f=h#rev1.9
> 
> Footnote:
> I noticed that rcsnum_free() is just free() so maybe that could be
> removed also (not included in this patch).
> 
> - Michael
> 
> 
> Index: add.c
> ===
> RCS file: /cvs/src/usr.bin/cvs/add.c,v
> retrieving revision 1.111
> diff -u -p -u -r1.111 add.c
> --- add.c 16 Jan 2015 06:40:06 -  1.111
> +++ add.c 5 Nov 2015 02:49:20 -
> @@ -20,6 +20,7 @@
>  
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  
> @@ -146,7 +147,7 @@ cvs_add_entry(struct cvs_file *cf)
>   entlist = cvs_ent_open(cf->file_wd);
>   cvs_ent_add(entlist, entry);
>  
> - xfree(entry);
> + free(entry);
>   } else {
>   add_entry(cf);
>   }
> @@ -252,7 +253,7 @@ cvs_add_tobranch(struct cvs_file *cf, ch
>   (void)xsnprintf(attic, PATH_MAX, "%s/%s/%s%s", repo,
>   CVS_PATH_ATTIC, cf->file_name, RCS_FILE_EXT);
>  
> - xfree(cf->file_rpath);
> + free(cf->file_rpath);
>   cf->file_rpath = xstrdup(attic);
>  
>   cf->repo_fd = open(cf->file_rpath, O_CREAT|O_RDONLY);
> @@ -277,7 +278,7 @@ cvs_add_tobranch(struct cvs_file *cf, ch
>   if (rcs_rev_add(cf->file_rcs, RCS_HEAD_REV, msg, -1, NULL) == -1)
>   fatal("cvs_add_tobranch: failed to create first branch "
>   "revision");
> - xfree(msg);
> + free(msg);
>  
>   if (rcs_findrev(cf->file_rcs, cf->file_rcs->rf_head) == NULL)
>   fatal("cvs_add_tobranch: cannot find newly added revision");
> @@ -359,7 +360,7 @@ add_directory(struct cvs_file *cf)
>  
>   entlist = cvs_ent_open(cf->file_wd);
>   cvs_ent_add(entlist, p);
> - xfree(p);
> + free(p);
>   }
>   }
>  
> @@ -381,10 +382,8 @@ add_directory(struct cvs_file *cf)
>   }
>   cvs_printf("%s\n", msg);
>  
> - if (tag != NULL)
> - xfree(tag);
> - if (date != NULL)
> - xfree(date);
> + free(tag);
> + free(date);
>  
>   cvs_get_repository_name(cf->file_path, repo, PATH_MAX);
>   line_list = cvs_trigger_getlines(CVS_PATH_LOGINFO, repo);
> @@ -400,8 +399,7 @@ add_directory(struct cvs_file *cf)
>  
>   cvs_trigger_freeinfo(_info);
>   cvs_trigger_freelist(line_list);
> - if (loginfo != NULL)
> - xfree(loginfo);
> + free(loginfo);
>   }
>   }
>  
> @@ -564,5 +562,5 @@ add_entry(struct cvs_file *cf)
>   entlist = cvs_ent_open(cf->file_wd);
>   cvs_ent_add(entlist, entry);
>   }
> - xfree(entry);
> + free(entry);
>  }
> Index: admin.c
> ===
> RCS file: /cvs/src/usr.bin/cvs/admin.c,v
> retrieving revision 1.65
> diff -u -p -u -r1.65 admin.c
> --- admin.c   16 Jan 2015 06:40:06 -  1.65
> +++ admin.c   5 Nov 2015 02:49:20 -
> @@ -23,6 +23,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  
> @@ -304,8 +305,8 @@ cvs_admin_local(struct cvs_file *cf)
>   while (!TAILQ_EMPTY(&(cf->file_rcs->rf_access))) {
>   rap = TAILQ_FIRST(&(cf->file_rcs->rf_access));
>   TAILQ_REMOVE(&(cf->file_rcs->rf_access), rap, ra_list);
> - xfree(rap->ra_name);
> - xfree(rap);
> + free(rap->ra_name);
> + free(rap);
>   }
>   /* no synced anymore */
>   cf->file_rcs->rf_flags &= ~RCS_SYNCED;
> Index: annotate.c
> ===
> RCS file: /cvs/src/usr.bin/cvs/annotate.c,v
> retrieving revision 1.64
> diff -u -p -u -r1.64 annotate.c
> --- annotate.c16 Jan 2015 06:40:06 -  1.64
> +++ annotate.c5 Nov 2015 02:49:20 -
> @@ -235,7 +235,7 @@ cvs_annotate_local(struct cvs_file *cf)
>   p[line->l_len] = '\0';
>  
>   if (line->l_needsfree)
> - xfree(line->l_line);
> + free(line->l_line);
>   line->l_line = p;
>   line->l_len++;
>   line->l_needsfree = 1;
> @@ -244,9 +244,9 @@ cvs_annotate_local(struct cvs_file *cf)
>   line->l_delta->rd_author, date, line->l_line);
>  
>   if