Re: cannot disable output processing in pseudo-terminal (between openpty(3) parent and child)

2023-07-31 Thread pouya+lists . netbsd
Valery Ushakov writes:
>
> The fact that TERM affects this is a sign that you likely have
> something like a shell with line-editing interfering as RVP suggested
> in another message.

This indeed seems to have been the case.  Issuing

stty raw ; cat file

as a single command instead of two in the child shell (standard-issue
NetBSD /bin/sh) made a difference.


Re: cannot disable output processing in pseudo-terminal (between openpty(3) parent and child)

2023-07-31 Thread pouya+lists . netbsd
RVP writes:
> On Sun, 30 Jul 2023, pouya+lists.net...@nohup.io wrote:
> You'll have to turn off the NL -> CRNL output processing the kernel
> does on your behalf (after all, this is still a "terminal" we're
> talking about and not a simple pipe). Don't set the ONLCR bit in
> the output-modes of the child pty.

Thank you.  I thought I had done that, but it seemed not to take
effect.  However as you pointed out below it looks like the shell
(NetBSD's default /bin/sh fwiw) was resetting this.

>
> Note that the shell you're running on the child-side also matters.
> If that does any fancy line-editing using libedit or readline, say,
> then that could change the ONLCR bit every time the shell has to
> read input.


Re: zfs pool behavior - is it ever freed?

2023-07-31 Thread Greg Troxel
I took your patch and have been adding comments to help me understand
things, as well as debug logging.  I also switched how it works, to have
an ifdef for netbsd approch vs others, to make it less confusing -- but
it amounts to the same thing.  (I understand your intent was to touch as
few lines as possible and agree that your approach is also sensible.)

I conclude:

  the default behavior is to set the ARC size to all memory except 1
  GB

  Even on a high-memory machine, without memory pressure mechanisms, the
  current code is dangerous -- even if in practice it is usually ok.

  If the ARC size is more moderate, things are ok

  The ARC tends to fill with metadata, and I believe this is because the
  vnode cache has refs to the ARC so it is not evictable

  We don't have any real evidence that huge ARC is much better than
  biggish ARC on real workloads.

I attach my patch, which I am not really proposing for committing this
minute.  But I suggest anyone trying to run zfs on 8G and below try it.
I think it would be interesting to hear how it affects systems with lots
of memory.

On a system with 6000 MB (yes, that's not a power of 2 - xen config), I
end up with 750 MB of arc_c.   There is often 2+ GB of pool usage, but
that's of course also non-zfs.  The system is stable, so far.  (I have
pkgsrc, distfiles, binary packages in zfs; the OS is in UFS2.)

ARCI 002 arc_abs_min 16777216
ARCI 002 arc_c_min 196608000
ARCI 005 arc_c_max 786432000
ARCI 010 arc_c_min 196608000
ARCI 010 arc_p 393216000
ARCI 010 arc_c 786432000
ARCI 010 arc_c_max 786432000
ARCI 011 arc_meta_limit 196608000



DIFF.arc
Description: Binary data


Re: cannot disable output processing in pseudo-terminal (between openpty(3) parent and child)

2023-07-31 Thread Valery Ushakov
On Mon, Jul 31, 2023 at 06:41:13 +, pouya+lists.net...@nohup.io wrote:

> One difference I see with your code is that you don't seem to change
> the controlling terminal in the child with setsid(2) and a TIOCSCTTY
> ioctl(2).

I only need to feed some input to the terminal emulation code, so the
scaffolding is minimal :).  The child programs are usually just cat,
printf, tputs.


> It made no difference.  In both cases running stty(1) with operands
> "raw" or "-opost" are ineffective and OPOST remains set after
> running them.
[...]
> I'm still puzzled as to why I can't disable OPOST programmatically
> or using stty.
> 
> FWIW I have set TERM=dumb in my terminal and that also seems to
> affect input and output processing.

The fact that TERM affects this is a sign that you likely have
something like a shell with line-editing interfering as RVP suggested
in another message.


-uwe


Re: cannot disable output processing in pseudo-terminal (between openpty(3) parent and child)

2023-07-31 Thread pouya+lists . netbsd
Valery Ushakov writes:
> FWIW, I have a simple program I've been using to debug kernel vt100
> emulator, and:
>
> $ ./wsemul -- printf '\n'
> $ hexdump -C wsemul.raw 
>   0d 0a |..|
> 0002
> $ ./wsemul -- sh -c 'stty raw; printf "\n";'
> $ hexdump -C wsemul.raw 
>   0a|.|
> 0001
>
> where wsemul.raw is the (unfortunately named) dump of everything
> received from the slave on the master side.
>
> The program doesn't do anything fancy with the pty.
>
> https://hg.sr.ht/~nbuwe/wsemul/browse/run.c?rev=tip

Thanks for sharing this.  I was using the BSD-style openpty(3) call
before but now replicated your use of POSIX posix_openpt(3) etc.
It made no difference.  In both cases running stty(1) with operands
"raw" or "-opost" are ineffective and OPOST remains set after
running them.

One difference I see with your code is that you don't seem to change
the controlling terminal in the child with setsid(2) and a TIOCSCTTY
ioctl(2).

I'm still puzzled as to why I can't disable OPOST programmatically
or using stty.

FWIW I have set TERM=dumb in my terminal and that also seems to
affect input and output processing.

Pouya


Re: cannot disable output processing in pseudo-terminal (between openpty(3) parent and child)

2023-07-31 Thread RVP

On Sun, 30 Jul 2023, pouya+lists.net...@nohup.io wrote:


One of my use cases requires the child process to emit a binary
stream to be read by the parent.  The parent is notified first via
a specific escape sequence.  This almost works, except I can't
figure out how to disable output processing on this stream, and in
particular the parent receives extra CR characters for every LF
(NL) the child sends.



You'll have to turn off the NL -> CRNL output processing the kernel
does on your behalf (after all, this is still a "terminal" we're
talking about and not a simple pipe). Don't set the ONLCR bit in
the output-modes of the child pty.

Note that the shell you're running on the child-side also matters.
If that does any fancy line-editing using libedit or readline, say,
then that could change the ONLCR bit every time the shell has to
read input.


P.S. This other one's not as important but I'm also confused by
why disabling ECHO on the parent stdin also disables it for the
pseudo-terminal.



It shouldn't--unless you've unset that bit on the master then copied
those settings over onto the child pty FD.

-RVP