Re: OpenBSD `ftp` port for MSVC (Windows)

[Ingo Schwarze]

Hi Samuel,

first, note that this is off-topic on tech@.  If you see a need to
continue the thread, please move it to misc@.

Samuel Marks wrote on Sun, Sep 05, 2021 at 02:10:14PM -0400:

> WiP, basically just started this:
> https://github.com/offscale/libacquire/tree/423b751/libacquire/openbsd_ftp

I feel unsure what you are trying to achieve.  The README.md in that
directory is so terse that i don't really understand what the point is.

> What:
> - My project allows for switching between the HTTPS interface provided
> by your OS, and libcurl's HTTPS interface.

I don't really understand what that is supposed to mean either.

> Why OpenBSD `ftp` port:
> - I don't have a "pure" socket solution, and OpenBSD does not have a C
> interface version of their `ftp` (like, `fetch` from FreeBSD does, for
> example)
> - SLOC is short
> - It's from OpenBSD which gives certain trust for protocol conformity,
> safety, security, lower rates of change, and all-in-all quality

None of that applies to our current implementation of the ftp(1) client.
Yes, like everything in OpenBSD, it did receive some auditing, some
maintenance, and some feature additions helping security in the past.

But all the same, our -current ftp(1) client consists of ancient,
terribly ill-designed, technically totally outdated and obsolete
code.  So much so that developers felt it was time to throw it all
away and write a completely new ftp(1) program from scratch.  One
developer started an effort to do so, and that new program can
already do most of what ftp(1) should, but before it was completely
ready for commit, the effort unfortunately somehow petered out, so
we still have the old shit in the tree.

> - Once the MSVC port is done, the port to other OSs should be
> relatively trivial

Porting OpenBSD software to other systems is often a good idea.
Worthy examples include OpenSSH, OpenBGPD, OpenNTPD, OpenSMTPD,
OpenIKED, LibreSSL, and there are likely many more.

But OpenBSD ftp(1) is most definitely not a worthy target for
porting.

Just look at this utter mess:

 $ cd /usr/src/usr.bin/ftp/
 $ grep jmp *.c
cmds.c: (void)setjmp(jabort);
cmds.c: (void)setjmp(jabort);
cmds.c: (void)setjmp(jabort);
cmds.c:jmp_buf abortprox;
cmds.c: longjmp(abortprox, 1);
cmds.c: if (setjmp(abortprox)) {
fetch.c:static jmp_buf  httpabort;
fetch.c:if (setjmp(httpabort)) {
fetch.c:if (setjmp(httpabort)) {
fetch.c:longjmp(httpabort, 1);
fetch.c:if (setjmp(toplevel)) {
ftp.c:jmp_buf   ptabort;
ftp.c:  longjmp(ptabort, 1);
ftp.c:jmp_buf   sendabort;
ftp.c:  longjmp(sendabort, 1);
ftp.c:  if (setjmp(sendabort)) {
ftp.c:  if (setjmp(sendabort))
ftp.c:jmp_buf   recvabort;
ftp.c:  longjmp(recvabort, 1);
ftp.c:  if (setjmp(recvabort)) {
ftp.c:  if (setjmp(recvabort))
ftp.c:  longjmp(ptabort, 1);
ftp.c:  if (setjmp(ptabort))
ftp.c:jmp_buf forceabort;
ftp.c:  longjmp(forceabort, 1);
ftp.c:  if (cout == NULL || setjmp(forceabort)) {
main.c:jmp_buf  toplevel;
main.c: if (setjmp(toplevel))
main.c: top = setjmp(toplevel) == 0;
main.c: longjmp(toplevel, 1);
small.c:jmp_buf jabort;
small.c:longjmp(jabort, 1);
small.c:longjmp(jabort, 1);
small.c:(void)setjmp(jabort);

Or look at this in small.c:

/* XXX - Signal race. */
/* ARGSUSED */
void
mabort(int signo)
{
int save_errno = errno;

alarmtimer(0);
(void) write(fileno(ttyout), "\n\r", 2);
#ifndef SMALL
if (mflag && fromatty) {
/* XXX signal race, crazy unbelievable stdio misuse */
if (confirm(mname, NULL)) {
errno = save_errno;
longjmp(jabort, 1);
}
}
#endif /* !SMALL */
mflag = 0;
errno = save_errno;
longjmp(jabort, 1);
}

Then look at util.c what confirm() is.  The word "unbelievable" is an
accurate description of the situation.

Or look at intr() in main.c.  On top of calling longjmp(3), that
signal handler also calls alarmtimer() in util.c, which happily
does multiple non-atomic stores into a struct, then calls the
function setitimer(2) which is not async-signal-safe either.

Or the updateprogressmeter() signal handler in util.c.  That one
calls progressmeter() in the same file, which is just as bad as
confirm().

We have a policy to mark broken code with /* XXX */ in the tree when
it is found and whoever discovers the breakage has no time to fix
it right away.  But ftp(1) is so bad that i feel like i don't even
have the time to mark stuff as broken.  No matter where i look,
almost all of it is broken.

I'm usually a great fan of fixing broken code by step-by-step
refactoring, even if it is broken in quite serious ways.  But that
is totally out of the question for OpenBSD ftp(1).  That code is
so thoroughly ill-designed throughout that starting over from scratch
is the only viable option.

> So, want me to finish working on this?

It sounds like a waste of time to me.

> - And 

Re: riscv64: icache flush using sysarch(2)

[Jeremie Courreges-Anglas]

On Sat, Sep 04 2021, Jeremie Courreges-Anglas  wrote:
> The first problem I was able to diagnose using egdb on riscv was
> lang/python/2.7 using libffi and aborting in libcompiler-rt (the
> compilerrt_abort() call below).
>
> --8<--
> #elif defined(__riscv) && defined(__linux__)
> #define __NR_riscv_flush_icache (244 + 15)
>   register void *start_reg __asm("a0") = start;
>   const register void *end_reg __asm("a1") = end;
>   const register long flags __asm("a2") = 0;
>   const register long syscall_nr __asm("a7") = __NR_riscv_flush_icache;
>   __asm __volatile("ecall"
>: "=r"(start_reg)
>: "r"(start_reg), "r"(end_reg), "r"(flags), 
> "r"(syscall_nr));
>   assert(start_reg == 0 && "Cache flush syscall failed.");
> #else
> #if __APPLE__
>   // On Darwin, sys_icache_invalidate() provides this functionality
>   sys_icache_invalidate(start, end - start);
> #else
>   compilerrt_abort();
> #endif
> #endif
> }
> -->8--
>
> The ususal way we provide this functionality is through sysarch(2).
> Since the RISC-V ISA only provides fence.i as an extension, and that
> fence.i doesn't support parameters to only act on a specific range,
> I figured I would reflect that in the API for the sake of clarity.
>
> If people expect the spec to evolve and new CPUs to ship with
> support for finer-grained invalidation, a more forward-looking approach
> would be to mimic ARM_SYNC_ICACHE and struct arm_sync_icache_args, and
> let the kernel ignore the parameters if appropriate.
>
> In the diff below I'm moving the core of the code to cpu.c since it
> doesn't look pmap-specific, but I don't feel strongly about it.
> I haven't even built this since I'm still on the way back from k2k21 but
> I figured I'd ask for feedback early.  Input welcome.

Updated diff that builds (one #include was missing) and that appears to
do the right thing.  The diff also includes the compiler-rt change.


Index: gnu/llvm/compiler-rt/lib/builtins/clear_cache.c
===
RCS file: /cvs/src/gnu/llvm/compiler-rt/lib/builtins/clear_cache.c,v
retrieving revision 1.3
diff -u -p -r1.3 clear_cache.c
--- gnu/llvm/compiler-rt/lib/builtins/clear_cache.c 2 Jan 2021 17:14:13 
-   1.3
+++ gnu/llvm/compiler-rt/lib/builtins/clear_cache.c 19 Aug 2021 17:28:40 
-
@@ -33,7 +33,7 @@ uintptr_t GetCurrentProcess(void);
 #include 
 #endif
 
-#if defined(__OpenBSD__) && (defined(__arm__) || defined(__mips__))
+#if defined(__OpenBSD__) && (defined(__arm__) || defined(__mips__) || 
defined(__riscv))
 // clang-format off
 #include 
 #include 
@@ -157,6 +157,8 @@ void __clear_cache(void *start, void *en
: "=r"(start_reg)
: "r"(start_reg), "r"(end_reg), "r"(flags), 
"r"(syscall_nr));
   assert(start_reg == 0 && "Cache flush syscall failed.");
+#elif defined(__riscv) && defined(__OpenBSD__)
+   sysarch(RISCV_SYNC_ICACHE_ALL, NULL);
 #else
 #if __APPLE__
   // On Darwin, sys_icache_invalidate() provides this functionality
Index: sys/arch/riscv64/include/cpufunc.h
===
RCS file: /cvs/src/sys/arch/riscv64/include/cpufunc.h,v
retrieving revision 1.4
diff -u -p -r1.4 cpufunc.h
--- sys/arch/riscv64/include/cpufunc.h  18 May 2021 09:14:49 -  1.4
+++ sys/arch/riscv64/include/cpufunc.h  18 Aug 2021 23:24:59 -
@@ -92,6 +92,7 @@ extern int64_t icache_line_size;
 extern void (*cpu_dcache_wbinv_range)(paddr_t, psize_t);
 extern void (*cpu_dcache_inv_range)(paddr_t, psize_t);
 extern void (*cpu_dcache_wb_range)(paddr_t, psize_t);
+extern void icache_flush(void);
 
 static __inline void
 load_satp(uint64_t val)
Index: sys/arch/riscv64/include/sysarch.h
===
RCS file: sys/arch/riscv64/include/sysarch.h
diff -N sys/arch/riscv64/include/sysarch.h
--- /dev/null   1 Jan 1970 00:00:00 -
+++ sys/arch/riscv64/include/sysarch.h  18 Aug 2021 23:24:59 -
@@ -0,0 +1,38 @@
+/* $OpenBSD$   */
+
+/*
+ * Copyright (c) 2021 Jeremie Courreges-Anglas 
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#ifndef_RISCV64_SYSARCH_H_
+#define_RISCV64_SYSARCH_H_
+
+/*
+ * Architecture specific syscalls (riscv64)
+ 

OpenBSD `ftp` port for MSVC (Windows)

[Samuel Marks]

WiP, basically just started this:
https://github.com/offscale/libacquire/tree/423b751/libacquire/openbsd_ftp

What:
- My project allows for switching between the HTTPS interface provided
by your OS, and libcurl's HTTPS interface.

Why OpenBSD `ftp` port:
- I don't have a "pure" socket solution, and OpenBSD does not have a C
interface version of their `ftp` (like, `fetch` from FreeBSD does, for
example)
- SLOC is short
- It's from OpenBSD which gives certain trust for protocol conformity,
safety, security, lower rates of change, and all-in-all quality
- Once the MSVC port is done, the port to other OSs should be relatively trivial

So, want me to finish working on this? - And if so, once it's ready,
want me to offer it back to you?

Samuel Marks
https://github.com/SamuelMarks

dwctwo(4) isoc

[Marcus Glocker]

In case you are playing around with isoc transfers (uvideo, uaudio) on
dwctwo(4), and you are facing kernel crashes, you can try this diff.
Or if you just want to regression test it.  It basically does:

* Fix DMA allocation out of interrupt context.
* Fix a memory leak on DMA allocation during pipe closing/opening.
* Fix a kernel crash for low/full speed devices which use split in
  transfers (often seen for uaudio devices).


Index: dev/usb/dwc2/dwc2.c
===
RCS file: /cvs/src/sys/dev/usb/dwc2/dwc2.c,v
retrieving revision 1.58
diff -u -p -u -p -r1.58 dwc2.c
--- dev/usb/dwc2/dwc2.c 30 Jul 2021 18:56:01 -  1.58
+++ dev/usb/dwc2/dwc2.c 4 Sep 2021 10:22:16 -
@@ -455,7 +455,10 @@ dwc2_poll(struct usbd_bus *bus)
 STATIC void
 dwc2_close_pipe(struct usbd_pipe *pipe)
 {
-   /* nothing */
+   struct dwc2_pipe *dpipe = DWC2_PIPE2DPIPE(pipe);
+   struct dwc2_softc *sc = DWC2_DPIPE2SC(dpipe);
+
+   dwc2_hcd_endpoint_disable(sc->sc_hsotg, dpipe, 250);
 }
 
 /*
Index: dev/usb/dwc2/dwc2_core.h
===
RCS file: /cvs/src/sys/dev/usb/dwc2/dwc2_core.h,v
retrieving revision 1.11
diff -u -p -u -p -r1.11 dwc2_core.h
--- dev/usb/dwc2/dwc2_core.h27 Jul 2021 13:36:59 -  1.11
+++ dev/usb/dwc2/dwc2_core.h4 Sep 2021 10:22:17 -
@@ -556,6 +556,19 @@ struct dwc2_hregs_backup {
 };
 
 /**
+ * struct dwc2_usbdma - Holds the pre-allocated USB DMA memory slots
+ *
+ * @use:   -1 = usb dma memory slot un-initialized
+ *  0 = usb dma memory slot initialized but un-used
+ *  1 = usb dma memory slot initialized and used
+ * @dma:usb_dma memory which gets initialized by usb_allocmem()
+ */
+struct dwc2_usbdma {
+   int use;
+   struct usb_dma dma;
+};
+
+/**
  * struct dwc2_hsotg - Holds the state of the driver, including the 
non-periodic
  * and periodic schedules
  *
@@ -804,6 +817,10 @@ struct dwc2_hsotg {
u32 frame_list_sz;
struct kmem_cache *desc_gen_cache;
struct kmem_cache *desc_hsisoc_cache;
+
+#define DWC2_USBDMA_SLOTS 8
+   struct dwc2_usbdma usbdma_norm[DWC2_USBDMA_SLOTS];
+   struct dwc2_usbdma usbdma_isoc[DWC2_USBDMA_SLOTS];
 
 #ifdef DEBUG
u32 frrem_samples;
Index: dev/usb/dwc2/dwc2_hcd.c
===
RCS file: /cvs/src/sys/dev/usb/dwc2/dwc2_hcd.c,v
retrieving revision 1.25
diff -u -p -u -p -r1.25 dwc2_hcd.c
--- dev/usb/dwc2/dwc2_hcd.c 4 Sep 2021 10:19:28 -   1.25
+++ dev/usb/dwc2/dwc2_hcd.c 4 Sep 2021 10:22:17 -
@@ -537,6 +537,64 @@ dwc2_hcd_urb_dequeue(struct dwc2_hsotg *
return 0;
 }
 
+/* Must NOT be called with interrupt disabled or spinlock held */
+int dwc2_hcd_endpoint_disable(struct dwc2_hsotg *hsotg,
+ struct dwc2_pipe *dpipe, int retry)
+{
+   struct dwc2_qtd *qtd, *qtd_tmp;
+   struct dwc2_qh *qh;
+   unsigned long flags;
+   int rc;
+
+   spin_lock_irqsave(>lock, flags);
+
+   qh = dpipe->priv;
+   if (!qh) {
+   rc = -EINVAL;
+   goto err;
+   }
+
+   while (!list_empty(>qtd_list) && retry--) {
+   if (retry == 0) {
+   dev_err(hsotg->dev,
+   "## timeout in dwc2_hcd_endpoint_disable() 
##\n");
+   rc = -EBUSY;
+   goto err;
+   }
+
+   spin_unlock_irqrestore(>lock, flags);
+   usleep_range(20, 25);
+   spin_lock_irqsave(>lock, flags);
+   qh = dpipe->priv;
+   if (!qh) {
+   rc = -EINVAL;
+   goto err;
+   }
+}
+
+   dwc2_hcd_qh_unlink(hsotg, qh);
+
+   /* Free each QTD in the QH's QTD list */
+   list_for_each_entry_safe(qtd, qtd_tmp, >qtd_list, qtd_list_entry)
+   dwc2_hcd_qtd_unlink_and_free(hsotg, qtd, qh);
+
+   dpipe->priv = NULL;
+
+   if (qh->channel && qh->channel->qh == qh)
+   qh->channel->qh = NULL;
+
+   spin_unlock_irqrestore(>lock, flags);
+
+   dwc2_hcd_qh_free(hsotg, qh);
+
+   return 0;
+
+err:
+   dpipe->priv = NULL;
+   spin_unlock_irqrestore(>lock, flags);
+
+   return rc;
+}
 
 /*
  * Initializes dynamic portions of the DWC_otg HCD state
@@ -720,20 +778,12 @@ STATIC int dwc2_hc_setup_align_buf(struc
u32 buf_size;
 
if (!qh->dw_align_buf) {
-   int err;
-
-   if (chan->ep_type != USB_ENDPOINT_XFER_ISOC)
-   buf_size = hsotg->core_params->max_transfer_size;
-   else
-   /* 3072 = 3 max-size Isoc packets */
-   buf_size = 3072;
-
qh->dw_align_buf = NULL;
qh->dw_align_buf_dma = 0;
-   err = usb_allocmem(>hsotg_sc->sc_bus, 

Re: Update to pcap-filter.5/tcpdump.8 (was: update to tcpdump(8))

[Jason McIntyre]

On Sun, Sep 05, 2021 at 04:43:34PM +0200, Denis Fondras wrote:
> Le Sat, Sep 04, 2021 at 09:57:10PM +0100, Jason McIntyre a ?crit :
> > the diff looks ok to me. but run any doc changes through "mandoc
> > -Tlint", and look at any issues your diff may have introduced. in this
> > case it's just trailing whitespace, but it's super helpful to check your
> > work.
> > 
> 
> Thank you Jason. There is still a warning in tcpdump.8.
> 
> Here is a new version including changes to pcap-filter.5 and tcpdump.8
> I did not change the examples though as tcpdump examples are broader than
> filters.
> 

hi.

the warning in tcpdump is fine.

the diff reads ok to me, but let's wait for a technical ok ;)

jmc

> Index: lib/libpcap/pcap-filter.5
> ===
> RCS file: /cvs/src/lib/libpcap/pcap-filter.5,v
> retrieving revision 1.9
> diff -u -p -r1.9 pcap-filter.5
> --- lib/libpcap/pcap-filter.5 2 Sep 2021 10:59:13 -   1.9
> +++ lib/libpcap/pcap-filter.5 5 Sep 2021 13:35:41 -
> @@ -40,27 +40,31 @@ or
>  .Pp
>  The filter expression consists of one or more
>  .Em primitives .
> -Primitives usually consist of an ID (name or number)
> +Primitives usually consist of an
> +.Ar id
> +.Pq name or number
>  preceded by one or more qualifiers.
>  There are three different kinds of qualifier:
>  .Bl -tag -width "proto"
> -.It type
> -Type qualifiers say what kind of thing the ID name or number refers to.
> +.It Ar type
> +Specify which kind of address component the
> +.Ar id
> +name or number refers to.
>  Possible types are
>  .Cm host ,
> -.Cm net ,
> +.Cm net
>  and
>  .Cm port .
> -For example,
> +E.g.,
>  .Dq host foo ,
>  .Dq net 128.3 ,
> -and
>  .Dq port 20 .
>  If there is no type qualifier,
>  .Cm host
>  is assumed.
> -.It dir
> -Dir qualifiers specify a particular transfer direction to and/or from an ID.
> +.It Ar dir
> +Specify a particular transfer direction to and/or from
> +.Ar id .
>  Possible directions are
>  .Cm src ,
>  .Cm dst ,
> @@ -73,11 +77,13 @@ Possible directions are
>  .Cm addr3 ,
>  and
>  .Cm addr4 .
> -For example,
> -.Cm src foo ,
> -.Cm dst net 128.3 ,
> -.Cm src or dst port ftp-data .
> -If there is no dir qualifier,
> +E.g.,
> +.Dq src foo ,
> +.Dq dst net 128.3 ,
> +.Dq src or dst port ftp-data .
> +If there is no
> +.Ar dir
> +qualifier,
>  .Cm src or dst
>  is assumed.
>  The
> @@ -89,57 +95,85 @@ The
>  and
>  .Cm addr4
>  qualifiers are only valid for IEEE 802.11 Wireless LAN link layers.
> -For some link layers, such as SLIP and the "cooked" Linux capture mode
> -used for the "any" device and for some other device types, the
> +For null link layers (i.e., point-to-point protocols such as SLIP
> +.Pq Serial Line Internet Protocol
> +or the
> +.Xr pflog 4
> +header), the
>  .Cm inbound
>  and
>  .Cm outbound
>  qualifiers can be used to specify a desired direction.
> -.It proto
> -Proto qualifiers restrict the match to a particular protocol.
> -Possible
> -protos are:
> +.It Ar proto
> +Restrict the match to a particular protocol.
> +Possible protocols are:
> +.Cm ah ,
> +.Cm arp ,
> +.Cm atalk ,
> +.Cm decnet ,
> +.Cm esp ,
>  .Cm ether ,
>  .Cm fddi ,
> -.Cm tr ,
> -.Cm wlan ,
> +.Cm icmp ,
> +.Cm icmp6 ,
> +.Cm igmp ,
> +.Cm igrp ,
>  .Cm ip ,
>  .Cm ip6 ,
> -.Cm arp ,
> +.Cm lat ,
> +.Cm mopdl ,
> +.Cm moprc ,
> +.Cm pim ,
>  .Cm rarp ,
> -.Cm decnet ,
> +.Cm sca ,
> +.Cm stp ,
>  .Cm tcp ,
> +.Cm udp ,
>  and
> -.Cm udp .
> -For example,
> +.Cm wlan .
> +E.g.,
>  .Dq ether src foo ,
>  .Dq arp net 128.3 ,
>  .Dq tcp port 21 ,
>  and
>  .Dq wlan addr2 0:2:3:4:5:6 .
> -If there is no proto qualifier,
> +If there is no protocol qualifier,
>  all protocols consistent with the type are assumed.
> -For example,
> +E.g.,
>  .Dq src foo
>  means
> -.Dq (ip or arp or rarp) src foo
> -(except the latter is not legal syntax);
> +.Do
> +.Pq ip or arp or rarp
> +src foo
> +.Dc
> +.Pq except the latter is not legal syntax ;
>  .Dq net bar
>  means
> -.Dq (ip or arp or rarp) net bar ;
> +.Do
> +.Pq ip or arp or rarp
> +net bar
> +.Dc ;
>  and
>  .Dq port 53
>  means
> -.Dq (tcp or udp) port 53 .
> +.Do
> +.Pq TCP or UDP
> +port 53
> +.Dc .
>  .Pp
>  .Cm fddi
>  is actually an alias for
>  .Cm ether ;
>  the parser treats them identically as meaning
> -"the data link level used on the specified network interface".
> -FDDI headers contain Ethernet-like source and destination addresses,
> +.Qo
> +the data link level used on the specified network interface
> +.Qc .
> +FDDI
> +.Pq Fiber Distributed Data Interface
> +headers contain Ethernet-like source and destination addresses,
>  and often contain Ethernet-like packet types,
> -so it's possible to filter these FDDI fields just as with the analogous 
> Ethernet fields.
> +so it's possible to filter these FDDI fields just as with the analogous
> +Ethernet fields.
>  FDDI headers also contain other fields,
>  but they cannot be named explicitly in a filter expression.
>  .Pp
> @@ -156,8 

Update to pcap-filter.5/tcpdump.8 (was: update to tcpdump(8))

[Denis Fondras]

Le Sat, Sep 04, 2021 at 09:57:10PM +0100, Jason McIntyre a écrit :
> the diff looks ok to me. but run any doc changes through "mandoc
> -Tlint", and look at any issues your diff may have introduced. in this
> case it's just trailing whitespace, but it's super helpful to check your
> work.
> 

Thank you Jason. There is still a warning in tcpdump.8.

Here is a new version including changes to pcap-filter.5 and tcpdump.8
I did not change the examples though as tcpdump examples are broader than
filters.

Index: lib/libpcap/pcap-filter.5
===
RCS file: /cvs/src/lib/libpcap/pcap-filter.5,v
retrieving revision 1.9
diff -u -p -r1.9 pcap-filter.5
--- lib/libpcap/pcap-filter.5   2 Sep 2021 10:59:13 -   1.9
+++ lib/libpcap/pcap-filter.5   5 Sep 2021 13:35:41 -
@@ -40,27 +40,31 @@ or
 .Pp
 The filter expression consists of one or more
 .Em primitives .
-Primitives usually consist of an ID (name or number)
+Primitives usually consist of an
+.Ar id
+.Pq name or number
 preceded by one or more qualifiers.
 There are three different kinds of qualifier:
 .Bl -tag -width "proto"
-.It type
-Type qualifiers say what kind of thing the ID name or number refers to.
+.It Ar type
+Specify which kind of address component the
+.Ar id
+name or number refers to.
 Possible types are
 .Cm host ,
-.Cm net ,
+.Cm net
 and
 .Cm port .
-For example,
+E.g.,
 .Dq host foo ,
 .Dq net 128.3 ,
-and
 .Dq port 20 .
 If there is no type qualifier,
 .Cm host
 is assumed.
-.It dir
-Dir qualifiers specify a particular transfer direction to and/or from an ID.
+.It Ar dir
+Specify a particular transfer direction to and/or from
+.Ar id .
 Possible directions are
 .Cm src ,
 .Cm dst ,
@@ -73,11 +77,13 @@ Possible directions are
 .Cm addr3 ,
 and
 .Cm addr4 .
-For example,
-.Cm src foo ,
-.Cm dst net 128.3 ,
-.Cm src or dst port ftp-data .
-If there is no dir qualifier,
+E.g.,
+.Dq src foo ,
+.Dq dst net 128.3 ,
+.Dq src or dst port ftp-data .
+If there is no
+.Ar dir
+qualifier,
 .Cm src or dst
 is assumed.
 The
@@ -89,57 +95,85 @@ The
 and
 .Cm addr4
 qualifiers are only valid for IEEE 802.11 Wireless LAN link layers.
-For some link layers, such as SLIP and the "cooked" Linux capture mode
-used for the "any" device and for some other device types, the
+For null link layers (i.e., point-to-point protocols such as SLIP
+.Pq Serial Line Internet Protocol
+or the
+.Xr pflog 4
+header), the
 .Cm inbound
 and
 .Cm outbound
 qualifiers can be used to specify a desired direction.
-.It proto
-Proto qualifiers restrict the match to a particular protocol.
-Possible
-protos are:
+.It Ar proto
+Restrict the match to a particular protocol.
+Possible protocols are:
+.Cm ah ,
+.Cm arp ,
+.Cm atalk ,
+.Cm decnet ,
+.Cm esp ,
 .Cm ether ,
 .Cm fddi ,
-.Cm tr ,
-.Cm wlan ,
+.Cm icmp ,
+.Cm icmp6 ,
+.Cm igmp ,
+.Cm igrp ,
 .Cm ip ,
 .Cm ip6 ,
-.Cm arp ,
+.Cm lat ,
+.Cm mopdl ,
+.Cm moprc ,
+.Cm pim ,
 .Cm rarp ,
-.Cm decnet ,
+.Cm sca ,
+.Cm stp ,
 .Cm tcp ,
+.Cm udp ,
 and
-.Cm udp .
-For example,
+.Cm wlan .
+E.g.,
 .Dq ether src foo ,
 .Dq arp net 128.3 ,
 .Dq tcp port 21 ,
 and
 .Dq wlan addr2 0:2:3:4:5:6 .
-If there is no proto qualifier,
+If there is no protocol qualifier,
 all protocols consistent with the type are assumed.
-For example,
+E.g.,
 .Dq src foo
 means
-.Dq (ip or arp or rarp) src foo
-(except the latter is not legal syntax);
+.Do
+.Pq ip or arp or rarp
+src foo
+.Dc
+.Pq except the latter is not legal syntax ;
 .Dq net bar
 means
-.Dq (ip or arp or rarp) net bar ;
+.Do
+.Pq ip or arp or rarp
+net bar
+.Dc ;
 and
 .Dq port 53
 means
-.Dq (tcp or udp) port 53 .
+.Do
+.Pq TCP or UDP
+port 53
+.Dc .
 .Pp
 .Cm fddi
 is actually an alias for
 .Cm ether ;
 the parser treats them identically as meaning
-"the data link level used on the specified network interface".
-FDDI headers contain Ethernet-like source and destination addresses,
+.Qo
+the data link level used on the specified network interface
+.Qc .
+FDDI
+.Pq Fiber Distributed Data Interface
+headers contain Ethernet-like source and destination addresses,
 and often contain Ethernet-like packet types,
-so it's possible to filter these FDDI fields just as with the analogous 
Ethernet fields.
+so it's possible to filter these FDDI fields just as with the analogous
+Ethernet fields.
 FDDI headers also contain other fields,
 but they cannot be named explicitly in a filter expression.
 .Pp
@@ -156,8 +190,8 @@ and the source address is the SA field;
 the BSSID, RA, and TA fields aren't tested.
 .El
 .Pp
-In addition to the above,
-there are some special primitives that don't follow the pattern:
+In addition to the above, there are some special primitive
+keywords that don't follow the pattern:
 .Cm gateway ,
 .Cm broadcast ,
 .Cm less ,
@@ -170,14 +204,18 @@ More complex filter expressions are buil
 .Cm or ,
 and
 .Cm not
-to combine primitives.
-For example,
-.Dq host foo and not port ftp and not port ftp-data .
-To save typing, 

Re: man sed(1) diff

[Andreas Kusalananda Kähäri]

On Sun, Sep 05, 2021 at 02:01:37PM +0200, Ingo Schwarze wrote:
> Hello,
> 
> i see where Ian's confusion is coming from, even though arguably,
> the existing text is accurate.  But it is not a good idea to insert
> exceptions as parenthetic remarks in the middle of an enumeration
> of steps that is already somewhat long and complicated.
> 
> I think it is better to explain the special rules for D processing
> in the paragraph describing D (surprise, surprise) rather than in
> the middle of the general description of what a "cycle" is.
> 
> 
> Consider the following minimal example:
> 
>$ printf 'axb' | sed -n 'y/x/\n/;s/^b/c/;P;D'
>a
>c
> 
> The D deletes "a\n" from the pattern space.
> After that, the next cycle is entered with "b" in the pattern space,
> without copying a new line into the pattern space.
> 
> 
> OK?
>   Ingo
> 
> 
> Index: sed.1
> ===
> RCS file: /cvs/src/usr.bin/sed/sed.1,v
> retrieving revision 1.48
> diff -u -r1.48 sed.1
> --- sed.1 17 Mar 2016 05:27:10 -  1.48
> +++ sed.1 5 Sep 2021 11:46:48 -
> @@ -140,9 +140,6 @@
>  cyclically copies a line of input, not including its terminating newline
>  character, into a
>  .Em pattern space ,
> -(unless there is something left after a
> -.Ic D
> -function),
>  applies all of the commands with addresses that select that pattern space,
>  copies the pattern space to the standard output, appending a newline, and
>  deletes the pattern space.
> @@ -331,7 +328,8 @@
>  Delete the pattern space and start the next cycle.
>  .It [2addr] Ns Ic D
>  Delete the initial segment of the pattern space through the first
> -newline character and start the next cycle.
> +newline character and start the next cycle without copying the next
> +line of input into the pattern space.
>  .It [2addr] Ns Ic g
>  Replace the contents of the pattern space with the contents of the
>  hold space.


This is an improvement.  This should be compared to the POSIX text:

If the pattern space contains no , delete the pattern
space and start a normal new cycle as if the d command was
issued.  Otherwise, delete the initial segment of the pattern
space through the first , and start the next cycle with
the resultant pattern space and without reading any new input.

The difference is that the POSIX text explicitly says that "D" should
act like "d" if there is no newline character in the pattern space.  The
OpenBSD text does not actually say what happens if there is no newline
character in the pattern space (I believe that OpenBSD sed behaves as
POSIX describes).

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: man sed(1) diff

[Andreas Kusalananda Kähäri]

On Sun, Sep 05, 2021 at 09:51:44AM +0100, ropers wrote:
> I.
> 
> Not to engage in pointless bikeshedding, but I find this clearer and
> --if I understand things correctly-- also more technically accurate:
> 
> Index: sed.1
> ===
> RCS file: /cvs/src/usr.bin/sed/sed.1,v
> retrieving revision 1.60
> diff -C8 -u -r1.60 sed.1
> cvs server: conflicting specifications of output style
> --- sed.1 8 Mar 2021 02:47:28 -   1.60
> +++ sed.1 5 Sep 2021 08:23:25 -
> @@ -141,19 +141,19 @@
>  Normally,
>  .Nm
>  cyclically copies a line of input, not including its terminating newline
>  character, into a
>  .Em pattern space ,
>  (unless there is something left after a
>  .Ic D
>  function),
> -applies all of the commands with addresses that select that pattern space,
> -copies the pattern space to the standard output, appending a newline, and
> -deletes the pattern space.
> +applies all of the commands with addresses that select that pattern,

An address does not select a pattern, it selects the pattern space.

> +copies the pattern space contents to the standard output, appending a

The word "contents" is not needed.

> +newline, and deletes them from the pattern space.

It's unclear what "them" refers to here (possibly "the contents"?).

The proposed text seems to want to separate "the pattern space" from
"the contents of the pattern space".  I don't think this distinction
is helpful, just like it's not helpful ta make a distinction between
"a string" and "the contents of a string".  The manual (and the POSIX
standard text) refers to the data that is read from the input and
currently subject to the commands of the editing script as "the pattern
space" throughout, not just in this introductory paragraph.


>  .Pp
>  Some of the functions use a
>  .Em hold space
>  to save all or part of the pattern space for subsequent retrieval.
>  .Sh SED ADDRESSES
>  An address is not required, but if specified must be a number (that counts
>  input lines
>  cumulatively across input files), a dollar character
> 
> (I used the diff -C 8 option to show a little more context.)
> 
> 
> II.
> 
> [Link for easier reading: ]
> 
> Also, does the "(unless there is something left after a D function)"
> part really relate to the preceding parenthetical clause of "not
> including its terminating newline character"?  Should it be moved to
> directly follow that instead of following the "into a pattern space"
> part?
> Alternatively, would it be better to move the "(...)" part to a
> separate subsequent sentence like this:
> > (A newline character may be present in the pattern space
> > if left behind after a
> > .Ic D
> > function.)
> Is it even important to include that information in the man page?  Is
> it ever relevant that there may technically be some string and a
> newline left in the pattern space?
> 
> 
> Thank you,
> Ian

-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: man sed(1) diff

[Ingo Schwarze]

Hello,

i see where Ian's confusion is coming from, even though arguably,
the existing text is accurate.  But it is not a good idea to insert
exceptions as parenthetic remarks in the middle of an enumeration
of steps that is already somewhat long and complicated.

I think it is better to explain the special rules for D processing
in the paragraph describing D (surprise, surprise) rather than in
the middle of the general description of what a "cycle" is.


Consider the following minimal example:

   $ printf 'axb' | sed -n 'y/x/\n/;s/^b/c/;P;D'
   a
   c

The D deletes "a\n" from the pattern space.
After that, the next cycle is entered with "b" in the pattern space,
without copying a new line into the pattern space.


OK?
  Ingo


Index: sed.1
===
RCS file: /cvs/src/usr.bin/sed/sed.1,v
retrieving revision 1.48
diff -u -r1.48 sed.1
--- sed.1   17 Mar 2016 05:27:10 -  1.48
+++ sed.1   5 Sep 2021 11:46:48 -
@@ -140,9 +140,6 @@
 cyclically copies a line of input, not including its terminating newline
 character, into a
 .Em pattern space ,
-(unless there is something left after a
-.Ic D
-function),
 applies all of the commands with addresses that select that pattern space,
 copies the pattern space to the standard output, appending a newline, and
 deletes the pattern space.
@@ -331,7 +328,8 @@
 Delete the pattern space and start the next cycle.
 .It [2addr] Ns Ic D
 Delete the initial segment of the pattern space through the first
-newline character and start the next cycle.
+newline character and start the next cycle without copying the next
+line of input into the pattern space.
 .It [2addr] Ns Ic g
 Replace the contents of the pattern space with the contents of the
 hold space.

snaps: scp uses SFTP protocol by default

[Damien Miller]

Hi,

Just letting you know that the snaps rolling out now have scp defaulting
to use the SFTP protocol by default. We hope to keep this change in the
next release, so please report any problems you encounter either here
(tech@), to bugs@ or to openssh@.

One thing to be aware of: copying to/from a remote path that includes a
~user/ prefix requires a SFTP protocol extension that is only available
in OpenSSH 8.7 and later. If you need this, and upgrading the server is
not an option, then you should force the use of the scp/rcp protocol
using the -O flag.

Thanks,
Damien

Re: Fix: tcp_output window calculation error

[Martin Pieuchot]

On 22/07/21(Thu) 15:03, Jan Klemkow wrote:
> Hi,
> 
> This calculation of the receive window has a logic error:
> 
> If win is 0 it will be overwritten by (rcv_adv - rcv_nxt).  Thus, win
> will be (rcv_adv - rcv_nxt) even if its below (sb_hiwat / 4).

Why is this a problem?

> We could just remove the dead (sb_hiwat / 4) code, or reorder the
> conditions to keep the original feature.
> 
> OK?
> 
> bye,
> Jan
> 
> Index: netinet/tcp_output.c
> ===
> RCS file: /cvs/src/sys/netinet/tcp_output.c,v
> retrieving revision 1.130
> diff -u -p -r1.130 tcp_output.c
> --- netinet/tcp_output.c  8 Feb 2021 19:37:15 -   1.130
> +++ netinet/tcp_output.c  22 Jul 2021 12:33:13 -
> @@ -812,12 +812,12 @@ send:
>* Calculate receive window.  Don't shrink window,
>* but avoid silly window syndrome.
>*/
> - if (win < (long)(so->so_rcv.sb_hiwat / 4) && win < (long)tp->t_maxseg)
> - win = 0;
>   if (win > (long)TCP_MAXWIN << tp->rcv_scale)
>   win = (long)TCP_MAXWIN << tp->rcv_scale;
>   if (win < (long)(int32_t)(tp->rcv_adv - tp->rcv_nxt))
>   win = (long)(int32_t)(tp->rcv_adv - tp->rcv_nxt);
> + if (win < (long)(so->so_rcv.sb_hiwat / 4) && win < (long)tp->t_maxseg)
> + win = 0;
>   if (flags & TH_RST)
>   win = 0;
>   th->th_win = htons((u_int16_t) (win>>tp->rcv_scale));
> 

man sed(1) diff

[ropers]

I.

Not to engage in pointless bikeshedding, but I find this clearer and
--if I understand things correctly-- also more technically accurate:

Index: sed.1
===
RCS file: /cvs/src/usr.bin/sed/sed.1,v
retrieving revision 1.60
diff -C8 -u -r1.60 sed.1
cvs server: conflicting specifications of output style
--- sed.1   8 Mar 2021 02:47:28 -   1.60
+++ sed.1   5 Sep 2021 08:23:25 -
@@ -141,19 +141,19 @@
 Normally,
 .Nm
 cyclically copies a line of input, not including its terminating newline
 character, into a
 .Em pattern space ,
 (unless there is something left after a
 .Ic D
 function),
-applies all of the commands with addresses that select that pattern space,
-copies the pattern space to the standard output, appending a newline, and
-deletes the pattern space.
+applies all of the commands with addresses that select that pattern,
+copies the pattern space contents to the standard output, appending a
+newline, and deletes them from the pattern space.
 .Pp
 Some of the functions use a
 .Em hold space
 to save all or part of the pattern space for subsequent retrieval.
 .Sh SED ADDRESSES
 An address is not required, but if specified must be a number (that counts
 input lines
 cumulatively across input files), a dollar character

(I used the diff -C 8 option to show a little more context.)


II.

[Link for easier reading: ]

Also, does the "(unless there is something left after a D function)"
part really relate to the preceding parenthetical clause of "not
including its terminating newline character"?  Should it be moved to
directly follow that instead of following the "into a pattern space"
part?
Alternatively, would it be better to move the "(...)" part to a
separate subsequent sentence like this:
> (A newline character may be present in the pattern space
> if left behind after a
> .Ic D
> function.)
Is it even important to include that information in the man page?  Is
it ever relevant that there may technically be some string and a
newline left in the pattern space?


Thank you,
Ian